Emotional Experience and the Senses

1 Sensory Experience

This paper defends the claim that emotional experience is, at least in part, non-sensory. My claim is, on the face of it, reasonably weak and rather plausible. Given that emotion is not the same thing as sensation, it is natural to presume that emotional experience will also not be the same thing as sensory experience. Yet the opposite thesis – namely the thesis that all emotional experience is sensory – is very widespread. It is the downstream specification of a much more general and ambitious view in the philosophy of consciousness, which is currently accepted by most philosophers of mind and cognitive scientists as the standard view, and which purports to restrict all conscious experience within the relatively narrow domain of the sensory.¹ I will call this general view ‘restrictivism’.

According to restrictivism, all conscious experience is sensory, including, of course, emotional experience. The arguments I present in §§2 and 3 of this paper aim to show that restrictivism is false of emotional experience. Given the generalist ambitions of the view, this is equivalent to showing that restrictivism is false simpliciter. I will call the denial of restrictivism, for which this paper argues, ‘expansionism’.²

The present section is dedicated to clarifying the terms of the debate between restrictivism and expansionism, with an eye to highlighting the conditions under which the restrictivist thesis is substantive and worthy of attention. I begin by motivating restrictivism as part of a reductive approach to mind’s place in nature. Next, I explain how my defense of expansionism differs from the argumentative strategies recently pursued by the proponents of so-called ‘cognitive phenomenology’. Finally, I draw upon Plato to further articulate the confines of the restrictivist view.

Let us begin to explore the details of the restrictivist position with the aid of an example. Take any non-sensory mental state of yours:³ for instance, the mental state you are in when, at your sister’s wedding, you find yourself thinking that your uncle’s shoes really don’t go with his tie. A thought is a paradigmatic example of a non-sensory mental state, and the restrictivist maintains that, in virtue of this fact, it cannot have any proprietary experiential properties;⁴ insofar as your thought does indeed figure in the stream of your conscious experience of the wedding, this must be in virtue of some sensory experiential property. In other words, taken on its own, a non-sensory mental state can never be experienced; but when it is experienced, it is never on its own. Any consciously experienced non-sensory mental state – the restrictivist insists – would be completely unconscious if taken in isolation. If it is indeed experienced, it is so only indirectly, i.e. in virtue of being contingently associated with some sensory mental state, such as a visual image or an auditory subvocalization. So, if you are consciously thinking that your uncle’s shoes really don’t go with his tie, this thought of yours is experienced by you only insofar as, and in virtue of the fact that, it co-occurs with your conscious visualization of your uncle’s outfit, as well as with the auditory experience of your inner monologue as you tell yourself, probably in English, “Oh my, where did Uncle Oscar get these shoes. What was he thinking? They really don’t go with the tie. I bet Steve noticed it too. Let me go tell him. That’ll give me a nice excuse to catch up with him.”

Restrictivism is a popular view in contemporary philosophy of mind. Peter Carruthers, Eric Lormand, Jesse Prinz, William Robinson, Michael Tye, Bénédicte Veillet, and Briggs Wright have all defended it in recent years, but many more endorse it without argument.⁵ Its popularity is due, among other reasons, to the fact that the majority of current philosophical theories of conscious experience have been developed under two fundamental assumptions: (i) sensory experience is the paradigm of phenomenal consciousness;⁶ (ii) phenomenal consciousness is the sort of consciousness whose understanding poses a threat to the physicalist conception of reality.⁷ Restrictivism is a natural way of strengthening (i), and its endorsement promises great theoretical advantages: if restrictivism is true, i.e. if sensory experience is not only the paradigm of phenomenal consciousness but also the only kind of phenomenal consciousness there is, then a complete theory of sensory experience amounts to a complete theory of phenomenal consciousness. Given (ii), this in turn entails that by producing a physicalist account of sensory experience, one also neutralizes the main threat to a physicalist conception of reality. And defending a physicalist conception of reality is, most certainly, a research project in which many are invested.⁸

Nonetheless, a number of philosophers of mind have recently called restrictivism into question. These philosophers, who aim to do justice to their expansionist intuitions on the nature of consciousness, have focused their efforts around the careful analysis of a paradigmatic kind of cognitive phenomenology, i.e. the conscious experience of occurrent propositional thought, with the intent of bringing out its non-sensory character at its clearest.⁹ As I understand it, however, expansionism is a more general and ambitious approach. In its most interesting version, its explanandum is limited to neither propositional thought nor propositional thought. Let me explain.

First, it is clear that propositional thought is only a subcategory of thought, and there is no reason for expansionism to target only this subcategory. You may think that it has been too long since you last called your mother, but you may also simply think of your mother. As Martha Nussbaum has pointed out, if we want to provide an adequate account of animal and developmental cognition, and thereby of cognition in general, we need to “substitute a broader and more capacious account of cognition for the original emphasis on the grasp of linguistically formulable propositions”.¹⁰ The same requirement naturally extends to any adequate account of cognitive experience.

Second, if thought is understood as something akin to occurrent belief-that, then it is not the only mental attitude introduced by a that-clause whose experiential profile should be of interest to the expansionist; desire, intention, and emotion, insofar as they can be distinguished from sensation, deserve just as much attention.¹¹ In choosing to focus on the non-sensory aspects of emotional attitudes, I aim to illustrate how a broader conception of the expansionist project can produce results.

The debate between restrictivism and expansionism proceeds on the assumption that the distinction between sensory and non-sensory experience makes sense; in other words, that there is at least one principled way of distinguishing the sensory from the non-sensory such that the conscious mental states which have been taken to be paradigmatic instances of one and the other will fall on the right side of the distinction. The truth of this assumption is not obvious, however, and developing an account of sensation which is neither trivial nor so clearly implausible as to render any view relying on it immediately uninteresting is, in fact, notoriously difficult. In the remainder of this section, I argue that there is no way around this difficulty, but I also offer some guidelines on how to overcome the difficulty in a way that is both principled and friendly to the restrictivist project.

The notion of sensory modality derives from Aristotle. In the second book of De Anima, he famously discusses five: sight, hearing, touch, taste, and smell. The idea that there are five senses is still part of our general culture, but among specialists it is today fairly uncontroversial to supplement the Aristotelian inventory so as to include several additional sensory modalities – among them are our sense of temperature, balance, movement, and embodiment. In general, it is helpful to divide sensory modalities into two broad categories: exteroceptive modalities, like sight and hearing, which carry information about the external world as it affects our body; and interoceptive modalities, like balance and proprioception, which carry information about internal bodily states. The question which most interests us is, do these modalities have anything in common, in virtue of which our experience of them is called sensory?

Before Aristotle, Plato brings up the topic of sensation in Theaetetus. Here Socrates wonders whether “it is more correct to say that the eyes are that with which we see, or that through which we see”; whether “we hear with the ears, or through the ears”. Theaetetus answers “through” and Socrates praises him for his choice, adding that there is a “single form, soul or whatever one ought to call it, to which all [senses] converge—something with which, through the senses, as if they were instruments, we perceive all that is perceptible”.¹²

I find Plato’s way of framing the issue especially helpful; his remarks are insightful yet also intuitive enough not to beg any questions. Let us assume that the “single form” of which Plato speaks is what we call ‘mind’. Plato suggests that a distinguishing feature of sensory modalities of mental activity is the following: they display the involvement of a specific part of the body – the sense organ – as their “instrument”. Following Plato, we can say that the mind is that with which we see and hear in the same sense that it is that with which we think and desire: whether sensory or non-sensory, what is experienced is in any case a state of mind. The difference is that a sensory mental state consciously appears to us to be shaped by the workings of a specific bodily system (visual, auditory, tactile, etc.) in a way that a non-sensory mental state does not: as if it were its instrument.¹³ When you look into the crowd and see a friendly face, it is not only with your mind but also through your eyes that you see it.

Of course, spelling out more precisely what this sort of instrumental bodily involvement consists in is a challenge for expansionists and restrictivists alike. One approach to which the restrictivist may be immediately attracted appeals to empirical data. The restrictivist may start by assuming, reasonably enough, that the central nervous system is the mind’s primary residence. She may then distinguish the central nervous system from all other bodily systems. Finally, she may lay out her empirical criterion: a mental state is sensory just in case it is realized by a state of the central nervous system which robustly correlates with a state of some other bodily system. Contrapositively, a mental state is non-sensory just in case no such correlation can be established.¹⁴

This specification of the meaning of Plato’s ‘through’ is however too narrow, and it is therefore unable to sustain the restrictivist project. Not all sensory experience robustly correlates with relevant changes in a corresponding bodily system; i.e. not all sensory experience is perceptual. Consider, for example, memory and imagination. If I now recall the sound of the ocean in Cape Spear last August, there is no correlation between my experience of this memory and relevant changes in my auditory sense organs: since I am not in Cape Spear right now, the sound of the ocean is not affecting my body; yet my memory is auditory. Similarly, if I now imagine the face my brother is going to make when he finally opens the present I sent him, my imaginative experience is visual even though it is not a response to the current state of my visual sense organs.

Might this suggest that only perceptions are truly sensory and that there is no such thing as ‘sensory’ memory or imagination, strictly speaking? This view is not implausible, but neither is it compatible with restrictivism. Remembering and imagining things comprises a great deal of our conscious experience. If memories and imaginings are experientially conscious, this must be in virtue of some experiential property of theirs – be it proprietary or not. But if memories and imaginings are non-sensory mental states, then according to restrictivism they cannot have any proprietary experiential properties. Neither can they have non-proprietary experiential properties, however, because there are no co-occurrent perceptual states to which such properties might belong. So if memories and imaginings are non-sensory mental states, then restrictivism is false. The restrictivist should not adopt a definition of the sensory that only perceptual mental states can meet, because this would undermine her own position.¹⁵

At this point, the restrictivist might seek a more subtle empirical specification of the sensory. Perhaps it will be one which appeals to a mental state’s dispositional properties, or perhaps to its association with brain areas whose activation is also associated with perception. The details of how this sort of account might be made to work shall not concern us. For our purposes, all that matters is this: whichever account the restrictivist might develop, she should not get lost in the technical details so much that she loses sight of Plato’s original insight. The risk, in particular, is that by weakening the strength of the correlation to actual bodily activity, and by assigning crucial importance to brain areas instead, the restrictivist will be unable to capture the embodied character of sensation and, as a consequence, she will lose track of what sets it apart from other mental faculties.

An oversight of this kind would mystify the terms of the debate to such an extent that it would become difficult to even understand the restrictivist claim, let alone assess its truth. ‘Sensory’ is not a synonym for ‘mental’, and the category of the sensory can only expand so far if it is to retain its meaning. Non-sensory modalities of mental activity are also located in the brain, and therefore the restrictivist must, in addition to individuating dedicated brain areas, also find a way to explain the distinctive embodied character of sensory modalities. This task can surely be accomplished in a variety of ways, but we will not explore them. Instead, we can keep in mind Plato’s insight as a sort of general test; a vague and intuitive necessary condition, whose main function is to keep restrictivism in check by preventing that its truth come at the cost of resorting to a gerrymandered notion of the sensory.

We should now be able to see with sufficient clarity not only what restrictivism claims but also that its claim could be true: as a philosophical position, it is neither trivial nor obviously false. At the same time, proving that it is true of emotional experience requires taking on a real and difficult challenge.

Try to remember the last time you looked into a crowd and saw a friendly face, or simply imagine looking into a crowd and seeing a friendly face right now. Although no light reflections from the crowd are hitting your retinas with their electromagnetic radiation, your memory or imaginative state is visual insofar as you are imagining or remembering the scene as if through your eyes: you imagine the crowd from a perspective centered at a location that corresponds to your eyes and in shapes and colors analogous to the ones that your visual organs are apt to perceptually detect. Now consider the last time you looked into a crowd and hoped to see a friendly face. Your experience of hope is not visual, in that it does not appear to be filtered through your visual system. If your experience of hope is sensory, the bodily system through which you experience it – its sense organ – is a different and more mysterious one. The challenge for the restrictivist is to clarify which one it is and how it works, and similarly for all other varieties of emotional experience.

In the rest of this paper, I present two arguments to the conclusion that this challenge cannot be met. Although emotional experience is partly sensory, it is not entirely sensory; some features of emotional experience are, one might say, irreducibly cognitive.¹⁶ If correct, my conclusion speaks against current attempts to theorize emotion by analogy with sensory perception. If my reasoning is sound, then the idea that emotion is a kind of sensation is mistaken, and continuing to conduct emotion research under this mistaken assumption is detrimental. The restrictivist understanding of emotional experience is, perhaps unsurprisingly, just too restrictive; it is preventing us from attending to some of the most interesting aspects of emotion, and it should therefore be abandoned.

2 Getting Acquainted With Our Emotions

We can now delve into the heart of the matter. My plan for this section is as follows. First, I introduce some concrete examples of emotional experience, borrowed from a literary passage. Then I explain how the restrictivist would account for these examples. Finally, I lay out my argument against the restrictivist account. The argument is as follows. A century-long empirical search for the distinctive sensory manifestations of emotion has failed to discriminate even among basic emotion types. This suggests that most emotional states have no distinctive sensory manifestation. If an emotional state has no distinctive sensory manifestation, then either it is impossible to know by acquaintance that one is in that state, or restrictivism is false. But for at least some of these emotional states, it is possible to know by acquaintance that one is experiencing them. Hence, restrictivism is false.

To avoid grounding our discussion in simplistic descriptions of emotional experience, let us borrow our examples from Emily Brontë’s Wuthering Heights. In the passage below, a young girl, Cathy, meets her best friend, Heathcliff, after a long stay at her neighbors’ estate. During her visit, both Cathy and Heathcliff have changed, though in different ways. Cathy – whose brother is the current master of Wuthering Heights, Hindley – has become more sophisticated and well mannered; Heathcliff, an adopted servant whom Hindley hates, has instead fallen into a state of disgrace, neglect, and isolation:

“Heathcliff, you may come forward,” cried Mr. Hindley, enjoying his discomfiture, and gratified to see what a forbidding young blackguard he would be compelled to present himself. “You may come and wish Miss Catherine welcome, like the other servants.” Cathy, catching a glimpse of her friend in his concealment, flew to embrace him; she bestowed seven or eight kisses on his cheek within the second, and then stopped, and drawing back, burst into a laugh, exclaiming, “Why, how very black and cross you look! and how—how funny and grim! But that’s because I’m used to Edgar and Isabella Linton. Well, Heathcliff, have you forgotten me?” She had some reason to put the question, for shame and pride threw double gloom over his countenance, and kept him immovable. “Shake hands, Heathcliff,” said Mr. Earnshaw, condescendingly; “once in a way that is permitted.” “I shall not,” replied the boy, finding his tongue at last; “I shall not stand to be laughed at. I shall not bear it!” And he would have broken from the circle, but Miss Cathy seized him again. “I did not mean to laugh at you,” she said; “I could not hinder myself: Heathcliff, shake hands at least! What are you sulky for? It was only that you looked odd. If you wash your face and brush your hair, it will be all right: but you are so dirty!” She gazed concernedly at the dusky fingers she held in her own, and also at her dress; which she feared had gained no embellishment from its contact with his. “You needn’t have touched me!” he answered, following her eye and snatching away his hand. “I shall be as dirty as I please: and I like to be dirty, and I will be dirty.” With that he dashed headforemost out of the room, amid the merriment of the master and mistress, and to the serious disturbance of Catherine; who could not comprehend how her remarks should have produced such an exhibition of bad temper.¹⁷

When Cathy meets Heathcliff, she feels great joy and affection for her friend, whom she has dearly missed, but also some disdain for his present state. She and Heathcliff share a history of having been wild and untamed together. Cathy’s commitment to this shared history is never fully shaken, and it will eventually re-emerge in all its dramatic force – yet her present attempt to distance herself from it finds expression in a sense of social superiority and a somewhat detached, contemptuous attitude. Heathcliff understands that a change in Cathy’s conception of her own social condition now sets her apart from him, and this causes him to feel ashamed and angry. He responds to the situation by clinging to his pride and asserting, in false consciousness, that he could not care less about whether she thinks he is beneath her. His behavior surprises Cathy and causes in her confusion and distress. Meanwhile, Hindley takes great pleasure in Heathcliff’s humiliation and momentary estrangement from Cathy: the perverse delight he experiences is that of schadenfreude.¹⁸

Consider now Heathcliff’s emotional condition. Among other things, he is angry at the fact that Cathy thinks he is beneath her. Of course, in order to be angry at this fact, it is necessary that he on some level understand it; his judgment that Cathy feels disdain for him is at least a necessary pre-condition for his anger. And Heathcliff’s judgment has a perceptual basis: he heard her make fun of him and saw her look apprehensively at her hands and dress right after having touched him. On the basis of hearing her words and observing her movements, Heathcliff judges that Cathy looks down on him, and this makes him very angry.

Independently of whether Heathcliff’s judgment is a constitutive component of his anger, the restrictivist is committed to the view that it is not a constitutive component of his experience of anger.¹⁹ According to restrictivism, emotion does not have any proprietary, non-sensory experiential properties, because, of course, there are no such properties. Hence, a correct description of Heathcliff’s experience of anger will appeal only to the experiential properties of Heathcliff’s sensory mental states. The same will be true of Cathy’s mixed feelings of affection, disdain, surprise, and confusion.

If the restrictivist is correct, each emotional experience consists in the right combination of sensory experiential properties and nothing more. Such properties can be proprietary to a variety of perceptual states—most saliently, perception of changes occurring in the body. When Heathcliff forms the judgment that Cathy looks down on him, for instance, his autonomic system responds: his blood pressure rises, his heart and breathing rates increase, his muscles become tense, more blood flows through his extremities, and the release of hormones triggers a state of arousal. Heathcliff also conceives and performs intentional actions which only make sense conditionally on his judgment. Before storming out of the room, he loudly proclaims his indifference to Cathy’s attitude in an attempt to hurt her back and restore his dignity. These behaviors are consciously perceived by Heathcliff’s interoceptive and exteroceptive senses. The restrictivist would like to convince us that Heathcliff’s experience of anger is entirely constituted by his experience of such perceptions.

William James famously defended a version of this view:

Common sense says, we lose our fortune, are sorry and weep; we meet a bear, are frightened and run; we are insulted by a rival, are angry and strike. The hypothesis here to be defended says that this order of sequence is incorrect, that one mental state is not immediately induced by the other, that the bodily manifestations must first be interposed between, and that the more rational statement is that we feel sorry because we cry, angry because we strike, afraid because we tremble, and not that we cry, strike, or tremble, because we are sorry, angry, or fearful, as the case may be. Without the bodily states following on the perception, the latter would be purely cognitive in form, pale, colourless, destitute of emotional warmth. We might then see the bear, and judge it best to run, receive the insult and deem it right to strike, but we could not actually feel afraid or angry.²⁰

We can all agree that the phenomenological resources afforded by the varieties of sensory experience are impressive. Even so, they are insufficient. My argument against restrictivism has an empirical basis. Since James’s groundbreaking contributions, empirical psychology has been attempting to discover the ‘bodily signatures’ of emotion. According to what is known as the ‘basic emotion theory’, it should be possible to establish a high correlation between each basic emotion and a distinctive set of changes in the human body and behavior.²¹ The restrictivist assumes that, once this is done, each basic kind of emotional experience will be reducible to the perceptual experience of its distinctive set of changes.

The empirical search for ‘bodily signatures’ of basic emotion has, however, not been successful so far. In 1929, Walter Cannon argued, against James, that the same physiological changes are typical of fear as well as anger, and therefore physiological changes are insufficient to individuate basic emotion types.²² Since then, the situation has not significantly changed. In 2018, Lisa Feldman Barrett’s lab, guided by Erika Siegel, published the largest meta-analysis of physiological studies to date and confirmed Cannon’s conviction. The study concludes that “there is no one-to-one mapping between an emotion category and a specific autonomic nervous system response pattern”.²³ If this is right, the restrictivist assumption does not hold, at least insofar as changes triggered by the autonomic system are concerned: our experience of fear, for example, cannot be reduced to our sensory perceptual experience of the set of autonomic changes which are distinctive of fear, because there isn’t any such set.

In addition to autonomic system response patterns, emotions are tightly linked to patterns of behavior. The restrictivist could therefore insist that perception of one’s own autonomic response plus perception of one’s own behavioral response are jointly sufficient for individuating basic emotions and that our experience of emotion can be reduced to our experience of this conjunction of perceptual states. But amending the theory in this way only muddies the water; instead of providing a solution, it makes the problem more difficult to see. Behavioral expressions of emotion are only less distinctive and more variable than changes in the autonomic nervous system. From observing Cathy’s behavior, Heathcliff infers that she looks down on him. In response to this, he shouts his hasty response and makes a dash for the door, but he need not respond in this way. If he had remained quiet instead and pretended that everything was fine just to disrupt Hindley’s enjoyment of his humiliation, for example, his behavioral profile would not have shown him to be any less angry. Cases in which the subject experiences anger but is incapable of initiating an aggressive response, perhaps because of a physical or psychological impairment, are an especially powerful illustration of this.

The restrictivist may point out that her account should rely on the perceptual experience not of intentional action, but of more subtle behavioral responses such as facial expressions and skeletal muscle contractions. The empirical foundations of this version of the view are however also shaky, and the studies reporting that certain facial expressions are distinctive of particular emotions have been criticized for relying on defective methods that exploit bias and inflate consensus.²⁴ Consciously perceiving oneself frowning seems neither necessary nor sufficient for experiencing sadness. Moreover, compared to other kinds of bodily changes to which the restrictivist can appeal, the extent to which we are even conscious of the details of our facial and skeletal muscle contractions is less clear, and it is more difficult to see why they should at all be relevant to determining whether my experience is one of anger or fear, joy or scorn, shame or guilt.

As a last resort, the restrictivist may attempt to identify distinctive brain areas whose activation is both associated with basic emotions and meaningfully categorized as sensory. This strategy fails on two fronts. First, as I argued in §1, to earn the qualification, sensory experience must be shown to be embodied in a more robust way than non-sensory experience, and correlating it with brain activity, however specifically localized, is by itself not sufficient; some substantive link to the sense organs must also be established. Second, empirical evidence shows that there are no distinctive brain areas to whose activation basic emotions may be reliably linked. According to a recently published meta-analysis, there is no consistent correlation between basic emotion types and specifically localized brain regions.²⁵

If it is true that basic emotion types cannot consistently be associated with distinctive patterns of sensory activity, this has unwelcome consequences for restrictivism. Under this assumption, if the emotional experience of anger is merely sensory, then it is not distinctive. In other words, restrictivism predicts that we are unable to discriminate, on purely experiential grounds, what basic kind of emotion we are feeling. But this prediction is wrong. Of course we are able, at least sometimes, to discriminate what we feel on the basis of how we feel.

To better explain where the problem lies, let us start from a fairly uncontroversial claim in epistemology, namely the claim that knowledge by acquaintance is possible.²⁶ According to Fred Dretske, for instance, knowledge by visual acquaintance is captured by the following four conditions:

• (i)

  S sees b;

• (ii)

  b is P;

• (iii)

  the conditions under which S sees b are such that b would not look, L, the way it now looks to S unless it was P;

• (iv)

  S, believing the conditions are as described in (iii), takes b to be P.²⁷

Whenever a subject satisfies these four conditions, Dretske argues, she thereby knows that b is P.²⁸ Asking her to provide any further justification as to how she knows that b is P would be entirely inappropriate; she can see that it is!

David Pitt has recently pointed out that acquaintance is limited to neither vision nor perception, and so it can deliver knowledge not only about perceptual objects but also about one’s own conscious mental states – most crucially for his purposes, conscious thought.²⁹ If I am acquainted with the thought that P, it is possible for me to gain knowledge of some of its properties by acquaintance, i.e. without inductive or deductive inference, solely on the basis of my own experience. For example, it is possible for me to gain knowledge of the fact that the content of my thought is P. This however, Pitt argues, can only be possible if the experiential properties with which I am acquainted are distinctive, individuative, and proprietary of my thought that P both vis-à-vis any other kind of mental state (in particular, any kind of sensory mental state) and also, more specifically, vis-à-vis any other thought.

Jesse Prinz has rejected this last argumentative step in Pitt’s argument as not warranted. He does not deny that, in order to gain knowledge by acquaintance, my experience of the thought that P must be distinctive and individuative, but he denies that it must be proprietary. He points out that we are acquainted with sentences in inner speech, and sentences “stand in for” thoughts. We are so habituated to translating sentences into thoughts unconsciously, Prinz continues, that we are under the illusion that the experiential properties with which we are acquainted are proprietary of the thoughts themselves rather than the sentences in which we express them. He highlights three things: first, that the experiential properties of a sentence rehearsed in inner speech are sufficient to pick out any thought not only as a thought but also as the thought that P; second, that any unconscious cognitive mechanism can rely on this fact to link the rehearsed sentence to the thought it expresses; and, third, that the experiential properties of a sentence rehearsed in inner speech are sensory, i.e. visual and auditory. In light of all this, one can see that restrictivism and the possibility of knowledge of one’s own thought by acquaintance are not incompatible after all: the conditions under which the subject experiences the thought that P are such that her experience would not be the way it is unless she was thinking that P, and this is sufficient to meet Dretske’s conditions for knowledge by acquaintance.³⁰

I would like to test both sides of this debate by considering what results they produce when applied to the case of emotion. Let us consider an emotional experience for which Dretske’s conditions obtain:

• (i)

  Heathcliff feels a certain emotion (i.e. he is acquainted with his emotion in a way that is analogous to acquaintance with visual objects);

• (ii)

  the emotion he feels is, in fact, anger;

• (iii)

  the conditions under which Heathcliff feels his emotion are such that the emotion would not feel the way it now feels to him unless it was anger;

• (iv)

  Heathcliff, believing the conditions are as described in (iii), takes his emotion to be anger.

Under these conditions, Heathcliff has gained knowledge of his own emotional state by acquaintance. There is no need for him to provide any additional justification as to how he knows that he is angry; he can feel that he is!³¹

Now assume that, as the empirical literature suggests, the autonomic sensory constituents of Heathcliff’s emotional state are not distinctive of the kind of state it is, and there is no other way of individuating emotions on the basis of sensory experience.³² Were restrictivism true, condition (iii) would never obtain, and so gaining knowledge of one’s own emotional state by acquaintance would be impossible. But this sort of knowledge is not impossible. In fact, Heathcliff’s case happens to be favorable to the satisfaction of condition (iii) even regarding more subtle, richer features of the emotional state he is in; in particular, its intentional object. The depth of Heathcliff’s devotion to his relationship with Cathy, as well as his callousness to anything else, ensures that nothing in the world would make him feel angry in quite the same way as Cathy’s attitude towards him now makes him feel. And it seems therefore perfectly possible for Heathcliff to know by acquaintance not only that what he feels is anger – which would already be sufficient to prove my point – but even, more precisely, that it is anger at the fact that Cathy thinks he is beneath her.³³

Of course, sometimes we attend to the experiential properties of our emotional state and form false beliefs about it. But the claim defended here is not that acquaintance always delivers knowledge. The claim is rather that it is sometimes possible to gain emotional knowledge by acquaintance. There are cases in which I attend to the experiential properties of my emotional state and, solely on the basis of that, I do not know what state I am in; there are also cases in which I do know, and this is sufficient. Considering the case of emotion shows, contra Pitt, that the fact that we are talking about conscious mental states does not, by itself, guarantee that condition (iii) always obtains.³⁴ At the same time, it also shows, against the restrictivist, that the opposite idea, namely the idea that condition (iii) never obtains (not even for the most basic properties of the most fundamental types of emotional states), is just as mistaken. My conclusion is that – unless the restrictivist can show that to each kind of emotional experience apt to be known by acquaintance corresponds a distinctive set of sensory states to which the experiential properties of the emotion are proprietary – restrictivism about emotional experience is false. ³⁵

An attempt to apply Prinz’s inner speech strategy to the case of emotional knowledge reveals its shortcomings. If Heathcliff’s emotional experience is distinctive enough to enable him to know that he is angry, this cannot be explained by the fact of inner speech. If Heathcliff is experiencing sensations that are common to both anger and fear, yet he knows by acquaintance that he is angry rather than afraid, his knowledge cannot be based on the sensory experiential properties of his hearing himself innerly assert that Cathy’s attitude is outrageous and upsetting (rather than, say, dangerous and scary). This is because one’s experience of inner assertion of one’s own thoughts is only tangentially related to one’s emotional experience. Heathcliff may go ahead and tell himself that he could not care less about Cathy’s attitude towards him. He may even go so far as to overtly proclaim it in front of everybody, as he in fact does in the novel. This would change neither the fact that he is experiencing anger nor the fact that, if he paid attention to the experiential properties of the emotional state he is in, he would know that it is anger.

If emotion has cognitive constituents, whatever they are, they must not be confused with the standard, non-emotional varieties of thought that inner speech supposedly articulates. Even if we were to concede, perhaps rather too quickly, that a person’s experience of her own thoughts is reducible to her experience of inner speech, there is only so much cognitive activity that inner speech can reasonably be expected to track. If inner speech is similar enough to actual speech for it to be plausible that our experience of it is sensory, then it cannot be the bearer of the experiential properties of emotion. If, however, we require that inner speech be the bearer of the experiential properties not only of thought but also of emotion, we start to put great strain on the notion of inner speech and, in particular, on the idea that it is in fact sensory. It becomes then difficult to suppress the burgeoning suspicion that ‘inner speech’ is just another name for inner cognitive experience, in which case the notion is of no use to the restrictivist.

At this point the restrictivist may want to consider whether she should not simply deny the possibility of gaining emotional knowledge by acquaintance. Perhaps she is confident that she might be able to account for all emotional knowledge by appeal to deductive or inductive inferential capacities. This view pairs naturally with eliminativism about emotional experience: if emotion had no experiential properties, then one could not be acquainted with it, and it would therefore make sense to say that gaining emotional knowledge by acquaintance is impossible. But pairing any view with eliminativism about experience means taking on the burden of its implausibility, and that burden is too heavy for the view not to sink. Restrictivism was initially interesting precisely because it promised to make sense of emotional experience; if it must deny that there is anything to make sense of, it betrays that promise more than it fulfills it.

If, however, the restrictivist denies the possibility of knowledge by acquaintance while also rejecting eliminativism about emotional experience, her position becomes an awkward one to maintain. Once it is granted that emotion does indeed have experiential properties, it follows that its subject is acquainted with it. The restrictivist would therefore have to concoct an odd epistemology according to which acquaintance with the experiential properties of emotion, differently from other kinds of acquaintance, never delivers knowledge. This might require more effort and ingenuity than it is worth. As far as I can see, contesting the empirical results is the better, if stubborn, alternative.

3 The Experience of Valence

Among the features of emotional experience about which it seems possible for us to acquire knowledge by acquaintance is one of great ethical significance: valence. When in experimental settings people are asked to establish similarity relations among various sorts of emotion, two dimensions emerge along which emotions are reliably ranked as more or less experientially similar to one another: the first is arousal, or activation, and the second is valence, or hedonic tone.³⁶ This evidence from empirical psychology confirms a commonsensical idea: normally, we can tell whether an emotional experience of ours feels good, bad, neither, or a mix of both.

Surprise, for example, can vary in valence depending on the circumstances, and this variation is consciously felt. Although Cathy’s surprise at Heathcliff’s bad temper may be intertwined with the lingering joy of seeing him again, and perhaps some confusion about the situation, overall it is clearly a surprise of the unpleasant sort. Had Cathy been pleasantly surprised, her emotional state would have felt quite different. Schadenfreude is instead always experienced as positively valent, even in its bitterness. When Hindley sees Heathcliff humiliated and isolated, he is delighted. If it didn’t feel good, Hindley’s experience of schadenfreude would be so radically altered as to not even be recognizable as such.

Restrictivism is unable to provide a satisfactory account of our experience of valence – or so I contend in this section. My argument has a simple structure: If restrictivism about emotional experience is true, then emotional valence is not experientially felt. But emotional valence is experientially felt; the positive or negative character of our emotions makes a direct, substantive contribution to the quality of our conscious lives. Hence, restrictivism is false.

Let us start by examining the second premise of the argument, i.e. the idea that emotional valence is experientially felt. Our confidence in the truth of this premise is hard to shake: the fact that experiencing a negative emotion feels bad while experiencing a positive emotion feels good seems difficult to deny. Nonetheless, a possible strategy for resisting the conclusion of the argument is to maintain that, contrary to appearance, this premise is false. This is the strategy adopted by Jesse Prinz. In support of his claim, Prinz appeals to William James’s famous argument from introspective subtraction:³⁷

If we fancy some strong emotion, and then try to abstract from our consciousness of it all the feelings of its characteristic bodily symptoms, we find we have nothing left behind, no “mind-stuff” out of which the emotion can be constituted, and that a cold and neutral state of intellectual perception is all that remains.³⁸

Prinz applies James’s argument to the case of emotional valence with the aim of persuading us that valence is indeed not experientially felt. His reasoning proceeds in something like the following way. Take any emotional experience of yours and attend to it. Now subtract from it all its “bodily”, i.e. sensory, components. If, as James predicts, when you do this you find that nothing is left of the valent character of your emotional experience – i.e. nothing is left but “a cold and neutral state of intellectual perception” – then you must conclude that there is no experience of valence as such, i.e. no sense in which your emotion can be truly said to feel distinctively positive or negative to you. This would mean, for example, that although Heathcliff’s anger is negatively valent, Heathcliff does not directly experience its negative character; all he directly experiences are the sensory changes that accompany, and perhaps partly constitute, his anger.

This use of James’s subtraction argument extends beyond emotional valence, and it targets, more widely, all hedonically charged mental states, including sensory pleasure. Take, for instance, the pleasure of drinking mulled wine while walking downtown on a cold winter night, and attend to it. Now subtract from your pleasurable experience all of its hedonically neutral sensory components: the taste of the wine, the sounds you hear and things you see all around you, the feeling of warmth you perceive while you take a sip, your sense of the cup’s weight as you keep walking. If after you mentally perform this subtraction you find that nothing is left of your pleasure, you must conclude that sensory pleasure does not have any distinctively hedonic experiential properties.

But if your experience of sensory pleasure does not have any distinctively hedonic experiential properties, then what makes it an experience of pleasure cannot be experiential. This is because experiencing certain (hedonically neutral) sensations – even when they happen to be the object of your pleasure – is neither sufficient nor necessary to make them a pleasure: it is possible for you to experience the same sensations without finding them pleasurable, and it is possible for you to be in a state of pleasure in the absence of such sensations.

This point is known as ‘the heterogeneity objection’ to experiential theories of pleasure and pain, but it can also be used, more restrictedly, against experiential theories of emotional valence. Here introspective subtraction is combined with introspective comparison with the aim of showing that the experiential properties of positive and negative experiences can wholly change from instance to instance, thereby proving that pleasure and pain are not essentially experiences. Try to abstract away from the experiential peculiarities of your many pleasures and sorrows, and you will see for yourself that no common experiential property emerges.

The most influential formulation of the heterogeneity objection, which includes emotions among the pleasures and pains we experience, is due to Henry Sidgwick:³⁹

Shall we then say that there is a measurable quality of feeling expressed by the word “pleasure,” which is independent of its relation to volition, and strictly undefinable for its simplicity?—like the quality of feeling expressed by “sweet,” of which we are also conscious in varying degrees of intensity. This seems to be the view of some writers: but, for my own part, when I reflect on the notion of pleasure,—using the term in the comprehensive sense which I have adopted, to include the most refined and subtle intellectual and emotional gratifications, no less than the coarser and more definite sensual enjoyments,—the only common quality that I can find in the feelings so designated seems to be that relation to desire and volition expressed by the general term “desirable.”⁴⁰

That the heterogeneity objection draws strength from a restrictivist conception of experience is here hinted at by Sidgwick’s presumption that the feeling of pleasure, if there is such a thing, must be in some crucial respect akin to the sensory feeling of sweetness. This becomes even more apparent in recent formulations of the objection. In the following passage, for example, Fred Feldman describes two pleasures with no sensory experiential features in common, and from this he concludes, much too quickly, that they have no experiential features in common whatsoever:

Consider the warm, dry, slightly drowsy feeling of pleasure that you get while sunbathing on a quiet beach. By way of contrast, consider the cool, wet, invigorating feeling of pleasure that you get when drinking some cold, refreshing beer on a hot day...They do not feel at all alike.⁴¹

When deliberating on what conclusions to draw from introspective tests of this sort, it is important to keep in mind that changes in attention alter our conscious experience in ways that we do not yet fully understand and that this might influence these tests’ results. An emotion is normally directed at a certain intentional object. Diverting attention from its object naturally results in diverting attention from the emotion itself, including its valent character. If Cathy is shocked by Heathcliff’s bad-tempered behavior but stops paying attention to it in order to focus on the relation between her overall experience of shock and its bodily symptoms, her shock is likely to fade. If I am taking pleasure in the taste of the wine, and you ask me to subtract from my experience that very taste which is the object of my pleasure, it is not especially surprising that my pleasure will be gone as well.

In addition, there is reason to think that our experience of valence is especially vulnerable to attentional shifts. Focusing on your own suffering, as well as being distracted from it, can be a very effective means to lessen your pain,⁴² and full immersion in a task can weaken your experience of valence to a surprising degree, as it characteristically happens in the experience of flow: the positively valent feeling of “being in the zone.”⁴³ A possible explanation for these variations – tentatively put forth by Leonard Katz and supported by research by Marcus Raichle and his collaborators – is that all monitoring modes of mental activity, among which they include valence, are suppressed by attention-demanding tasks.⁴⁴

A further explanation, perhaps not incompatible with the first, is that valence is a feature of the attitudinal rather than objectual component of mental states. I can both fear and admire the dexterity in hunting of a mountain lion. My fear of it has a negative valence, whereas my admiration for it has a positive valence. Although my attitudes have the same intentional object, their valence is different. If valence is an attitudinal rather than objectual feature of emotional states, then we should not be surprised to discover that we can experience it better sideways, so to speak, than when we make it the central object of our attention; after all, this is exactly what the term ‘attitudinal’ (as opposed to ‘objectual’) is meant to capture. I conclude that the evidence provided by James’s subtraction argument does not give us sufficient reason to deny that emotional valence is experientially felt.

Let us now move on to consider the other premise of my argument. According to this premise, restrictivism entails that emotional valence is not experientially felt. To see why this entailment holds, consider the following. Restrictivism is the view that all experience is sensory. Given this, the most straightforward way to reconcile it with the claim that valence is experientially felt is to hold that valence is, at least in part, a sensory phenomenon. But under any plausible account, valence is not sensory.

A brief overview of contemporary theories will help me prove this point. Contemporary theories of valence can be organized into three broad families: evaluative theories, desire-based theories, and imperativist theories. My main goal here is not to assess these theories’ merits; it is rather, more modestly, to draw attention to something they all have in common: a commitment to the idea that valence is non-sensory. It is noteworthy that this commitment is shared even by the theories developed and endorsed by restrictivist philosophers.⁴⁵

Let us group the first family of views on valence under the label ‘evaluativism’. All evaluativist theories highlight the relation between valence and value and aim to illuminate the nature of the former by appeal to the latter. They hold that emotional valence is best understood as a kind of value assessment. Different versions of evaluativism specify the nature of this assessment in different ways.⁴⁶ The relation between a value assessment and a full-blown value judgment is an especially controversial and interesting node of discussion, but we can ignore such details given our present purposes.

The second family of views proposes to understand the nature of emotional valence by appeal to desire: these are the ‘desire-based’ theories. In general, they all hold that a subject’s emotional state has positive (or negative) valence just in case the subject has a desire for (or aversion against) some aspect of the emotion. Each variant of the theory will specify the nature and object of this special sort of desire differently, but again, given our purposes, we do not need to focus on these differences.⁴⁷

Finally, there is ‘imperativism’. Like desire-based theorists, imperativists emphasize the motivational power of valence. Rather than explaining this motivational power by appealing to a motivational mental state such as desire, however, they directly ground it in the capacity to produce behavioral and attitudinal changes. Accordingly, they hold that emotional valence is best understood as a sort of imperative command: more specifically, as a command to increase (or decrease) the probability of either the emotion’s own occurrence, or that of its object, depending on the specific version under consideration.⁴⁸

Take, for example, Hindley’s enjoyment of Heathcliff’s humiliation. Evaluativists will explain the positive valence of Hindley’s enjoyment by appeal to the fact that Hindley views or judges Heathcliff’s humiliation to be (at least seemingly) good. The explanation put forth by desire theorists, instead, will appeal to the fact that Hindley desires Heathcliff’s humiliation, either intrinsically or in some other distinctive way. Imperativists propose yet a third alternative: Hindley’s enjoyment is positively valent in virtue of the fact that it involves a command to increase the probability of its own occurrence, or else of the occurrence of Heathcliff’s humiliation.

Which of these theories is most convincing need not concern us here. What matters to us is that they all converge on the idea that emotional valence is non-sensory. The explanation for this is, I think, rather plain. Valence is a highly general mental phenomenon, with numerous and diverse instantiations; it is too general and diverse, in fact, not to cut across specific sensory modalities and have its functionality be, at least for the most part, independent of each. As a consequence, the nature of valence is most helpfully illuminated when it is considered in relation to a similarly general and multi-faceted mental phenomenon, be it evaluation, desire, or command.

Our capacity to have valent emotional states is analogous to our capacity to desire and evaluate things as good or bad in at least this one respect: it is too general and flexible to be mediated by any bodily system other than the central nervous system. The same is true of mental states with imperative content: triggering the implementation of strategies encouraging or discouraging the occurrence of a given state of affairs (or mental state) is not a job for any bodily system other than the central nervous system. From this I conclude that, regardless of which theory of valence is ultimately correct, we can be confident that valence is non-sensory.

The restrictivist may at this point switch strategy and attempt to say about the experience of valence what she usually says about the experience of thought: she may concede that valence is experientially felt, but she may also insist that its experiential properties are non-proprietary; they should ultimately be attributed to some co-occurrent sensory mental state. One hypothesis she may especially press us to consider is the following: valence may co-occur with a distinctive set of bodily changes, triggered by the autonomic nervous system, to whose perception the experiential properties of valence are proprietary.

This strategy is not especially satisfying when used to account for our experience of thought, but I think it works even less well for valence. We have just seen why any attempt to understand valence as a sensory phenomenon must fail: its functionality is too general, flexible, and diverse to be firmly linked to a bodily system other than the central nervous system. A similar reason prevents us from reducing our experience of valence to the sensory perceptual experience of a distinctive set of autonomic bodily changes: while our experience of valence is clearly noticeable and significant, it is also at the same time highly heterogeneous and hard to pin down. This is the truth behind the heterogeneity objection: although valence is indeed experientially felt, it is not the case that we can isolate its experiential features, and make them the focus of our attention, in the same way as we seem able to do for perceptual experience. In particular, the experiential features of valence do not seem to be ‘objectual’ but rather ‘attitudinal’. If our experience of valence is non-sensory, this important difference is vindicated.

If my reader is by now convinced that valence is both non-sensory and experientially felt, she will agree that restrictivism is false: it is not the case that all experience is sensory. Nonetheless, she may attempt to salvage the spirit of the view by weakening it. Although it is not true that all experience is sensory, it might still be true that all experience exhibits many of the features characteristic of sensory experience. This is the strategy adopted by Peter Carruthers. In a recent article, he maintains that valence is both experientially felt and non-sensory. He proposes an evaluativist account according to which valence is a mental representation of seeming goodness or badness but with some caveats: he insists that it is a “perception-like” and “non-conceptual” sort of mental representation.⁴⁹

I do not think that restrictivism can be effectively rescued in this way, however. Once the essential link between experience and sensation is severed, the view becomes unprincipled, and the expectation that the experience of valence be perception-like appears arbitrary given that perception is sensory, whereas valence is not. Non-sensory experience is an understudied phenomenon, and we know too little about it at this stage to say how similar to perceptual experience it may or may not be. In addition, once we reject the claim that the experience of valence is sensory, setting the condition that it be non-conceptual becomes irrelevant to the restrictivist cause. The experiential properties of any mental state can in principle meet this condition. This contradicts the spirit of restrictivism, which, as its name suggests, was originally designed to set restrictions on which mental states could have proprietary experiential properties. Consider that just as it can be said that valence non-conceptually represents a given intentional object as seemingly good or bad, so it can be said that belief non-conceptually represents a given intentional object as seemingly true or false. But belief is a paradigmatic cognitive attitude. If Carruthers’s reformulation renders restrictivism compatible with the idea that occurrent belief has proprietary experiential properties, then an endorsement of this version of restrictivism is hardly distinguishable from an endorsement of expansionism.

4 Conclusion

Sensory perception is a paradigmatic type of conscious experience, and most theories of consciousness are designed to account for it. However, many such theories are ill-equipped to account for other varieties of experience. Thus, a common approach in philosophy of mind and cognitive science seeks to reduce all conscious experience to the sensory, including emotional experience. I called this approach ‘restrictivism’ and argued against it. If sound, my argument has direct implications for the theory of consciousness and emotion. In particular, from the claim that emotional experience is partly non-sensory it follows that (i) conscious experience is not reducible to sensory experience and (ii) emotion is not reducible to sensory perception.

In §1, I clarified the terms of the debate between restrictivism and expansionism so as to highlight the conditions under which each view offers a substantive thesis in the philosophy of mind. I began by motivating restrictivism as part of a reductive approach to mind’s place in nature. Next, I explained how our conception of expansionism should be broadened. Finally, I drew on Plato to suggest an impartial criterion for distinguishing sensory from non-sensory experience: sensory experience displays the involvement of a bodily system other than the central nervous system – the sense organ – as its instrument. This brings out a challenge for restrictivism: if emotional experience is entirely sensory, it should display this sort of bodily involvement, and we should be able to articulate which sense organs are involved.

I then presented two arguments for the view that emotional experience is partly non-sensory: the first in §2, the second in §3. The first argument has an empirical and epistemological basis. Sometimes I know what emotion I am experiencing without inference, simply by being acquainted with it. But current empirical evidence suggests that the sensory constituents of emotion are insufficient to individuate basic emotion types and are therefore inadequate to ground such knowledge. So restrictivism entails, falsely, that knowledge of emotion by acquaintance is impossible.

My second argument appeals to the non-sensory nature of emotional valence. Although we are still mostly in the dark about what valence is and how it works, it is nonetheless apparent – even to restrictivists – that it is not sensory. Given this, restrictivism implies, falsely, that valence is not experientially felt. One can resist the argument in two ways: either by showing that, contrary to appearance, the implication is true or by redefining restrictivism so as to avoid the implication. I closed by explaining why neither strategy is satisfactory.⁵⁰