Written as a result of AI Safety Camp Virtual 2023. Thanks to the following people for feedback and helpful conversations: Oliver Bridge, Tim Gothard, Rasmus Jensen, Linda Linsefors.^[1]

This post reviews the literature on "wanting" and "liking", two primary components of what is commonly referred to together as the biological reward system. It is intended to be informative for AI safety-related work, especially within approaches that try to leverage insights from neuroscience for alignment.

Section 1 introduces the distinction between two high-level components of reward: wanting and liking. At this point, I simplify the topic and treat both as homogenous categories. Sections 2 and 3 delve deeper into each component and give a more fine-grained model, including a description of their neurobiological substrates. (2.2. and 3.2 are more technical/dry neuroscience, so you may want to skip them, if that's not your primary interest.) Section 4 discusses the functional relationships between them and why this kind of "division of labor" may have been favored by evolution.

1. Introduction

I will start by introducing four concepts central to this post: wanting, "wanting", liking, and "liking".^[2] They carve the space of human values^[3] along two dimensions. The first of those is the distinction between the things we are motivated to do/driven towards (wanting) versus the things we feel good about happening (or being about to happen) (liking). The second distinction is between the more basic/less-sophisticated components ("wanting" and "liking") of each and the more elaborated and cognitive components (wanting and liking).^[4] The two parts of this Section elaborate on these two dimensions.

1.1. Liking vs Wanting

Liking is when you take a bite of a tasty food and its taste brings you pleasure or, more generally, positively valenced affective states. Wanting is when you are motivated to act in order to obtain that food and bring it to your mouth in order to consume it.

This is probably enough to intuit the rough contours of the distinction, but at the same time, it may raise some questions. I can think about cases where I kind of want to get up and go to the kitchen, but I'm too tired (or my willpower is too depleted) to get up. So I don't get up, even though I know the food in the fridge is very good and if I did get up and get it, I would be very glad about doing so. On another note, what if the food is definitely tasty and brings me pleasure and I enjoy it "on some level", but at the same time believe (or at least some part of me believes) that I shouldn't eat it? Maybe I think I shouldn't even enjoy it? Maybe I consider it unhealthy or fear that somebody will judge me for eating it in that particular context, or my God/religion forbids eating pork.^[5] Don't these edge cases question the simplified distinction between liking and wanting as too distinct but coherent and homogenous things? Probably they do, but we still can learn quite a bit about what we might call the primitives of human values by studying liking and wanting on this coarse-grained level.

According to what I would call a naive view of the relationship between liking and wanting, the reason we want X is that we like X, or maybe at least expect/predict to like X. The correlation between realizing that we (will) like something and developing a want for it soon after that makes this view fit well enough most daily situations.

However, liking and wanting sometimes come apart. We may begin to want something, even if we neither had an opportunity to experience liking nor could predict that we would like it. To give a concrete example, humans and other animals typically develop sexual drive before their first sexual encounter. Sometimes they may not even know what sex is. Nevertheless, they implement somewhat intelligent behavior which was evolutionarily selected in order to reliably end in sexual intercourse.

In other situations, something that has so far always been disliked becomes an object of desire. Robinson and Berridge (2013)^[6] taught rats to associate a particular stimulus (henceforth the conditional stimulus; CS) with a repulsive experience of extremely salty water being injected straight into their mouth (henceforth the unconditional stimulus; UCS). The rats quickly realized that the UCS reliably followed the CS, so they learned to turn away and retreat from the CS whenever they saw it.^[7] In the next phase of the experiment, the researchers injected the rats with two compounds that mimicked the brain signals which under normal circumstances would convey information about dangerously low blood sodium levels. Importantly, since their diet had always had adequate amounts of salt, they had never had an opportunity to discover the extent to which their (dis)liking of salty food differs depending on the blood sodium levels.

Nevertheless, their behavior changed dramatically. Instead of retreating from the CS, they started approaching it, eager to get their precious dose of salt. A change in the (perceived) physiological state turned something aversive into something desirable, and the cue associated with it went along.

Examples of liking-wanting dissociation don't end here. One may develop an addiction to a drug even if they don't like the state this drug induces that much. Moreover, over a prolonged period of drug use, its positive subjective effects (liking) often wear off but addiction (wanting) keeps its hold.^[8] In other words, one may want to get a dose even if one doesn't like getting the dose and is aware that they're not going to like what happens once they get the dose.

Some people also develop compulsive desires or behavioral patterns that do not lead to positively valenced experiences, but that are nevertheless very hard to resist. Subclinical examples include compulsively checking one's phone, e-mail, or social media, doomscrolling, and addiction to gambling. Speaking at least from my anecdotal phenomenal perspective, such things certainly do feel like something I want but do not like.^[9]

Perhaps less obvious are examples of things that give us a lot of positive experiences, but for which we nevertheless don't develop any kind of robust desire. I recall Julia Galef mentioning on some podcast that she really likes apples but nevertheless never learns to "desire" apples, and whenever she happens to eat an apple, she is reminded of that. If the (main) reason we want something is that we like it, shouldn't she develop wanting for apples proportional to how much she likes them? On another, more speculative note, some people report extreme pleasure during some meditative states and yet developing no addiction for it. I discuss more experimental examples of selectively impacting liking but not wanting in Section 2.

1.2. Liking vs "Liking" and Wanting vs "Wanting"

Folk-psychological concepts are not guaranteed to be a good fit for brain/mind sciences. For some examples, concepts such as consciousness, emotion, memory, pain, or even the idea of "concept" itself turned out to lump together importantly distinct phenomena (cf. Ramsey, 2022, Section 2.3). We might expect liking and wanting to also go this way, and that they will need at least a bit of refinement if we want to use them as starting points for a neuroscientifically adequate ontology.

On its face, there seems to be an asymmetry between liking and wanting in that the latter can be, at least in many cases, inferred from behavior,^[10] whereas the former is a matter of "private" experience. Obviously, this is especially problematic in cases of animals incapable of verbalizing their ongoing subjective experience. However, the asymmetry may be weaker than it seems. After all, we can identify which automatic behavioral and/or physiological responses in humans correlate with (the verbal reports of)^[11] positively or negatively valenced experiences (at least specific to some domain, such as food). We can then turn to animals and look for analogous responses (e.g., engaging analogous muscle groups in roughly the same patterns of movement) in order to see whether they correlate with the same kinds of objectively observable events that we would predict the animal to like or dislike.

For example, in the case of food,^[12] it turned out that pleasant and unpleasant tastes robustly trigger specific facial expressions (see Berridge & Robinson, 2003, Figure I). Importantly, they can occur even in the absence of conscious functioning,^[13] e.g., in sleep or in individuals with deficient neocortical functioning,^[14] such as anencephalic infants (Steiner, 1973; cf. Berridge & Winkielman, 2003).

The observation that some objectively measurable manifestations of pleasure can occur without conscious awareness, motivated the introduction of the distinction between "liking" (core affective reactions that don't require consciousness) and liking (conscious pleasure)^[15] (Berridge & Robinson, 2003; Berridge & Kringelbach, 2015). Analogously, "wanting" (incentive salience, cue-triggered motivation that doesn't require consciousness) was distinguished from wanting (cognitive desires with declarative goals).^[16] Thus, the "liking"/liking and "wanting"/wanting distinctions capture the difference between implicit or objectively measurable components ("quoted") and explicit or subjective components (unquoted).

While the need for objective measures of reward was the original motivation for making the distinction, narrowing down on the explicit components of "liking" and "wanting" made it easier to identify their neural substrates. To a first approximation, we have (1) a "liking" system, concentrated around a handful of hedonic hot and cold spots, with opioids playing the main role in the generation and modulation of "(dis)liking" and (2) a "wanting" system, which is more distributed (although to some extent centered around the ventral tegmental area) and with the dopamine as the key neurotransmitter. The two systems are to some extent separate, but they also overlap.

2. Liking

2.1. "Liking" and liking

The idea of "unconscious pleasure" may seem contradictory. In what sense, can something that happens to us be pleasant but not be available to consciousness?

The introduction of an unconscious aspect of pleasure follows a particular pattern of concept extrapolation that we often see in psychology. We discover that a particular mind-related phenomenon has some objectively measurable "behavioral signature". For example, humans, other great apes, rats, and many other species of mammals, all protrude their tongues in response to tasty foods (see Berridge & Robinson, 2003, Figure I). Responses to aversive tastes, are also homologous across taxa to a large extent. Other than that, exposing people to a valenced stimulus unrelated to taste, (e.g., happy versus angry faces) influences how much tasty food they consume and this effect persists even when these stimuli are not consciously perceived (Winkielman et al., 2005).^[17]

Hence, the rationale for dividing pleasure/liking into a subconscious component of objectively measurable "core affective reactions" to valenced stimuli ("liking") and consciously perceived pleasure (liking). The latter is closer to the common meaning of the verb "to like".^[18] It denotes the valenced feeling available to the consciousness, an approval or disapproval of the ongoing state of affairs.^[19]

Since among these two, "liking" is the objectively measurable component and a majority of research in this domain was done on laboratory animals (whose subjective experience can't be measured by verbal reports), it is not surprising that we know much more about the neurobiological substrate of "liking" than about that of liking (and the same is true of "wanting" and wanting). Therefore, my discussion of the latter is more of a speculation than in the case of the former. Also, most animal studies of "liking" relied on a restricted set of "domains of rewarding stimuli" (mostly food, sex, and drugs), so our knowledge of how core affective reactions differ between these domains is still quite limited.

2.2. "Liking" in the brain

The most important components of the "liking" circuitry are a handful of hedonic hot spots and cold spots, which are small groups of neurons, whose stimulation selectively increases or decreases "liking" reactions, respectively. (In the case of the increase, it is sometimes called hedonic enhancement.) None of them^[20] are anatomically distinct structures (you're not going to find them in the index of a typical neuroanatomy textbook). Rather, they are "functional islands" embedded in bigger regions involved in many functions not closely related to "liking".

The two most important ones are located in the basal ganglia, specifically in the nucleus accumbens (NAc) and the ventral pallidum (VP). Both the NAc and the VP contain a hot spot and a cold spot. The NAc is divided into the core and the shell, of which the latter hosts a hot spot in the front and a cold spot in the back. In the VP, the arrangement is reversed, with the hot spot at the back and the cold spot at the front (Richard et al., 2013; Berridge & Kringelbach, 2013, 2015).

Beyond the basal ganglia, hedonic hot spots (but not cold spots, as far as I know) have been located in the orbitofrontal cortex (OFC), anterior insula (aIns), and the parabrachial nucleus of the pons (PBN; Söderpalm & Berridge, 2000; cf. Smith et al., 2010). However, the hot spots in the NAc and VP appear to be the most important, being the generators of pleasure. Lesioning or deactivating either the NAc hot spot or the VP hot spot eliminates hedonic reactions (Berridge & Kringelbach, 2015). Moreover, while damaging the NAc hot spot merely abolishes "liking", damaging the VP hot spot (or its temporary inactivation) causes "disliking" of normally positive things (e.g., sucrose).

Moreover, anencephalic children with little to no neocortex as well as people or non-human animals who have undergone extensive OFC lesions still retain intact "liking" reactions. People with OFC lesions also report conscious pleasure, suggesting that even liking is not strictly dependent on the cortex either (cf. Berridge & Winkielman, 2003).^[21] In contrast, some baseline level of activity is necessary in each of the two basal ganglia hot spots in order for additional stimulation of one to produce hedonic enhancement (Smith & Berridge, 2007, Smith et al., 2011; cf. Richard et al., 2013).

Importantly, it's not as simple as 'stimulate a hot spot to enhance "liking", stimulate a cold spot to decrease "liking"'. The choice of neurotransmitter used for stimulation matters. Here, the opioid receptors are the most relevant (cf. Berridge & Robinson, 2003; Berridge & Kringelbach, 2015; Smith et al., 2010). In the NAc hot spot, agonists of mu, delta, and kappa opioid receptors all cause hedonic enhancement, while in the VP hot spot, and the cortical hot spots, this role appears restricted to mu-opioid receptor (MOR) agonists. Depending on the region, stimulation of non-opioid receptors can give similar results. So far, the neurotransmitters shown to produce hedonic enhancement include anandamide (NAc), orexin (NAc, VP, PBN, OFC), and GABA (PBN; Söderpalm & Berridge, 2000).

Earlier, I mentioned that the "liking" system and the "wanting" system are closely connected. This is illustrated by the fact that, in most cases, stimulating a subcortical hedonic hot spot increases "wanting", in addition to "liking". Moreover, the range of compounds that increase "wanting" within a hedonic hot spot is much greater than the range of those that increase "liking" (Berridge & Kringelbach, 2013). Stimulation of the cold spots in the NAc and VP can also produce "wanting". I discuss this further in Section 3. Conversely, opioids can indirectly increase the activity of the ventral tegmental area, which is the main source of dopamine in the core "wanting" pathways (Zhang et al., 2022).

The NAc shell region encompassing the hot and cold spot is often described in terms of an "affective keyboard", where the placement of neurons strongly correlates with the affective reaction (e.g., "liking" versus "disliking") elicited by their activation (Richard et al., 2013; Berridge & Kringelbach, 2013, 2015). More specifically, there appears to be a gradient of valence extending from front to back. Stimulation of more frontally placed regions of the "keyboard" elicit "liking" reactions whereas more caudally placed neurons inhibit "liking" and/or elicit "disliking", sometimes together with species-specific responses to threats, such as predators.

Figure 2 from Richard et al. (2013) shows the distribution of populations of neurons in the NAc whose stimulation elicits particular kinds of behavior. We have the hedonic hot spot (red-orange), where activation typically enhances "liking" reactions. Behind it, we see two zones. At the ventral (lower) side, there is a group of neurons with the kind of functionality we would expect from a hedonic cold spot (blue). Their stimulation inhibits "liking". At the dorsal (upper) side, we have another cluster, which also has an inhibitory effect but instead of suppressing "liking", they suppress aversive reactions (purple). All of these sites, in addition to their impact on "liking", "still generate eating" (i.e., "wanting" to eat), as does the broader (green) region in which they are located and a part of dorsomedial neostriatum (roughly, the caudate nucleus and putamen located above the NAc), shown as the green spot in the top-left corner of the figure.

The cold spot region also contains cells whose function extends beyond the modulation of "(dis)liking". Stimulation of some of them produces fearful responses or aggression displays, such as throwing dirt at potentially threatening stimuli. Importantly, the result of stimulation of these cells can also be modulated by the environment. In a calm, peaceful, and safe setting, the regions whose stimulation increases positive reactions expand, whereas the aversive/fearful reaction regions shrink. Stressful, dangerous, unsafe environments do the opposite.

Similarly to the hot spot, the result of stimulating the cold spot depends on the kind of ligand used. Stimulation of the same three kinds of opioid receptors (mu, delta, kappa) that enhance "liking" in the NAc hot spot, produces intense "(dis)liking" or even fear-related behaviors in the neighboring cold spot. Mu receptor agonists also inhibit "liking" in the VP cold spot.

2.3. What is conscious pleasure good for?

It is, admittedly, not clear how and to what extent "liking" and liking depend on each other. What does it take for something that is "liked" to become liked? We know that "liking" can occur without liking, but what about the reverse? Can something satisfy our explicit hedonic feelings without impacting any of these "core affective reactions"? Or maybe "liked" things become liked whenever consciousness is "turned on" (at least over some threshold)? From my cursory review of the literature, it seems that we don't know, although the orbitofrontal cortex (OFC) emerges as a major candidate for the region whose activity (perhaps in addition to more basic "liking" structures) is important for liking (Kringelbach, 2005, 2010).

The global workspace theory of consciousness (GWT)^[22] (and the experiments it draws on) may give some suggestions about makes a "liked" stimulus/event liked. The brain can do unconsciously quite a bit of complex processing (e.g., semantic meaning of words). The experiments carried out within the GWT paradigm that showed this, used sensory masking in order to prevent the stimuli from reaching consciousness. According to GWT, the neural activity associated with representing/processing a particular stimulus must exceed some critical threshold in order to spread to other (in particular multimodal/associative/higher-level) brain regions, which then can start processing it in a somewhat synchronized manner. This is the neural basis of "becoming conscious of something". The representation becomes available to other brain systems, including the ones directly connected to the speech organs, so that we can report our consciousness of the stimulus. Otherwise, its processing remains unconscious, local, and circumscribed to a particular lower-level brain region.

Translating this view to the case of "liking" and liking, there may be a similar threshold of intensity of core hedonic impact ("liking") that a stimulus must exceed in order to be broadcast to the global workspace and become consciously liked. It's plausible that spread to some particular regions (such as the OFC) is particularly important.

Notably, in order to prevent the valenced stimuli from reaching consciousness, the experiments that showed the influence of unconsciously processed stimuli on objectively observable correlates of pleasure (e.g., Winkielman et al., 2005) used the same method as the GWT, namely sensory masking. This is a minor piece of evidence that the results from the GWT experiments may translate to the case of "liking" and liking.

If this perspective is right, it may point towards a possible function of conscious liking in that explicitizing the hedonic value increases the range of possible routes of impact on other brain systems (e.g., the motivational circuits discussed in Section 3). So perhaps the question "What is conscious pleasure good for?" is nothing but a special case of "What is consciousness good for"?

On a more speculative note regarding the possibility of liking without "liking," I wonder if top-down influences of such factors as self-image, normative convictions, social expectations, or (broadly understood) reflective evaluation of the current situation (e.g., how good I feel with my life, or the way things are going in the world in general) may induce liking without inducing core affective reactions and corresponding activity in the subcortical hedonic hot spots. On the other hand, I would also expect that at least in some cases (perhaps in a majority or even all cases), the subcortical (dis)pleasure generators would become secondarily activated as a result of this top-down influence.

3. Wanting

3.1. "Wanting" and wanting

It is hardly an original observation that our actions don't always reflect our explicit beliefs about what we should do. This phenomenon has been given many names, such as akrasia, weakness of will, or lack of willpower. This may make the distinction between "wanting" (incentive salience) and wanting (cognitive desire) more relatable and intuitive than the one between "liking" and liking.

"Wanting" can be seen as the unconscious counterpart of wanting, similarly to how "liking" is the unconscious counterpart of liking. Whereas wanting (cognitive incentives) refers to plans directed towards goals we are aware of and explicitly represented desires, "wanting" (incentive salience) refers to more impulsive, reactive, low-level motivation, which can act independently of what we (state that we) want or like.

More specifically, "wanting" is defined as "a conditioned motivation response of a brain, usually triggered by and assigned to a reward-related stimulus" (Berridge, 2007). A stimulus is reward-related if it is associated with an event that is "rewarding in itself", e.g., sweet taste. The association can be simple, like occurring very close in time, or it may involve some more sophisticated cognitive learning processes.^[23] The learned reward-related stimulus is often called the "conditioned stimulus" (CS), whereas the inherently rewarding event is called the "unconditioned stimulus" (UCS). However,^[24] not all inputs that drive "wanting" are learned, as brains are wired to respond with "wanting" to some stimuli, independently of learning (or in the absence of/prior to learning). Plausibly, the original adaptive value of "wanting" was to motivate the animals to pursue a small set of unconditioned rewards, such as food, sex, or favorable ranges of environmental conditions, such as appropriate temperature and acidity. Over time, as more sophisticated learning mechanisms evolved, the role of "wanting" became extendable by learning (Berridge, 2007).

According to Berridge and Robinson (2003) cognitive incentives (wanting) are distinguished from incentive salience ("wanting") by three (or maybe four) features:

… they are (1) known or imagined (cognitive incentive representation); (2) expected to be pleasant (hedonic expectation); (3) subjectively desired and intended to be gained (explicit cognitive representation of wanting) and, perhaps, (4) known to be obtainable by actions that cause it to occur (understanding of act–outcome causality).^[25]

I will speculate a bit more about the differences and relationships between "wanting" and wanting in Section 3.3. In the next Section 3.2, I outline the neurobiological basis of "wanting".

3.2. "Wanting" in the brain

"Liking" can be roughly located in a handful of hedonic hot spots and cold spots, with endogenous opioids being the main players in the circuitry (as discussed in Section 2). The neural substrate of "wanting" is more distributed throughout the brain, with dopamine as the key neurotransmitter.

The human brain has several dopaminergic pathways, the most relevant for "wanting" being the mesolimbic pathway. It goes from the ventral tegmental area (VTA) to the ventral striatum (including the nucleus accumbens) and some other areas. Although it is the biggest supplier of dopamine to brain regions involved in incentive salience (Ikemoto, 2010), it is not the only one, and, at least in some artificial setups, "wanting" can occur even when it stops working.

We know that, i.a., from studies of More specifically, animals that had their mesolimbic pathway ablated, can still develop a compulsion to self-stimulate via electrodes implanted in some brain regions (cf. Ikemoto, 2010, p. 131). It is not clear to what extent these results translate to "wanting" without the mesolimbic pathway more generally.

VTA and other regions involved in "wanting" form a highly interconnected network (cf. Ikemoto, 2010). However, many of them are not "wanting"-specific, but also involved in other aspects of reward. In Section 2, I discussed the hedonic hot and cold spots in the nucleus accumbens and the ventral pallidum. Stimulating them with many neurotransmitters (including those which tend to elicit "(dis)liking" reactions) tends to produce "wanting" behavior, both related to approach ("wanting" X) and aversion ("wanting" not-X). The same is true for the parabrachial nucleus of the hindbrain, whose GABA-A receptors' stimulation "wanting" but has a small region adjacent to it, where it also elicits "liking".

Other regions of the network are involved in learning. For example, Pavlovian learning seems to depend on a circuit whose major components are the basolateral amygdala, the orbitofrontal cortex, and the nucleus accumbens (Burke et al., 2010). On the other hand, stimulation of the central amygdala when paired with a highly salient stimulus (doesn't matter whether it's pleasant or unpleasant, it just needs to have strong valence in either direction) can establish very strong "wanting" even for highly unpleasant stimuli (Warlow et al., 2020).

Two other dopaminergic pathways that are important for "wanting" are the mesocortical pathway and the nigrostriatal pathway. The mesocortical pathway goes from the VTA to the prefrontal cortex and is involved in executive functioning. Its disorders, including those involving dopamine depletion or other interference with dopaminergic activity, are associated with impaired cognitive control and working memory (cf. Ott & Nieder, 2019).

The nigrostriatal pathway goes from the substantia nigra (SN) to the dorsal striatum and its main role is movement control. The death of a majority of SN cells is the proximate cause of Parkinson's disease. Parkinson's patients tend to develop symptoms associated with decreased functioning of the mesolimbic pathway (e.g., apathy) and the mesocortical pathway (e.g., attentional deficits). The severity of these symptoms is highly correlated with the severity of motor symptoms, suggesting some relevant degree of coupling between these systems (cf. Leyton, 2010, pp. 232-233).

The next few paragraphs discuss how dopaminergic activity in general impacts "wanting". It is not meant to be a complete overview of evidence or comparison with alternative hypotheses (for that see: Berridge, 2007), but rather as an informative illustration of the role played by this neurotransmitter.

Probably the most straightforward method to test how some neurotransmitter X influences some behavior Y is to lower or increase the levels of X and measure changes in Y. One way to do it is by breeding genetically modified animals that have abnormally low or high levels of the neurotransmitter in their synapses (cf. Berridge, 2007, pp. 403-405).

Dopamine-deficient (DD) mice, with almost no dopamine in their brains, can be created by knocking out the gene coding for tyrosine hydroxylase, an enzyme without which dopamine can't be produced. DD eat and drink barely anything at all, not enough to sustain themselves.^[26] In order for them to eat and drink normally, they need to be medicated with L-DOPA (a direct dopamine precursor, removed from dopamine by just one step in the production chain), which can temporally restore their dopamine to normal levels.

This makes it possible to test (1) whether DD mice display different affective/"liking" reactions for different kinds of stimuli (e.g., sugar solution versus water) and (2) whether after trying both of them, they learn to prefer one over the other, as measured by their choices on subsequent trials. It turns out that they can do both, which suggests that dopamine is not required for (at least some forms of) "liking" and reward-related learning. Similar patterns have been observed in wild-type (i.e., normal/not genetically modified) mice, whose dopaminergic systems were impaired later in their life by neurochemical lesions.

On the other hand, hyperdopaminergic mice, which have almost triple the normal amount of dopamine in their synapses (compared to the wild type), can be created by knocking out the gene coding for the dopamine transporter, a protein that removes dopamine from the synapse. Such mice are more motivated to obtain rewards, more resistant to stimuli distracting them from the focus on the current goal, and willing to work harder for rewards. In other words, they seem to "want" their rewards more than the wild type. Their ability to learn associations between stimuli and rewards or learn which actions lead to rewards, as well as "liking" reactions, remain unaffected.

What about dopamine disturbance diseases in humans (cf. Leyton, 2010)? Parkinson's disease (PD) is caused by the degradation of dopaminergic cells in the substantia nigra, which are not directly involved in the mesolimbic system. Still, many PD patients exhibit symptoms associated with decreased dopaminergic functioning in the mesolimbic (e.g., apathy, avolition) and mesocortical (e.g., worse attention and executive functioning) systems. The severity of these symptoms correlates with the strength of motor problems, more central to Parkinson's. Some PD patients treated with L-DOPA (~3-4%; Pezzella, 2005) develop dopamine dysregulation syndrome (DDS), where overcompensation for dopamine deficiency leads them to develop "pathological" "wanting" behavior (addiction, gambling, compulsive sexual activity, even if they had no history of such before the medication) making them illustrative cases of hyperdopaminergy.^[27]

Many highly addictive potentials are dopaminergic. Central examples include amphetamine, cocaine, and their analogs, which work primarily by increasing the amount of dopamine that stays in the synaptic cleft. Interestingly, in animal studies, their addictive potential can be reduced (perhaps even (almost?) completely eliminated?) if they are given together with DA antagonists, i.e., compounds that bind to dopamine receptors without activating them, which prevents dopamine itself from binding to them and exerting its typical effects (cf. Puglisi-Allegra & Ventura, 2012). At the same time, dopamine antagonism does not eliminate other effects of these dopaminergic drugs. For example, some euphoric effects remain when amphetamine is given with DA antagonists (cf. Leyton, 2010), suggesting that these are either mediated through mechanisms other than dopamine or perhaps some dopamine receptors that are not blocked by the particular used in the study^[28] (Nader et al., 1997; Ikemoto, 2010).

Highly addictive drugs that don't interact with the dopamine system directly tend to have secondary dopaminergic effects. For example, the agonists of mu-opioid receptors (such as morphine, heroin, and fentanyl) typically work by inhibiting GABA-ergic neurons located in the posterior part of the VTA or an area adjacent to it, called rostromedial tegmentum (RMTg). On the other hand, these GABA-ergic neurons inhibit the dopaminergic neurons of the VTA, which drive "wanting". Therefore, inhibition of the former means disinhibition of the latter and thus an increase in the concentrations of DA in regions targeted by the VTA (cf. Zhang et al., 2022).

Caffeine is another compound with indirect dopaminergic effects (and a relatively mild addictive potential). Although its main mechanism of action is blocking the adenosine receptors, it also increases dopamine release, contributing to its psychostimulant and reinforcing properties (Ferré, 2016). We can see that in experiments, where adding caffeine to yogurt strengthened the preference developed for that yogurt, as compared to the same yogurt but without caffeine (Panek et al., 2013). Analogous experiments with similar results were performed on bees and caffeine-enriched nectar, with similar results (Wright et al., 2013).^[29]

Pavlovian-instrumental transfer is what happens when an animal that is already working in order to obtain some reward/UCS, starts working even harder upon perceiving a CS associated with the UCS it's currently pursuing. This effect has been closely associated with an increase in mesolimbic dopaminergic activity and can be modulated by intervening in the mesolimbic system, e.g., with dopamine agonists (Berridge, 2007, pp. 420-421; Cartoni et al., 2016; Salamone et al., 2016).

3.3. What are cognitive incentives/conscious wanting good for?

Why do we need wanting in addition to "wanting"? In Section 2 I asked an analogous question with respect to liking and "liking" and gave a provisional hypothesis that consciousness of a valenced stimulus makes it accessible to other parts of the brain, allowing them to interoperate and make use of each other's outputs. Are wanting and "wanting" in an analogous relationship?

Berridge and Robinson (2003) seem to endorse something like this. In their view, wanting allows the animal to achieve objectives, that can't be achieved through simple learning of associations and require more complex inference, working memory, etc. They write:

One essence of rational cognition is its inferential exploitation of lawful consistencies in the world and, typically, future value is best inferred from past value. In addition, the rat must use its understanding of which actions cause which outcomes to select from several possible actions the one that will produce the best reward.

Relatedly, wanting, as a process under conscious executive control, is more stable with respect to changing local incentives, which makes planning and execution of plans possible in the first place.

Thus, the mesocortical pathway which goes from the VTA to some parts of the prefrontal cortex is a likely candidate for a major substrate of wanting, as one of its major roles is executive function.

Echoing the speculations from the end of Section 2, can we want something without "wanting" it? Perhaps we can take again the self-image angle: we model ourselves as wanting X, but this model is not accurate and not strong enough to override "wanting" that pushes in the other direction. Perhaps sometimes wanting without "wanting" is adaptive because it causes the organism to think about plans of action that can be executed once a proper context occurs, so that "wanting" is triggered and makes use of the information contained in the plans developed due to wanting.

4. Why "like" something if you can just "want" it?

Ex ante, we might expect that "wanting" itself, paired with a sufficiently good learning algorithm, should be enough to achieve any goals necessary for survival and reproductive fitness.

Maybe it is necessary or more efficient to have separate systems for (1) adaptive but "mindless" responses to local incentives and (2) goal-directed behavior that relies on taking into account broader context; hence "wanting" and wanting, respectively. Still, this leaves us with a question about the adaptive value of "liking" and liking. Had they not contributed to our ancestors' fitness in one way or another, evolution would not have selected for them.^[30]

It seems that the field has not reached a consensus on that question. Here I present three hypotheses. Like much of evolutionary psychology, they are all tentative, and if evidence for them is at best indirect. Importantly, these hypotheses are neither exhaustive nor mutually exclusive.

Hypothesis 1: Liking extends wanting

On this account, we start with a small set of default motivations that evolution selected for ("wants"), and "liking" helps repurpose the motivational system towards new motivations. The "wanting" system is more evolutionarily ancient. "Liking" emerged relatively recently, in animals living in more cognitively demanding environments that necessitated acquiring new motivational mechanisms over the lifetime. Pleasure is an additional training signal for the "wanting" system, allowing the brain to repurpose systems specialized for being motivated towards one domain of stimuli/events towards another domain. The need for this "lifetime reprogramming" may arise due to the environment being too complex or too variable for evolution to encode appropriate sources of motivation into the genome.

Kent Berridge (a pioneer of this line of research) seems to lean towards this hypothesis (podcast interview link). He gives credit for it to, among others, Anthony Dickinson (e.g., Dickinson & Balleine, 2010).

Quoting directly from the episode (lightly edited by me):

[…] pleasure exists because it essentially allows brain "wanting" systems that might have evolved for one thing (e.g., food) to experience a new pleasant event (e.g., social accomplishment) and to enjoy that event and to bring to bear the brain "wanting" systems for the old thing on to this new target, basically giving us a new target of desire.

On this account, it seems to be somewhat similar to the picture presented by the Shard Theory view of human values, except that not all the values (contextually activated motivations/"wants") are learned from scratch.

Here, Berridge doesn't distinguish between "liking" and liking. My interpretation is that he views them as serving a similar function, just on different levels of "cognitive sophistication", similar to "wanting" and wanting.

There is an interesting category of cases, where the learned change in valence happens prior to a corresponding change in motivation. In other words, your experience changes whether/how much you "(dis)like" X but without updating your motivation for X. Your motivation for X is updated the next time you encounter X and experience altered valence.

Dickinson and Balleine (Balleine, 2010) give an example, where the first author (A.D.) who really liked watermelons, at some point got sick shortly after eating a watermelon (most likely the disease and eating the fruit were not related). A few days later, he went to eat a watermelon and although it was most likely basically the same kind of watermelon, it tasted awful. Apparently, his "liking"/liking system wrongly inferred the watermelon to be the causal factor behind the sickness, which altered his taste, but not his motivation to eat watermelons, until he tasted a no-longer-tasty watermelon. These cases have been reproduced in rat experiments.^[31]

Note that this is different from the Salt Sea experiment (Robinson & Berridge, 2013), where rats' motivation was already altered by their physiological state before being presented with the stimulus. In the watermelon case, on the other hand, the aversion occurs unexpectedly.

Hypothesis 2: Preparing physiology

"Liking" reactions associated with food and fluids seem to prepare the organism for intake of nutrients. Increased salivation facilitates pre-digestion of the food in the mouth, licking the lips ensures that some bits of the food are not left out, increased gastric movements prepare the rest of the digestive system, etc.

Hypothesis 3: Implicit social communication

An animal's physiological reactions, like the ones discussed above, often carry socially important information. Thus, it's plausible that in more social species these behaviors would evolve to become more pronounced, in other to facilitate implicit social communication. On the other hand, they may also become less reflective of the actual physiological state, in order to produce signals that are more likely to influence the behavior of the other animal in the direction that is beneficial for the signaller.

Hypothesis 4: Additional information to update behavior on

Valence ("liking"/liking) may also route partially processed information to the motivational circuits in order to update already ongoing behavior. If we do X for the first time and it quickly turns out that we like it, we do more of it. An obvious caveat is that we may not be able to experimentally disentangle the indirect effect mediated by valence from the direct effect. We have seen that it is possible to very quickly develop strong motivation for something without "liking" it, e.g., in the wireheading studies.

References

Berridge, K. C. (2007). The debate over dopamine's role in reward: The case for incentive salience. Psychopharmacology, 191(3), 391–431. https://doi.org/10.1007/s00213-006-0578-x
Berridge, K. C., & Kringelbach, M. L. (2008). Affective neuroscience of pleasure: Reward in humans and animals. Psychopharmacology, 199(3), 457–480. https://doi.org/10.1007/s00213-008-1099-6
Berridge, K. C., & Kringelbach, M. L. (2013). Neuroscience of affect: Brain mechanisms of pleasure and displeasure. Social and Emotional Neuroscience, 23(3), 294–303. https://doi.org/10.1016/j.conb.2013.01.017
Berridge, K. C., & Kringelbach, M. L. (2015). Pleasure Systems in the Brain. Neuron, 86(3), 646–664. https://doi.org/10.1016/j.neuron.2015.02.018
Berridge, K. C., & Robinson, T. E. (2003). Parsing reward. Trends in Neurosciences, 26(9), 507–513. https://doi.org/10.1016/S0166-2236(03)00233-9
Berridge, K. C., & Robinson, T. E. (2016). Liking, wanting, and the incentive-sensitization theory of addiction. The American Psychologist, 71(8), 670–679. https://doi.org/10.1037/amp0000059
Berridge, K., & Winkielman, P. (2003). What is an unconscious emotion? (The case for unconscious "liking"). Cognition and Emotion, 17(2), 181–211. https://doi.org/10.1080/02699930302289
Burke, K. A., Franz, T., Miller, D., & Schoenbaum, G. (2010). Conditioned Reinforcement and the Specialized Role of Corticolimbic Circuits in the Pursuit of Happiness and Other More Specific Rewards. In M. L. Kringelbach & K. C. Berridge (Eds.), Pleasures of the Brain (pp. 50–62). Oxford University Press.
Cartoni, E., Balleine, B., & Baldassarre, G. (2016). Appetitive Pavlovian-instrumental Transfer: A review. Neuroscience & Biobehavioral Reviews, 71, 829–848. https://doi.org/10.1016/j.neubiorev.2016.09.020
dela Peña, I., Gevorkiana, R., & Shi, W.-X. (2015). Psychostimulants affect dopamine transmission through both dopamine transporter-dependent and independent mechanisms. European Journal of Pharmacology, 764, 562–570. https://doi.org/10.1016/j.ejphar.2015.07.044
Dickinson, A., & Balleine, B. (2010). Hedonics: The Cognitive–Motivational Interface. In M. L. Kringelbach & K. C. Berridge (Eds.), Pleasures of the Brain (pp. 74–84). Oxford University Press.
Ferré, S. (2016). Mechanisms of the psychostimulant effects of caffeine: Implications for substance use disorders. Psychopharmacology, 233(10), 1963–1979. https://doi.org/10.1007/s00213-016-4212-2
Ikemoto, S. (2010). Brain reward circuitry beyond the mesolimbic dopamine system: A neurobiological theory. Novel Perspectives on Drug Addiction and Reward, 35(2), 129–150. https://doi.org/10.1016/j.neubiorev.2010.02.001
Kringelbach, M. L. (2005). The human orbitofrontal cortex: Linking reward to hedonic experience. Nature Reviews Neuroscience, 6(9), 691–702. https://doi.org/10.1038/nrn1747
Kringelbach, M. L. (2010). The Hedonic Brain: A Functional Neuroanatomy of Human Pleasure. In M. L. Kringelbach & K. C. Berridge (Eds.), Pleasures of the Brain (pp. 202–221). Oxford University Press.
Leyton, M. (2010). The Neurobiology of Desire: Dopamine and the Regulation of Mood and Motivational States in Humans. In M. L. Kringelbach & K. C. Berridge (Eds.), Pleasures of the Brain (pp. 222–243). Oxford University Press.
Nader, K., Bechara, A., & van der Kooy, D. (1997). Neurobiological constraints on behavioral models of motivation. Annual Review of Psychology, 48(1), 85–114. https://doi.org/10.1146/annurev.psych.48.1.85
Ott, T., & Nieder, A. (2019). Dopamine and Cognitive Control in Prefrontal Cortex. Trends in Cognitive Sciences, 23(3), 213–234. https://doi.org/10.1016/j.tics.2018.12.006
Panek, L. M., Swoboda, C., Bendlin, A., & Temple, J. L. (2013). Caffeine increases liking and consumption of novel-flavored yogurt. Psychopharmacology, 227(3), 425–436. https://doi.org/10.1007/s00213-013-2971-6
Pezzella, F. R., Colosimo, C., Vanacore, N., Di Rezze, S., Chianese, M., Fabbrini, G., & Meco, G. (2005). Prevalence and clinical features of hedonistic homeostatic dysregulation in Parkinson's disease. Movement Disorders, 20(1), 77–81. https://doi.org/10.1002/mds.20288
Puglisi-Allegra, S., & Ventura, R. (2012). Prefrontal/accumbal catecholamine system processes high motivational salience. Frontiers in Behavioral Neuroscience, 6, 31. https://doi.org/10.3389/fnbeh.2012.00031
Ramsey, W. (2022). Eliminative Materialism. In E. N. Zalta (Ed.), The Stanford Encyclopedia of Philosophy (Spring 2022). Metaphysics Research Lab, Stanford University. https://plato.stanford.edu/archives/spr2022/entries/materialism-eliminative/
Richard, J. M., Castro, D. C., Difeliceantonio, A. G., Robinson, M. J. F., & Berridge, K. C. (2013). Mapping brain circuits of reward and motivation: In the footsteps of Ann Kelley. Neuroscience and Biobehavioral Reviews, 37(9 Pt A), 1919–1931. https://doi.org/10.1016/j.neubiorev.2012.12.008
Robinson, M. J. F., & Berridge, K. C. (2013). Instant transformation of learned repulsion into motivational "wanting". Current Biology : CB, 23(4), 282–289. https://doi.org/10.1016/j.cub.2013.01.016
Salamone, J. D., Pardo, M., Yohn, S. E., López-Cruz, L., SanMiguel, N., & Correa, M. (2016). Mesolimbic Dopamine and the Regulation of Motivated Behavior. In E. H. Simpson & P. D. Balsam (Eds.), Behavioral Neuroscience of Motivation (pp. 231–257). Springer International Publishing. https://doi.org/10.1007/7854_2015_383
Smith, K. S., & Berridge, K. C. (2007). Opioid limbic circuit for reward: Interaction between hedonic hotspots of nucleus accumbens and ventral pallidum. The Journal of Neuroscience : The Official Journal of the Society for Neuroscience, 27(7), 1594–1605. https://doi.org/10.1523/JNEUROSCI.4205-06.2007
Smith, K. S., Berridge, K. C., & Aldridge, J. W. (2011). Disentangling pleasure from incentive salience and learning signals in brain reward circuitry. Proceedings of the National Academy of Sciences of the United States of America, 108(27), E255-264. https://doi.org/10.1073/pnas.1101920108
Smith, K. S., Mahler, S. V., Peciña, S., & Berridge, K. C. (2010). Hedonic Hotspots: Generating Sensory Pleasure in the Brain. In M. L. Kringelbach & K. C. Berridge (Eds.), Pleasures of the Brain (pp. 27–49). Oxford University Press.
Söderpalm, A. H., & Berridge, K. C. (2000). The hedonic impact and intake of food are increased by midazolam microinjection in the parabrachial nucleus. Brain Research, 877(2), 288–297. https://doi.org/10.1016/s0006-8993(00)02691-3
Steiner, J. E. (1973). The gustofacial response: Observation on normal and anencephalic newborn infants. Symposium on Oral Sensation and Perception, 4, 254–278.
Szczypka, M. S., Rainey, M. A., Kim, D. S., Alaynick, W. A., Marck, B. T., Matsumoto, A. M., & Palmiter, R. D. (1999). Feeding behavior in dopamine-deficient mice. Proceedings of the National Academy of Sciences, 96(21), 12138–12143. https://doi.org/10.1073/pnas.96.21.12138
Warlow, S. M., Naffziger, E. E., & Berridge, K. C. (2020). The central amygdala recruits mesocorticolimbic circuitry for pursuit of reward or pain. Nature Communications, 11(1), 2716. https://doi.org/10.1038/s41467-020-16407-1
Winkielman, P., Berridge, K., & Wilbarger, J. (2005). Unconscious Affective Reactions to Masked Happy Versus Angry Faces Influence Consumption Behavior and Judgments of Value. Personality & Social Psychology Bulletin, 31, 121–135. https://doi.org/10.1177/0146167204271309
Wright, G. A., Baker, D. D., Palmer, M. J., Stabler, D., Mustard, J. A., Power, E. F., Borland, A. M., & Stevenson, P. C. (2013). Caffeine in Floral Nectar Enhances a Pollinator's Memory of Reward. Science, 339(6124), 1202–1204. https://doi.org/10.1126/science.1228806
Zhang, J.-J., Song, C.-G., Dai, J.-M., Li, L., Yang, X.-M., & Chen, Z.-N. (2022). Mechanism of opioid addiction and its intervention therapy: Focusing on the reward circuitry and mu-opioid receptor. MedComm, 3(3), e148. https://doi.org/10.1002/mco2.148

Ordered alphabetically, by last name. ↩︎
I italicize liking, "liking", wanting, and "wanting" in order to emphasize that I'm using these terms in a "technical" sense. "Non-technical" senses are non-italicized. In a few places of this section I also sometimes lump liking and "liking" into "liking" and wanting and "wanting" into "wanting". ↩︎
Not necessarily exhaustively, there may be some (things we might want to consider as) values that don't fit neatly into any of these categories. ↩︎
The distinction between explicit and implicit is also sometimes used, but I find it unintuitive. ↩︎
By the way, the taboo against eating pork has a very interesting origin. See this video from Religion for Breakfast. ↩︎
See also Steve Byrnes's post about that study. ↩︎
The CS itself can become aversive or desired, even when the UCS it predicts appears in a different place than the CS. In such cases, the CS is said to become a "motivational magnet" (e.g., Robinson & Berridge, 2013). ↩︎
Explaining this phenomenon in terms of wanting to avoid unpleasant effects of the withdrawal syndrome doesn't fit the empirical data (Berridge & Robinson, 2016). ↩︎
While here I am speaking about subclinical cases of behavioral addictions, I also expect this to be a factor in obsessive-compulsive disorder and related conditions. One reason I think so is that the neurotransmitter most consistently involved in OCD seems to be dopamine, which is strongly implicated in wanting (see Section 3). Moreover, the most successful pharmacological treatment for OCD is naltrexone, which is also used in many standard addictions and acts by regulating dopaminergic transmission from the VTA. ↩︎
Although it probably still requires making some assumptions about the agent's biases and cognitive limitations. See, e.g., Christiano (2018). ↩︎
I'm not going to discuss the topic of phenomenal consciousness and its relationship with verbal reports because I consider the former to be illusory. ↩︎
Food being obviously the easiest category of "rewards" to study. ↩︎
By "conscious functioning", I mean something like "the global workspace" being up and running. ↩︎
With the neocortex being the part of the brain we expect to be important for conscious awareness. ↩︎
From now on, I italicize liking, "liking", wanting, and "wanting", in order to emphasize that I'm using these terms in their "technical" sense. "Non-technical meanings" of liking and liking are non-italicized. ↩︎
Berridge and Robinson (2003) also introduced a third distinction between two forms of learning: cognitive and associative, but it is not the focus of this post. ↩︎
I found no studies on that, but I have a very confident guess that "liking" would also occur in animals that are asleep or even in some kinds of palliative states, such as coma, perhaps even locked-in syndrome. ↩︎
Of course, the correspondence is not perfect and the boundaries between the daily meanings of "to like" and "to want" are blurry. Still, semantic distance from "to like" to liking is smaller than to "liking". ↩︎
Importantly, "the ongoing state of affairs" can include things extended over a long timescale. ↩︎
At least none of the ones we know about, to the best of my knowledge. ↩︎
At the same time, some studies show that monkeys and rats with OFC damage, although they still respond to rewards, are impaired in using reward information to guide their behavior, relative to animals with intact OFC (Berridge & Kringelbach, 2008), perhaps pointing to the role of conscious pleasure in reward-related learning. ↩︎
For a great introduction to GWT, see Kaj Sotala's review of The Consciousness and the Brain by Stanislas Dehaene. ↩︎
E.g., when I realize that when I get back to exercising after a long break, I start feeling much better on a daily basis after a few weeks, which increases my motivation to exercise (although this is probably not a good example of "wanting"). ↩︎
Perhaps I am slightly deviating from the definition of "wanting" as "conditioned responses". This seems true though and in agreement with the idea (endorsed by Berridge) that the original adaptive of "wanting" was to drive the animal's behavior to satisfy some small set of needs. ↩︎
I take the mention of "pleasant" in (2) to refer both to "liking" and liking, with the latter being used in a broad sense, which includes (i.a.) reflective evaluation of the state of the world conditional on having achieved the wanted goal. I think this interpretation is justified because otherwise, the definition would exclude clear examples of wanting, such as a person doing hard work for which they are not going to receive any "pleasant reward", unless we take a very broad meaning of "pleasure" (not mentioning more extreme cases like suicide bombers and kamikaze). ↩︎
Szczypka et al. (1999) write that "young [DD] pups that had never been injected with l-DOPA would lick and swallow small drops of a liquid diet placed by their mouth. Apparently, these kinds of responses don't require dopamine, perhaps being a kind of "liking" reactions. ↩︎
See also Oliver Sacks's Awakenings. ↩︎
Most dopamine antagonists work only on a particular subtype of dopamine receptors. ↩︎
Interestingly, in addition to increasing DA directly, amphetamine and cocaine-like psychostimulants also appear to increase DA via an indirect route (Peña et al., 2015). ↩︎
Alternatively, they might be spandrels. This probably isn't the case for "liking", as spandrels typically (ever?) have distinct brain circuits. If liking is a natural consequence of "liking" plus global workspace/consciousness systems, then it would also probably not be a spandrel. ↩︎
Dickinson and Balleine's account of the function of reward is slightly different than the one I'm presenting here. You can read it yourself if you're curious. ↩︎