Previously: Seeing Red: Dissolving Mary's Room and Qualia, A Study of Scarlet: The Conscious Mental Graph

When we left off, we'd introduced a hypothetical organism called Martha whose actions are directed by a mobile graph of simple mental agents. The tip of the iceberg, consisting of the agents that are connected to Martha's language centers, we called the conscious subgraph. Now we're going to place Martha into a situation like Mary's Room: we'll say that a large unconscious agent of hers (like color vision) has never been active, we'll grant her an excellent conscious understanding of that agent, and then we'll see what happens when we activate it for the first time.

But first, there's one more mental agent we need to introduce, one which serves a key purpose in Martha's evolutionary history: a simple agent that identifies learning.

We recall that Martha's species had evolved ever-better fluency with language, and along the way their minds developed a conscious/unconscious distinction, based on the functional difference (in the ancestral environment) between things that might be beneficial to communicate and things that weren't. Any new conscious node or connection has the potential to be communicated, but not all such elements are created equal; some of them (like a key bit of gossip, or a pattern suddenly grasped) have much larger effects on the full graph (and the conscious subgraph) than others.

Since the minds of Martha's species are similar in structure, it's likely that the new conscious ideas that rearrange Martha's graph might have similar effects on her friends' graphs- and therefore, Martha's species has evolved to give such connections high priority for communication.

One way to do this is to have a simple mental agent which tells Martha when she's learned something of significance. That is, it becomes active when the conscious subgraph rearranges on a large scale¹. When this happens, it traces the cascade of changes backward until it finds the most basic new node or connection that started the revolution. When it finds the culprit, it forms the conscious idea "I just learned X", where X is the new node or connection; thus, when Martha is thinking or talking about related ideas, the important bit is more likely to occur to her.

In our toy example from yesterday's post, at the moment that the DREAD VILLAIN reveals himself as the LONE STALWART's father, a number of changes ripple through Martha's graph (once all the subconscious processes that transform patterns of sound into conscious concepts have gone through). The learning agent activates and seeks out the root cause; it finds the new connection that DV = AS, and the attached memory of the DREAD VILLAIN's mechanical bass voice revealing the secret. The learning agent creates a stronger memory of this idea and the moment it occurred, and strengthens the connection from these to the language centers. Then, when Martha retells the story, the nearby agents will trigger this particular memory (and Martha will include it unless she has other reasons not to). Everything functions as it ought.

So what happens when Martha has a fundamentally new experience?

Martha's Room

Let's put Martha in the situation of Mary's Room. She has all of the mental equipment for color vision, but has never had it activated by external experience. She has factual knowledge of the correspondence between particular objects and their colors. Her conscious graph has linked the names of cherry red and firetruck red and lime green to their proper RGB triplets, and knows which colors are between which. And she has factual knowledge of the structure of her subconscious visual-processing agents, and her mind in general- that is, her conscious graph contains a subgraph that is a good (though of course not infinitely detailed) model of the full graph, and this subgraph can model quite well what the full graph will do once color vision is activated.

At last, we introduce color vision for the first time. Vast subconscious agents form new connections to each other, rearranging Martha's mind in myriad ways; as in the third image in the previous post, the connections formed by subconscious agents add to the strength of the conscious connections, altering the conscious graph like a puppeteer moves a puppet. The learning agent activates and then starts looking for the culprit. It zeroes in on the epicenter of change in the conscious subgraph: Martha's representations of color knowledge.

And there it gets stuck. No new conscious connection formed between, say, "apple" and "red", and the learning agent is only looking for information in the conscious subgraph. The learning agent goes through a dozen false starts looking for something that isn't there. Martha articulates the thought of having learned something, but has nothing to say about what that new knowledge might be, except that it's something about the colors. This new knowledge is, somehow, ineffable to her.

However, this is no contradiction to her complete factual knowledge about the brain! Martha's mind has a conscious representation of what will happen to her mental graph once she sees colors, but this doesn't itself rearrange her subconscious visual processing agents, any more than drawing a new bridge on your map of California causes its construction in reality. (The potential for confusion of levels, it seems to me, gives the Mary's Room argument much of its persuasive force.)

Martha would not be surprised at this new rearrangement (since she could model in advance this snafu with her learning agent), but her mind would nonetheless have the reaction of learning something ineffable. No amount of prior factual knowledge would suffice to prevent the learning agent from malfunctioning in this way. Furthermore, this ineffability has the same boundaries as that we wanted to investigate: when Martha digests a new food, it doesn’t strongly affect agents that are part of the conscious subgraph, so the learning-agent isn't activated; when she counts sheep, anything she learns is a trivial development of her conscious knowledge of quantity².

Since our model–which seems to be thoroughly reductionistic–gives rise to this reaction without paradox or mystery, the Mary's Room argument cannot be a logically valid argument against physical reductionism. But there's more to it than that...

Martha in the Mirror

Now that we've seen how the model gives rise to thoughts and reactions like those of human beings, why not carry the correspondence farther? There's perhaps no need for the Cartesian theater at all, if our ineffable qualia can arise from a similar interplay between vast subconscious agents and the conscious graph³. In the case of familiar qualia, we might look not at an agent which notices learning, but rather one which traces patterns of activation; as before, our conscious connections aren't enough to cause the pattern of activation by themselves, and another mental agent might well characterize this as ineffability in the same fashion.

But by focusing on the conscious thoughts of ineffability, are we neglecting the actual essence of the problem? It seems to me that once we remove the seeming paradox, the conclusion that different mental agents feel different to us is hardly a mysterious one. Your mileage may, perhaps, vary.

This is by no means a full or completed theory, but I find it promising enough as a start on the hard problem– and I think it might be helpful for those who find subjective experience a stumbling block on the way to reductionism. Thanks to several people on Less Wrong and elsewhere who've given me feedback on early versions, and thanks for all your thoughts now!

Addenda:

We should also consider the question of what we're actually doing when we think about the thought experiment, since we form our intuitions on Mary's Room well in advance of any experimental test. We're probably using our subconscious mental machinery to simulate what such a person might think and feel by empathetically feeling it ourselves, a process which is extremely useful on an evolutionary level for predicting other people's future actions, and which doesn't restrict itself to simulating only those aspects of psychology which we consciously understand⁴. (One might suspect that this ability to model other minds is the real origin of the recent arms race in brain size among hominids.)

Our subconscious model includes our conscious subgraph, the effect of a new sense experience, and the agents that recognize learning, and it rightly recognizes the feeling of ineffability that would result. But it's not designed to pass along a conscious understanding of the structure behind this feeling, so it's understandable that we naturally take it as a mysterious property of subjective experience rather than as a hiccup in our cognitive algorithms.

Finally, how much should we worry about Occam's Razor? Well, while the concepts I've introduced have taken some careful explanations, they're relatively basic entities which we could (if we wanted) program on a computer if we so chose. The phenomenon arises from the interaction of the following entities: a mental graph that rearranges upon new connections, the subgraph connected to language, and the simple learning agent. As far as cognitive science is concerned, we're being quite parsimonious.

Footnotes:

1. I'm positing that it focuses on the conscious subgraph, because there's no benefit to communicate events that only affect the agents that are useless to communicate about.

2. Come to think of it, I suppose there should be a qualia response to counting a new order of magnitude- "Ah! So that's what a crowd of 100,000 looks like." But intermediate numbers between known quantities should still be relatively qualia-free. It turns out to be really difficult to construct an example of mental processing that doesn't have a qualia aspect...

3. Our conscious/subconscious distinction may be a bit more complex than Martha's, but in the main it seems to correspond well to the boundary between what we evolved to communicate and what we did not.

4. Note that this is quite different from the conscious modeling of the human mind we posited for Mary and Martha. It's crucial, from an evolutionary perspective, that our subconscious models of human action aren't handicapped by the paltry state of our conscious notions of psychology. The model needs to actually get the right prediction to be successful.