Consider the clever arguer from The Bottom Line:

Now suppose there is a clever arguer, holding a sheet of paper, and they say to the owners of box A and box B: “Bid for my services, and whoever wins my services, I shall argue that their box contains the diamond, so that the box will receive a higher price.” So the box-owners bid, and box B’s owner bids higher, winning the services of the clever arguer.

The clever arguer begins to organize their thoughts. First, they write, “And therefore, box B contains the diamond!” at the bottom of their sheet of paper. Then, at the top of the paper, the clever arguer writes, “Box B shows a blue stamp,” and beneath it, “Box A is shiny,” and then, “Box B is lighter than box A,” and so on through many signs and portents; yet the clever arguer neglects all those signs which might argue in favor of box A.

This is a great example of a broken chain: the chain of cause-and-effect between the actual contents of the boxes and the arguer’s conclusion is broken. With the chain broken, no amount of clever arguing can make the conclusion at the bottom of the paper more true.

Much of the sequences can be summarized as “to determine the accuracy of a belief, look at the cause-and-effect process which produced that belief”. A few examples:

The Bottom Line
Asking “Why do I believe what I believe?”
Examining systematic flaws in our belief-producing process, i.e. biases
Noticing beliefs stemming from inaccurate sources, i.e. anchoring, fictional evidence, changing our minds less often than we think
Engines of cognition

If the causal chain from territory to map is broken outright, then that’s a pretty obvious problem, but some of the links above provide more subtle examples too. In general, map-making processes should produce accurate maps to exactly the extent that they approximate Bayesian reasoning.

Point is: looking at the cause-and-effect processes which produce maps/beliefs is pretty central to rationality.

Given all that, it’s rather odd that we don’t have a nice mathematical theory of map-making processes - cause-and-effect systems which produce accurate maps/beliefs from territories.

We have many of the pieces needed for such a theory laying around already. We have a solid theory of causality, so we know how to formalize “cause-and-effect processes”. Information theory lets us quantify map-territory correspondence. We even have an intuitive notion that accurate map-making processes should approximate Bayesian inference.

Yet there’s some large chunks missing. For instance, suppose Google collects a bunch of photos from the streets of New York City, then produces a streetmap from it. The vast majority of the information in the photos is thrown away in the process - how do we model that mathematically? How do we say that the map is “accurate”, despite throwing away all that information? More generally, maps/beliefs tend to involve some abstraction - my beliefs are mostly about macroscopic objects (trees, chairs, etc) rather than atoms. What does it mean for a map to be “accurate” at an abstract level, and what properties should my map-making process have in order to produce accurate abstracted maps/beliefs?