Context for LW audience: Ramana, Steve and John regularly talk about stuff in the general cluster of agency, abstraction, optimization, compression, purpose, representation, etc. We decided to write down some of our discussion and post it here. This is a snapshot of us figuring stuff out together.

Hooks from Ramana:

Where does normativity come from?
Two senses of "why" (from Dennett): How come? vs What for? (The latter is more sophisticated, and less resilient. Does it supervene on the former?)
An optimisation process is something that produces/selects things according to some criterion. The products of an optimisation process will have some properties related to the optimisation criterion, depending on how good the process is at finding optimal products.
- The products of an optimisation process may or may not themselves be optimisers (i.e. be a thing that runs an optimisation process itself), or may have goals themselves. But neither of these are necessary.
- Things get interesting when some optimisation process (with a particular criterion) is producing products that are optimisers or have goals. Then we can start looking at what the relationship is between the goals of the products, or the optimisation criteria of the products, vs the optimisation criterion of the process that produced them.
If you’re modeling “having mental content” as having a Bayesian network, at some point I think you’ll run into the question of where the (random) variables come from. I worry that the real-life process of developing mental content mixes up creating variables with updating beliefs a lot more than the Bayesian network model lets on.

A central question regarding normativity for me is “Who/what is doing the enforcing?”, “What kind of work goes into enforcing?”

Also to clarify, by normativity I was trying to get at the relationship between some content and the thing it represents. Like, there’s a sense of the content is “supposed to” track or be like the thing it represents. There’s a normative standard on the content. It can be wrong, it can be corrected, etc. It can’t just be. If it were just being, which is how things presumably start out, it wouldn’t be representing.

Intrinsic Purpose vs Purpose Grounded in Evolution

Steve

As you know, I totally agree that mental content is normative - this was a hard lesson for philosophers to swallow, or at least the ones that tried...

Posts

Wikitag Contributions

Comments