All of Tetraspace's Comments + Replies

Law of Logical Causality: If conditioning on any event changes the probability an agent assigns to its own action, that event must be treated as causally downstream.

If I'm interpreting things correctly, this is just because anything that's upstream gets screened off, because the agent knows what action it's going to take.

You say that LICDT pays the blackmail in XOR blackmail because it follows this law of logical causality. Is this because, conditioned on the letter being sent, if there is a disaster the age...

2Abram Demski3y
Not quite. The agent might play a mixed strategy if there is a predictor in the environment, e.g., when playing rock/paper/scissors with a similarly-intelligent friend you (more or less) want to predict what you're going to do and then do something else than that. (This is especially obvious if you assume the friend is exactly as smart as you, IE, assigns the same probability to things if there's no hidden information -- we can model this by supposing both of you use the same logical inductor.) You don't know what you're going to do, because your deliberation process is unstable: if you were leaning in any direction, you would immediately lean in a different direction. This is what it means to be playing a mixed strategy. In this situation, I'm nonetheless still claiming that what's "downstream" should be what's logically correlated with you. So what screens off everything else is knowledge of the state of your deliberation, not the action itself. In the case of a mixed strategy, you know that you are balanced on a razor's edge, even though you don't know exactly which action you're taking. And you can give a calibrated probability for that action. I don't recall whether I've written the following up, but a while after I wrote the OP here, I realized that LICDT/LIEDT can succeed in XOR Blackmail (failing to send the money), but for an absolutely terrible reason. Suppose that the disaster is sufficiently rare -- much less probable than the exploration probability ϵ. Furthermore, suppose the exploration mechanism is p-chicken, IE "if you're too sure of what you do, do something else." (The story is more complicated for other exploration methods.) Now how often does the agent respond to the letter? Now suppose that, overall, the agent responds to the letter with frequency at least ϵ (including rounds where the agent doesn't receive the letter). Then, conditional on the letter being sent, the agent is pretty sure it will respond to the letter -- it believes this wi