Law of Logical Causality: If conditioning on any event changes the probability an agent assigns to its own action, that event must be treated as causally downstream.
If I'm interpreting things correctly, this is just because anything that's upstream gets screened off, because the agent knows what action it's going to take.
You say that LICDT pays the blackmail in XOR blackmail because it follows this law of logical causality. Is this because, conditioned on the letter being sent, if there is a disaster the age... (read more)
Not quite. The agent might play a mixed strategy if there is a predictor in the
environment, e.g., when playing rock/paper/scissors with a similarly-intelligent
friend you (more or less) want to predict what you're going to do and then do
something else than that. (This is especially obvious if you assume the friend
is exactly as smart as you, IE, assigns the same probability to things if
there's no hidden information -- we can model this by supposing both of you use
the same logical inductor.) You don't know what you're going to do, because your
deliberation process is unstable: if you were leaning in any direction, you
would immediately lean in a different direction. This is what it means to be
playing a mixed strategy.
In this situation, I'm nonetheless still claiming that what's "downstream"
should be what's logically correlated with you. So what screens off everything
else is knowledge of the state of your deliberation, not the action itself. In
the case of a mixed strategy, you know that you are balanced on a razor's edge,
even though you don't know exactly which action you're taking. And you can give
a calibrated probability for that action.
I don't recall whether I've written the following up, but a while after I wrote
the OP here, I realized that LICDT/LIEDT can succeed in XOR Blackmail (failing
to send the money), but for an absolutely terrible reason.
Suppose that the disaster is sufficiently rare -- much less probable than the
exploration probability ϵ. Furthermore, suppose the exploration mechanism is
p-chicken, IE "if you're too sure of what you do, do something else." (The story
is more complicated for other exploration methods.)
Now how often does the agent respond to the letter?
Now suppose that, overall, the agent responds to the letter with frequency at
least ϵ (including rounds where the agent doesn't receive the letter). Then,
conditional on the letter being sent, the agent is pretty sure it will respond
to the letter -- it believes this wi
The statement of the law of logical causality is:
If I'm interpreting things correctly, this is just because anything that's upstream gets screened off, because the agent knows what action it's going to take.
You say that LICDT pays the blackmail in XOR blackmail because it follows this law of logical causality. Is this because, conditioned on the letter being sent, if there is a disaster the age... (read more)