Dweomite

Rather than talking about reversibility, can this situation be described just by saying that the probability of certain opportunities is zero?  For example, if John and David somehow know in advance that no one will ever offer them pepperoni in exchange for anchovies, then the maximum amount of probability mass that can be shifted from mushrooms to pepperoni by completing their preferences happens to be zero.  This doesn't need to be a physical law of anchovies; it could just be a characteristic of their trade partners.

But in this hypothetical, t... (read more)

I agree that figuring out what you "should have" precommitted can be fraught.

One possible response to that problem is to set aside some time to think about hypotheticals and figure out now what precommitments you would like to make, instead of waiting for those scenarios to actually happen.  So the perspective is "actual you, at this exact moment".

I sometimes suspect you could view MIRI's decision theories as an example of this strategy.

Alice:  Hey, Bob, have you seen this "Newcomb's problem" thing?

Bob:  Fascinating.  As we both have un

Suppose you run your twins scenario, and the twins both defect.  You visit one of the twins to discuss the outcome.

Consider the statement:  "If you had cooperated, your twin would also have cooperated, and you would have received $1M instead of $1K."  I think this is formally provable, given the premises.

Now consider the statement:  "If you had cooperated, your twin would still have defected, and you would have received $0 instead of $1K."  I think this is also formally provable, given the premises.  Because we have assumed a ... (read more)