In "Principles of Disagreement," Eliezer Yudkowsky shared the following anecdote:

Nick Bostrom and I once took a taxi and split the fare. When we counted the money we'd assembled to pay the driver, we found an extra twenty there.

"I'm pretty sure this twenty isn't mine," said Nick.

"I'd have been sure that it wasn't mine either," I said.

"You just take it," said Nick.

"No, you just take it," I said.

We looked at each other, and we knew what we had to do.

"To the best of your ability to say at this point, what would have been your initial probability that the bill was yours?" I said.

"Fifteen percent," said Nick.

"I would have said twenty percent," I said.

I have left off the ending to give everyone a chance to think about this problem for themselves. How would you have split the twenty?

In general, EY and NB disagree about who deserves the twenty. EY believes that EY deserves it with probability p, while NB believes that EY deserves it with probability q. They decide to give EY a fraction of the twenty equal to f(p,q). What should the function f be?

In our example, p=1/5 and q=17/20

Please think about this problem a little before reading on, so that we do not miss out on any original solutions that you might have come up with.

I can think of 4 ways to solve this problem. I am attributing answers to the person who first proposed that dollar amount, but my reasoning might not reflect their reasoning.

f=p/(1+p-q) or $11.43 (Eliezer Yodkowsky/Nick Bostrom) -- EY believes he deserves p of the money, while NB believes he deserves 1-q. They should therefore be given money in a ratio of p:1-q.
f=(p+q)/2 or $10.50 (Marcello) -- It seems reasonable to assume that there is a 50% chance that EY reasoned properly and a 50% chance that NB reasoned properly, so we should take the average of the amounts of money that EY would get under these two assumptions.
f=sqrt(pq)/(sqrt(pq)+sqrt((1-p)(1-q))) or $10.87 (GreedyAlgorithm) -- We want to chose an f so that log(f/(1-f)) is the average of log(p/(1-p)) and log(q/(1-q)).
f=pq/(pq+(1-p)(1-q)) or $11.72 -- We have two observations that EY deserves the money with probability p and probability q respectively. If we assume that these are two independent pieces of evidence as to whether or not EY should get the money, then starting with equal likelihood of each person deserving the money, we should do a Bayesian update for each piece of information.

I am very curious about this question, so if you have any opinions, please comment. I have some opinions on this problem, but to avoid biasing anyone, I will save them for the comments. I am actually more interested in the following question. I believe that the two will have the same answer, but if anyone disagrees, let me know.

I have two hypotheses, A and B. I assign probability p to A and probability q to B. I later find out that A and B are equivalent. I then update to assign the probability g(p,q) to both hypotheses. What should the function g be?