An edited transcript of a long instant-messenger conversation that took place regarding the phrase, "I don't know", sparked by Robin Hanson's previous post, "You Are Never Entitled to Your Opinion."

[08:50] Eliezer: http://www.overcomingbias.com/2006/12/you_are_never_e.html

[09:01] X: it still seems that saying "i don't know" in some situations is better than giving your best guess

[09:01] X: especially if you are dealing with people who will take you at your word who are not rationalists

[09:02] Eliezer: in real life, you have to choose, and bet, at some betting odds

[09:02] Eliezer: i.e., people who want to say "I don't know" for cryonics still have to sign up or not sign up, and they'll probably do the latter

[09:03] Eliezer: "I don't know" is usually just a screen that people think is defensible and unarguable before they go on to do whatever they feel like, and it's usually the wrong thing because they refused to admit to themselves what their guess was, or examine their justifications, or even realize that they're guessing

[09:02] X: how many apples are in a tree outside?

[09:02] X: i've never seen it and neither have you

[09:02] Eliezer: 10 to 1000

[09:04] Eliezer: if you offer to bet me a million dollars against one dollar that the tree outside has fewer than 20 apples, when neither of us have seen it, I will take your bet

[09:04] X: is it better to say "maybe 10 to 1000" to make it clear that you are guessing when talking to people

[09:04] Eliezer: therefore I have assigned a nonzero and significant probability to apples < 20 whether I admit it or not

[09:05] Eliezer: what you *say* is another issue, especially when speaking to nonrationalists, and then it is well to bear in mind that words don't have fixed meanings; the meaning of the sounds that issue from your lips is whatever occurs in the mind of the listener. If they're going to misinterpret something then you shouldn't say it to them no matter what the words mean inside your own head

[09:06] Eliezer: often you are just screwed unless you want to go back and teach them rationality from scratch, and in a case like that, all you can do is say whatever creates the least inaccurate image

[09:06] X: 10 to 1000 is misleading when you say it to a nonrationalist?

[09:06] Eliezer: "I don't know" is a good way to duck when you say it to someone who doesn't know about probability distributions

[09:07] Eliezer: if they thought I was certain, or that my statement implied actual knowledge of the tree

[09:07] Eliezer: then the statement would mislead them

[09:07] Eliezer: and if I knew this, and did it anyway for my own purposes, it would be a lie

[09:08] Eliezer: if I just couldn't think of anything better to say, then it would be honest but not true, if you can see the distinction

[09:08] Eliezer: honest for me, but the statement that formed in their minds would still not be true

[09:09] X: most people will say to you.... but you said....10-1000 apples

[09:09] Eliezer: then you're just screwed

[09:10] Eliezer: nothing you can do will create in their minds a true understanding, not even "I don't know"

[09:10] X: why bother, why not say i don't know?

[09:10] Eliezer: honesty therefore consists of misleading them the least and telling them the most

[09:10] X: it's better than misleading them with 10-1000

[09:10] Eliezer: as for "why bother", well, if you're going to ask that question, just don't reply to their email or whatever

[09:11] Eliezer: what if you're dealing with someone who thinks my saying "I don't know" is a license for them to make up their own ideas, which will be a lot worse?

[09:11] X: they may act on your guess, and then say "but you said...." and lose money or get in trouble or have less respect for you

[09:11] Eliezer: then you choose to wave them off

[09:11] Eliezer: with "I don't know"

[09:11] Eliezer: but it's for your own sake, not for their sake

[09:12] X: [09:11] Eliezer: what if you're dealing with someone who thinks my saying "I don't know" is a license for them to make up their own ideas, which will be a lot worse?

[09:12] X: here i could see why

[09:12] X: but it's difficult working with typical people in the real world

[09:13] Eliezer: the first thing to decide is, are you trying to accomplish something for yourself (like not getting in trouble) or are you trying to improve someone else's picture of reality

[09:13] Eliezer: "I don't know" is often a good way of not getting in trouble

[09:13] Eliezer: as for it being difficult to work with people in the real world, well, yeah

[09:13] X: if you say...10-1000, and you are wrong, and they are mad, then you say, i don't know, they will be even madder

[09:13] Eliezer: are you trying not to get in trouble?

[09:14] Eliezer: or are you trying to improve their picture of reality?

[09:14] Eliezer: these are two different tasks

[09:14] X: especially if they have lost money or have been proven wrong by someone else

[09:14] Eliezer: if they intersect you have to decide what your tradeoff is

[09:14] Eliezer: which is more important to you

[09:14] Eliezer: then decide whether to explain for their benefit or say "I don't know" for yours

[09:15] X: well, if it was my job, i would say i don't know rather than be wrong, because who knows what your boss will do after he loses money listening to you

[09:15] Eliezer: okay

[09:16] Eliezer: just be clear that this is not because "I don't know" is the rational judgment, but because "I don't know" is the political utterance

[09:16] X: he may take your guess, and try to turn it into an actual anwser because no one around you has a better plan

[09:17] Eliezer: you can easily see this by looking at your stated reason: you didn't talk about evidence and reality and truth, but, how you might get in trouble based on someone's reaction

[09:17] X: yes

[09:17] X: that's what you have to put up with in the real world

[09:17] Eliezer: if you're really worried about your boss's welfare then you should consider that if you say "I don't know" he must do something anyway - refusing to choose is also a choice, and refusing to act is like refusing to let time pass - and he will construct that plan based on some information, which doesn't include your information

[09:18] Eliezer: if your life isn't worth more than someone else's, neither it is worth any less, and it is often proper to let fools make their own mistakes

[09:18] Eliezer: you can only throw yourself in front of so many bullets before you run out of flesh to stop them with

[09:19] X: ?

[09:19] Eliezer: in other words, you cannot always save people from themselves

[09:23] Eliezer: but all of this is wandering away from the original point, which is true and correct, that no one is ever entitled to their own opinion

[09:26] X: what is his name?

[09:26] Eliezer: ?

[09:26] X: a man outside

[09:26] X: random guy

[09:26] Eliezer: It's probably not "Xpchtl Vaaaaaarax"

[09:26] X: probably not

[09:27] Eliezer: I suppose I could construct a second-order Markov transition diagram for the letters in names expressed in English, weighted by their frequency

[09:27] Eliezer: but that would be a lot of work

[09:28] Eliezer: so I could say "I don't know" as shorthand for the fact that, although I possess a lot of knowledge about possible and probable names, I don't know anything *more* than you do

[09:28] X: ok, so you say ruling out what you see as likely not correct is ok?

[09:28] Eliezer: what I'm saying is that I possess a large amount of knowledge about possible names

[09:28] Eliezer: all of which influences what I would bet on

[09:28] Eliezer: if I had to take a real-world action, like, guessing someone's name with a gun to my head

[09:29] Eliezer: if I had to choose it would suddenly become very relevant that I knew Michael was one of the most statistically common names, but couldn't remember for which years it was the most common, and that I knew Michael was more likely to be a male name than a female name

[09:29] Eliezer: if an alien had a gun to its head, telling it "I don't know" at this point would not be helpful

[09:29] Eliezer: because there's a whole lot I know that it doesn't

[09:30] X: ok

[09:33] X: what about a question for which you really don't have any information?

[09:33] X: like something only an alien would know

[09:34] Eliezer: if I have no evidence I use an appropriate Ignorance Prior, which distributes probability evenly across all possibilities, and assigns only a very small amount to any individual possibility because there are so many

[09:35] Eliezer: if the person I'm talking to already knows to use an ignorance prior, I say "I don't know" because we already have the same probability distribution and I have nothing to add to that

[09:35] Eliezer: the ignorance prior tells me my betting odds

[09:35] Eliezer: it governs my choices

[09:35] X: and what if you don't know how to use an ignorance prior

[09:36] X: have never heard of it etc

[09:36] Eliezer: if I'm dealing with someone who doesn't know about ignorance priors, and who is dealing with the problem by making up this huge elaborate hypothesis with lots of moving parts and many places to go wrong, then the truth is that I automatically know s/he's wrong

[09:36] Eliezer: it may not be possible to explain this to them, short of training them from scratch in rationality

[09:36] Eliezer: but it is true

[09:36] Eliezer: and if the person trusts me for a rationalist, it may be both honest and helpful to tell them, "No, that's wrong"

[09:36] X: what if that person says, "i don't know what their name is", that ok?

[09:37] Eliezer: in real life you cannot choose "I don't know", it's not an option on your list of available actions

[09:37] Eliezer: in real life it's always, "I don't know, so I'm going to say Vkktor Blackdawn because I think it sounds cool"

[09:39] Eliezer: Vkktor Blackdawn is as (im)probable as anything else, but if you start assigning more probability to it than the ignorance prior calls for - because it sounds cool, because you don't have room in your mind for more than one possibility, or because you've started to construct an elaborate mental explanation of how the alien might end up named Vkktor Blackdawn

[09:39] Eliezer: then I know better

[09:40] Eliezer: and if you trust me, I may be able to honestly and usefully tell you so

[09:40] Eliezer: rather than saying "I don't know", which is always something to say, not to think

[09:40] Eliezer: this is important if someone asks you, "At what odds would you bet that the alien is named Vkktor Blackdawn?"

[09:41] Eliezer: or if you have to do anything else, based on your guesses and the weight you assign to them

[09:41] Eliezer: which is what probability is all about

[09:41] X: and if they say "I don't know, I don't know anything about probability"?

[09:41] Eliezer: then either they trust me blindly or I can't help them

[09:41] Eliezer: that's how it goes

[09:41] Eliezer: you can't always save people from themselves

[09:42] X: trust you blindly about what you are saying or about your guess as to what the alien's name is?

[09:42] Eliezer: trust me blindly when I tell them, "Don't bet at those odds."