Follow-up to: A Sketch of Good Communication

Question:

Why should you integrate an expert's model with your model at all? Haven’t you heard that people weigh their own ideas too heavily - you should just defer to them.

Here’s a quick reminder of the three step process:

And this new proposal, I think, suggests changing what you do after step 2.

The people who have the best ideas, as it seems to me, often change their plans as a result of their debates in all the other fields. Here’s three examples $^{[1]}$ of people doing this with AI timelines.

[Person 1] Oh, AI timelines? Well, I recall reading that it took evolution $10^{9}$ years to go from eukaryotes to human brains. I’d guess that human developers are about $10^{6}$ times more efficient than evolution, so I expect it to take 1000 years to get there from the point where we built computers. Which puts my date at 2956.
- If I’m wrong I’ll likely have to learn something new about developers competence relative to evolution, or about how humans get to do a type of intelligence evolution wasn’t allowed to for some reason.
[Person 2] Oh, AI timelines? Well, given my experience working on coding projects, it seems to me that projects take 50% extra time to run than you'd expect once you've got the theory down, so I'll take the date by which we should have enough hardware to build a human brain, estimate the coding work required for the necessary project, and add 1.5x time to it.
- And if you change my mind on this, it will help make my models of project time more accurate, and change how I do my job.
[Person 3] Oh, AI timelines? Well, given my basic knowledge of GDP growth rates I’d guess that being able to automate this percentage of the workforce would cause a doubling every X unit of time, which I expect for us (at current rates) only to be able to do after K years.
- If you show me I’m wrong it’ll either be because GDP is not as reliable as I think, or because I’ve made a mistake extrapolating the trend as it stands.

Can you see how a conversation between either of these two people would lead them both to learn not only about AI, but also about models of evolution/large scale coding projects/macroeconomics?

Recently, Jacob Lagerros and I were organising a paper-reading session on a recent Distill.pub paper, and Jacob was arguing for a highly-structured and detailed read-through of the paper. I wanted to focus m...