Followup to: Cascades, Cycles, Insight, Recursion, Magic
Reply to: Engelbart As Ubertool?

When Robin originally suggested that Douglas Engelbart, best known as the inventor of the computer mouse, would have been a good candidate for taking over the world via compound interest on tools that make tools, my initial reaction was "What on Earth? With a mouse?"

On reading the initial portions of Engelbart's "Augmenting Human Intellect: A Conceptual Framework", it became a lot clearer where Robin was coming from.

Sometimes it's hard to see through the eyes of the past. Engelbart was a computer pioneer, and in the days when all these things were just getting started, he had a vision of using computers to systematically augment human intelligence. That was what he thought computers were for. That was the ideology lurking behind the mouse. Something that makes its users smarter - now that sounds a bit more plausible as an UberTool.

Looking back at Engelbart's plans with benefit of hindsight, I see two major factors that stand out:

Engelbart committed the Classic Mistake of AI: underestimating how much cognitive work gets done by hidden algorithms running beneath the surface of introspection, and overestimating what you can do by fiddling with the visible control levers.
Engelbart anchored on the way that someone as intelligent as Engelbart would use computers, but there was only one of him - and due to point 1 above, he couldn't use computers to make other people as smart as him.

To start with point 2: They had more reverence for computers back in the old days. Engelbart visualized a system carefully designed to flow with every step of a human's work and thought, assisting every iota it could manage along the way. And the human would be trained to work with the computer, the two together dancing a seamless dance.

And the problem with this, was not just that computers got cheaper and that programmers wrote their software more hurriedly.

There's a now-legendary story about the Windows Vista shutdown menu, a simple little feature into which 43 different Microsoft people had input. The debate carried on for over a year. The final product ended up as the lowest common denominator - a couple of hundred lines of code and a very visually unimpressive menu.

So even when lots of people spent a tremendous amount of time thinking about a single feature of the system - it still didn't end up very impressive. Jef Raskin could have done better than that, I bet. But Raskins and Engelbarts are rare.

You see the same effect in Eric Drexler's chapter on hypertext in Engines of Creation: Drexler imagines the power of the Web to... use two-way links and user annotations to promote informed criticism. (As opposed to the way we actually use it.) And if the average Web user were Eric Drexler, the Web probably would work that way by now.

But no piece of software that has yet been developed, by mouse or by Web, can turn an average human user into Engelbart or Raskin or Drexler. You would very probably have to reach into the brain and rewire neural circuitry directly; I don't think any sense input or motor interaction would accomplish such a thing.

Which brings us to point 1.

It does look like Engelbart was under the spell of the "logical" paradigm that prevailed in AI at the time he made his plans. (Should he even lose points for that? He went with the mainstream of that science.) He did not see it as an impossible problem to have computers help humans think - he seems to have underestimated the difficulty in much the same way that the field of AI once severely underestimated the work it would take to make computers themselves solve cerebral-seeming problems. (Though I am saying this, reading heavily between the lines of one single paper that he wrote.) He talked about how the core of thought is symbols, and speculated on how computers could help people manipulate those symbols.

I have already said much on why people tend to underestimate the amount of serious heavy lifting that gets done by cognitive algorithms hidden inside black boxes that run out of your introspective vision, and overestimating what you can do by duplicating the easily visible introspective control levers. The word "apple", for example, is a visible lever; you can say it or not say it, its presence or absence is salient. The algorithms of a visual cortex that let you visualize what an apple would look like upside-down - we all have these in common, and they are not introspectively accessible. Human beings knew about apples a long, long time before they knew there was even such a thing as the visual cortex, let alone beginning to unravel the algorithms by which it operated.

Robin Hanson asked me:

"You really think an office worker with modern computer tools is only 10% more productive than one with 1950-era non-computer tools? Even at the task of creating better computer tools?"

But remember the parable of the optimizing compiler run on its own source code - maybe it makes itself 50% faster, but only once; the changes don't increase its ability to make future changes. So indeed, we should not be too impressed by a 50% increase in office worker productivity - not for purposes of asking about FOOMs. We should ask whether that increase in productivity translates into tools that create further increases in productivity.

And this is where the problem of underestimating hidden labor, starts to bite. Engelbart rhapsodizes (accurately!) on the wonders of being able to cut and paste text while writing, and how superior this should be compared to the typewriter. But suppose that Engelbart overestimates, by a factor of 10, how much of the intellectual labor of writing goes into fighting the typewriter. Then because Engelbart can only help you cut and paste more easily, and cannot rewrite those hidden portions of your brain that labor to come up with good sentences and good arguments, the actual improvement he delivers is a tenth of what he thought it would be. An anticipated 20% improvement becomes an actual 2% improvement. k way less than 1.

This will hit particularly hard if you think that computers, with some hard work on the user interface, and some careful training of the humans, ought to be able to help humans with the type of "creative insight" or "scientific labor" that goes into inventing new things to do with the computer. If you thought that the surface symbols were where most of the intelligence resided, you would anticipate that computer improvements would hit back hard to this meta-level, and create people who were more scientifically creative and who could design even better computer systems.

But if really you can only help people type up their ideas, while all the hard creative labor happens in the shower thanks to very-poorly-understood cortical algorithms - then you are much less like neutrons cascading through uranium, and much more like an optimizing compiler that gets a single speed boost and no more. It looks like the person is 20% more productive, but in the aspect of intelligence that potentially cascades to further improvements they're only 2% more productive, if that.

(Incidentally... I once met a science-fiction author of a previous generation, and mentioned to him that the part of my writing I most struggled with, was my tendency to revise and revise and revise things I had already written, instead of writing new things. And he said, "Yes, that's why I went back to the typewriter. The word processor made it too easy to revise things; I would do too much polishing, and writing stopped being fun for me." It made me wonder if there'd be demand for an author's word processor that wouldn't let you revise anything until you finished your first draft.

But this could be chalked up to the humans not being trained as carefully, nor the software designed as carefully, as in the process Engelbart envisioned.)

Engelbart wasn't trying to take over the world in person, or with a small group. Yet had he tried to go the UberTool route, we can reasonably expect he would have failed - that is, failed at advancing far beyond the outside world in internal computer technology, while selling only UberTool's services to outsiders.

Why? Because it takes too much human labor to develop computer software and computer hardware, and this labor cannot be automated away as a one-time cost. If the world outside your window has a thousand times as many brains, a 50% productivity boost that only cascades to a 10% and then a 1% additional productivity boost, will not let you win against the world. If your UberTool was itself a mind, if cascades of self-improvement could fully automate away more and more of the intellectual labor performed by the outside world - then it would be a different story. But while the development path wends inexorably through thousands and millions of engineers, and you can't divert that path through an internal computer, you're not likely to pull far ahead of the world. You can just choose between giving your own people a 10% boost, or selling your product on the market to give lots of people a 10% boost.

You can have trade secrets, and sell only your services or products - many companies follow that business plan; any company that doesn't sell its source code does so. But this is just keeping one small advantage to yourself, and adding that as a cherry on top of the technological progress handed you by the outside world. It's not having more technological progress inside than outside.

If you're getting most of your technological progress handed to you - your resources not being sufficient to do it in-house - then you won't be able to apply your private productivity improvements to most of your actual velocity, since most of your actual velocity will come from outside your walls. If you only create 1% of the progress that you use, then a 50% improvement becomes a 0.5% improvement. The domain of potential recursion and potential cascades is much smaller, diminishing k. As if only 1% of the uranium generating your neutrons, were available for chain reactions to be fissioned further.

We don't live in a world that cares intensely about milking every increment of velocity out of scientific progress. A 0.5% improvement is easily lost in the noise. Corporations and universities routinely put obstacles in front of their internal scientists that cost them more than 10% of their potential. This is one of those problems where not everyone is Engelbart (and you can't just rewrite their source code either).

For completeness, I should mention that there are generic obstacles to pulling an UberTool. Warren Buffett has gotten a sustained higher interest rate than the economy at large, and is widely believed to be capable of doing so indefinitely. In principle, the economy could have invested hundreds of billions of dollars as soon as Berkshire Hathaway had a sufficiently long track record to rule out chance. Instead, Berkshire has grown mostly by compound interest. We could live in a world where asset allocations were ordinarily given as a mix of stocks, bonds, real estate, and Berkshire Hathaway. We don't live in that world for a number of reasons: financial advisors not wanting to make themselves appear irrelevant, strange personal preferences on the part of Buffett...

The economy doesn't always do the obvious thing, like flow money into Buffett until his returns approach the average return of the economy. Interest rate differences much higher than 0.5%, on matters that people care about far more intensely than Science, are ignored if they're not presented in exactly the right format to be seized.

And it's not easy for individual scientists or groups to capture the value created by scientific progress. Did Einstein die with 0.1% of the value that he created? Engelbart in particular doesn't seem to have tried to be Bill Gates, at least not as far as I know.

With that in mind - in one sense Engelbart succeeded at a good portion of what he actually set out to do: computer mice did take over the world.

But it was a broad slow cascade that mixed into the usual exponent of economic growth. Not a concentrated fast FOOM. To produce a concentrated FOOM, you've got to be able to swallow as much as possible of the processes driving the FOOM into the FOOM. Otherwise you can't improve those processes and you can't cascade through them and your k goes down. Then your interest rates won't even be as much higher than normal as, say, Warren Buffett's. And there's no grail to be won, only profits to be made: If you have no realistic hope of beating the world, you may as well join it.