LA-602: Ignition of the Atmosphere with Nuclear Bombs, a research report from the Manhattan Project, is to the best of my knowledge the first technical analysis ever conducted of an uncertain danger of a human-caused extinction catastrophe.

Previously, Teller and Konopinski had been assigned the task of disproving a crazy suggestion by Enrico Fermi that a fission chain reaction could ignite a thermonuclear reaction in deuterium - what we now know as an H-Bomb. Teller and Konopinski found that, contrary to their initial skepticism, the hydrogen bomb appeared possible.

Good for their rationality! Even though they started with the wrong conclusion on their bottom line, they were successfully forced away from it by arguments that could only support one answer.

Still, in retrospect, I think that the advice the future would give to the past, would be: Start by sitting down and saying, "We don't know if a hydrogen bomb is possible". Then list out the evidence and arguments; then at the end weigh it.

So the hydrogen bomb was possible. Teller then suggested that a hydrogen bomb might ignite a self-sustaining thermonuclear reaction in the nitrogen of Earth's atmosphere. This also appeared extremely unlikely at a first glance, but Teller and Konopinski and Marvin investigated, and wrote LA-602...

As I understand LA-602, the authors went through the math and concluded that there were several strong reasons to believe that nitrogen fusion could not be self-sustaining in the atmosphere: it would take huge energies to start the reaction at all; the reaction would lose radiation from its surface too fast to sustain the fusion temperature; and even if the fusion reaction did grow, the Compton effect would increase radiation losses with volume(?).

And we're still here; so the math, whatever it actually says, seems to have been right.

Note that the Manhattan scientists didn't always get their math right. The Castle Bravo nuclear test on March 1, 1954 produced 15 megatons instead of the expected 4-8 megatons due to an unconsidered additional nuclear reaction that took place in lithium-7. The resulting fallout contaminated fishing boats outside the declared danger zone; at least one person seems to have died.

But the LA-602 calculations were done with very conservative assumptions, and came out with plenty of safety margin. AFAICT (I am not a physicist) a Castle Bravo type oversight could not realistically have made the atmosphere ignite anyway, and if it did, it'd have gone right out, etc.

The last time I know of when a basic physical calculation with that much safety margin, and multiple angles of argument, turned out to be wrong anyway, was when Lord Kelvin showed from multiple angles of reasoning that the Earth could not possibly be so much as a hundred million years old.

LA-602 concludes:

"There remains the distinct possibility that some other less simple mode of burning may maintain itself in the atmosphere... the complexity of the argument and the absence of satisfactory experimental foundations makes further work on the subject highly desirable."

Decades after LA-602, another paper would be written to analyze an uncertain danger of human-created existential risk: The Review of Speculative "Disaster Scenarios" at RHIC.

The RHIC Review was written in response to suggestions that the Relativistic Heavy Ion Collider might create micro black holes or strangelets.

A B.Sc. thesis by Shameer Shah of MIT, Perception of Risk: Disaster Scenarios at Brookhaven, chronicles the story behind the RHIC Review:

The RHIC flap began when Walter Wagner wrote to Scientific American, speculating that the Brookhaven collider might create a "mini black hole". A reply letter by Frank Wilczek of the Institute for Advanced Studies labeled the mini-black-hole scenario as impossible, but also introduced a new possibility, negatively charged strangelets, which would convert normal matter into more strange matter. Wilczek considered this possibility slightly more plausible.

Then the media picked up the story.

Shameer Shah interviewed (on Nov 22, 2002) Robert Jaffe, Director of MIT's Center for Theoretical Physics, a pioneer in the theory of strange matter, and primary author of the RHIC Review.

According to Jaffe, even before the investigative committe was convened, "No scientist who understood the physics thought that this experiment posed the slightest threat to anybody." Then why have the committee in the first place? "It was an attempt to take seriously the fears of science that they don't understand." Wilczek was asked to serve on the committee "to pay the wages of his sin, since he's the one that started all this with his letter."

Between LA-602 and the RHIC Review there is quite a difference of presentation.

I mean, just look at the names:

LA-602: Ignition of the Atmosphere with Nuclear Bombs

Review of Speculative "Disaster Scenarios" at RHIC

See a difference?

LA-602 began life as a classified report, written by scientists for scientists. You're assumed to be familiar with the meaning of terms like Bremsstrahlung, which I had to look up. LA-602 does not begin by asserting any conclusions; the report walks through the calculations - at several points clearly labeling theoretical extrapolations and unexplored possibilities as such - and finally concludes that radiation losses make self-sustaining nitrogen fusion impossible-according-to-the-math, even under the most conservative assumptions.

The RHIC Review presents a nontechnical summary of its conclusions in six pages at the start, relegating the math and physics to eighteen pages of appendices.

LA-602 concluded, "There remains the distinct possibility that some other less simple mode of burning may maintain itself in the atmosphere..."

The RHIC Review concludes: "Our conclusion is that the candidate mechanisms for catastrophic scenarios at RHIC are firmly excluded by existing empirical evidence, compelling theoretical arguments, or both. Accordingly, we see no reason to delay the commissioning of RHIC on their account."

It is not obvious to my inexpert eyes that the assumptions in the RHIC Review are any more firm than those in LA-602 - they both seem very firm - but the two papers arise from rather different causes.

To put it bluntly, LA-602 was written by people curiously investigating whether a hydrogen bomb could ignite the atmosphere, and the RHIC Review is a work of public relations.

Now, it does seem - so far as I can tell - that it's pretty damned unlikely for a particle accelerator far less powerful than random cosmic rays to destroy Earth and/or the Universe.

But I don't feel any more certain of that after reading the RHIC Review than before I read it. I am not a physicist; but if I was a physicist, and I read a predigested paper like the RHIC Review instead of doing the whole analysis myself from scratch, I would be fundamentally trusting the rationality of the paper's authors. Even if I checked the math, I would still be trusting that the equations I saw were the right equations to check. I would be trusting that someone sat down, and looked for unpleasant contrary arguments with an open mind, and really honestly didn't find anything.

When I contrast LA-602 to the RHIC Review, well...

Don't get me wrong: I don't feel the smallest particle of real fear about particle accelerators. The basic cosmic-ray argument seems pretty convincing. Nature seems to have arranged for the calculations in this case to have some pretty large error margins. I, myself, am not going to worry about risks we can actually calculate to be tiny, when there are incalculable large-looking existential risks to soak up my concern.

But there is something else that I do worry about: The primary stake on the table with things like RHIC, is that it is going to get scientists into the habit of treating existential risk as a public relations issue, where ignorant technophobes say the risk exists, and the job of scientists is to interface with the public and explain to them that it does not.

Everyone knew, before the RHIC report was written, what answer it was supposed to produce. That is a very grave matter. Analysis is what you get when physicists sit down together and say, "Let us be curious," and walk through all the arguments they can think of, recording them as they go, and finally weigh them up and reach a conclusion. If this does not happen, no analysis has taken place.

The general rule of thumb I sometimes use, is that - because the expected utility of thought arises from the utility of what is being reasoned about - a single error in analyzing an existential risk, even if it "doesn't seem like it ought to change the conclusion", is worth at least one human life.

The RHIC Review is not written to the standard of care that would be appropriate if, after the RHIC Review was written, some higher authority went through the paper; and if a single argument in it was wrong, anywhere, whether or not it changed the conclusion, a hostage got shot. That's how to think about analyzing existential risks. That way, for each and every element of the analysis, you can find it in yourself to be a little uncertain about that element, even if it doesn't seem "like it could possibly change the conclusion"; uncertainty invokes curiosity.

The RHIC Review was produced by authors who were already sure that the RHIC couldn't destroy the Earth, the problem-at-hand was explaining this to the public. If the authors decided just by eyeballing the problem that the RHIC couldn't destroy the Earth, then the only actual analysis that took place was conducted in 5 seconds. Yes, it's a lopsided issue, but it seems that as a general matter of policy, any existential risk at all deserves a longer and truly curious analysis than that.

Though I don't really blame the RHIC Review's authors. No one ever told them that there was such a thing as existential risk, or that it raised the standards of analysis beyond what was usual in a scientific paper, or that rational analysis requires placing yourself into a state of genuine uncertainty about each individual element's exact value...

And the much greater reason I don't blame them, is that between the 1940s and today, society has developed a "Gotcha!" attitude toward risk.

You can't admit a single particle of uncertain danger if you want your science's funding to survive. These days you are not allowed to end by saying, "There remains the distinct possibility..." Because there is no debate you can have about tradeoffs between scientific progress and risk. If you get to the point where you're having a debate about tradeoffs, you've lost the debate. That's how the world stands, nowadays.

So no one can do serious analysis of existential risks anymore, because just by asking the question, you're threatening the funding of your whole field.

The number one lesson I take from this whole issue is that where human-caused uncertain existential dangers are concerned, the only way to get a real, serious, rational, fair, evenhanded assessment of the risks, in our modern environment,

Is if the whole project is classified, the paper is written for scientists without translation, and the public won't get to see the report for another fifty years.

This is the lesson of LA-602: Ignition of the Atmosphere with Nuclear Bombs and the Review of Speculative "Disaster Scenarios" at RHIC. Read them and weep.