We put together a survey to study the opinions of timelines and probability of human extinction of the employees at Conjecture. The questions were based on previous public surveys and prediction markets, to ensure that the results are comparable with people’s opinions outside of Conjecture.

The survey results were polled in April, 2023. There were 23 unique responses from people across teams.

Section 1. Probability of human extinction from AI

Setup and limitations

The specific questions the survey asked were:

What probability do you put on human inability to control future advanced A.I. systems causing human extinction or similarly permanent and severe disempowerment of the human species?
What probability do A.I. systems causing human extinction or similarly permanent and severe disempowerment of the human species in general (not just because inability to control, but also stuff like people intentionally using AI systems in harmful ways)?

The difference between the two questions is that the first focuses on risk from misalignment, whereas the second captures risk from misalignment and misuse.

The main caveats of these questions are the following:

The questions were not explicitly time bound. I'd expect differences in people’s estimates of risk of extinction this century, in the next 1000 years, and anytime in the future. The longer of a timeframe we consider, the higher the values would be. I suspect employees were considering extinction risk roughly within this century when answering.
The first question is a subset of the second question. One employee gave a higher probability for the second question than the first; this was probably a misinterpretation.
The questions factor in interventions such as how Conjecture and others’ safety work will impact extinction risk. The expectation is the numbers would be higher if factored out their own or others’ safety work.

Responses

Out of the 23 respondents, one rejected the premise, and two people did not respond to one of the two questions but answered the other one. The main issue respondents raised was answering without a time constraint.

**Figure 1.** Probability of human extinction from autonomous AI. N = 22. No one reported less than a 10% chance. Most people reported over a 60% chance.

Generally, people estimate the extinction risk from autonomous AI / AI getting out of control to be quite high at Conjecture. The median estimati...

Posts

Wikitag Contributions

Comments