Edit: contest closed now, will start assessing the entries.
The contest
I'm offering $1,000 for good questions to ask of AI Oracles. Good questions are those that are safe and useful: that allows us to get information out of the Oracle without increasing risk.
To enter, put your suggestion in the comments below. The contest ends at the end[1] of the 31st of August, 2019.
Oracles
A perennial suggestion for a safe AI design is the Oracle AI: an AI confined to a sandbox of some sort, that interacts with the world only by answering questions.
Two of the safest designs seem to be the counterfactual Oracle, and the low bandwidth Oracle. These are detailed here, here, and here, but in short:
A counterfactual Oracle is one whose objective function (or reward, or loss function) is only non-trivial in worlds where its answer is not seen by humans. Hence it has no motivation to manipulate humans through its answer.
A low bandwidth Oracle is one that must select its answers off a relatively small list. Though this answer is a self-confirming prediction, the negative effects and potential for manipulation is restricted because there are only a few possible answers available.
Note that both of these Oracles are designed to be episodic (they are run for single episodes, get their rewards by the end of that episode, aren't asked further questions before the episode ends, and are only motivated to best perform on that one episode), to avoid incentives to longer term manipulation.
Getting useful answers
The counterfactual and low bandwidth Oracles are safer than unrestricted Oracles, but this safety comes at a price. The price is that we can no longer "ask" the Oracle any question we feel like, and we certainly can't have long discussions to clarify terms and so on. For the counterfactual Oracle, the answer might not even mean anything real to us - it's about another world, that we don't inhabit.
Despite this, its possible to get a surprising amount of good work out of these designs. To give one example, suppose we want to fund various one of a million projects on AI safety, but are unsure which one would perform better. We can't directly ask either Oracle, but there are indirect ways of getting advice:
We could ask the low bandwidth Oracle which team A we should fund; we then choose a team B at random, and reward the Oracle if, at the end of a year, we judge A to have performed better than B.
The counterfactual Oracle can answer a similar question, indirectly. We commit that, if we don't see its answer, we will select team A and team B at random and fund them for year, and compare their performance at the end of the year. We then ask for which team A[2] it expects to most consistently outperform any team B.
Both these answers get around some of the restrictions by deferring to the judgement of our future or counterfactual selves, averaged across many randomised universes.
But can we do better? Can we do more?
Your better questions
This is the purpose of this contest: for you to propose ways of using either Oracle design to get the most safe-but-useful work.
So I'm offering $1,000 for interesting new questions we can ask of these Oracles. Of this:
$350 for the best question to ask a counterfactual Oracle.
$350 for the best question to ask a low bandwidth Oracle.
$300 to be distributed as I see fit among the non-winning entries; I'll be mainly looking for innovative and interesting ideas that don't quite work.
Exceptional rewards go to those who open up a whole new category of useful questions.
Questions and criteria
Put your suggested questions in the comment below. Because of the illusion of transparency, it is better to explain more rather than less (within reason).
Comments that are submissions must be on their separate comment threads, start with "Submission", and you must specify which Oracle design you are submitting for. You may submit as many as you want; I will still delete them if I judge them to be spam. Anyone can comment on any submission. I may choose to ask for clarifications on your design; you may also choose to edit the submission to add clarifications (label these as edits).
It may be useful for you to include details of the physical setup, what the Oracle is trying to maximise/minimise/predict and what the counterfactual behaviour of the Oracle users humans are assumed to be (in the counterfactual Oracle setup). Explanations as to how your design is safe or useful could be helpful, unless it's obvious. Some short examples can be found here.
EDIT after seeing some of the answers: decide on the length of each episode, and how the outcome is calculated. The Oracle is run once an episode only (and other Oracles can't generally be used on the same problem; if you want to run multiple Oracles, you have to justify why this would work), and has to get objective/loss/reward by the end of that episode, which therefore has to be estimated in some way at that point.
A note on timezones: as long as it's still the 31 of August, anywhere in the world, your submission will be counted. ↩︎
These kind of conditional questions can be answered by a counterfactual Oracle, see the paper here for more details. ↩︎
Some assorted thoughts that might be useful for thinking about questions and answers:
a question is a schema with a blank to be filled in by the answerer after evaluation of the meaning of the question.
shared context is inferred as most questions are underspecified (domain of question, range of answers)
a few types of questions:
narrow down the field within which I have to search either by specifying a point or specifying a partition of the search space
question about specificities of variants: who where when
question about the invariants of a system: what, how
question about the backwards facing causation
question about the forward facing causation
meta questions about question schemas
what do we want a mysteriously powerful answerer to do?
zoom in on optimal points in intractably large search spaces
eg specific experiments to run to most easily invalidate major scientific questions
specify search spaces we don't know how to parameterize
eg human values
back chain from types of answers to infer taxonomy of questions
an explanation relative to a prediction:
a prediction returns the future state of the system
an explanation returns a more compact than previously held causal expalantion of the system, though it might still not generate sufficiently high resolution predictions
How to detect ontology errors using questions?
is this question malformed? if so, what are some alternative ways of framing the question that could return interesting answers
ie the implied search space of the question was not correct and can either be extended or transformed in this way
types of questions are types of search spaces
questions that change weightings on factors vs describe new factors
recogniziton of the many connection types in the human semantic network
recursive questions move along one dimension as they restrict the space. ie where is that at different spatial resolutions
qualitative and quantitative dimensions along which a query can be moved exchanges information about the implied search space
under vs overspecified questions
closed and open search spaces
navigational questions only work under the one off assumption if the search is stateless
termination guarantees
completeness guarantees
optimality guarantees
meta questions about the methods the system uses to navigate intractable spaces
time vs space vs....?
generating candidates is easy, checking is hard and vice versa
unknown metadata for answers
bias variance trade off
failure mode mapping, do failures imply directionality?
eg does a failure of a candidate change which candidate you go to next (stateful)
how to think about attack surfaces for question answer systems
can this get a human to run arbitrary code by counterfactually cooperating with itself on what step of the process it is on? Can this be tested by going through the whole process with human A then scrambling the steps and running through the same thing with human B and seeing if answers diverge?
what clever ways do hackers widen restricted bit streams?
Edit: contest closed now, will start assessing the entries.
The contest
I'm offering $1,000 for good questions to ask of AI Oracles. Good questions are those that are safe and useful: that allows us to get information out of the Oracle without increasing risk.
To enter, put your suggestion in the comments below. The contest ends at the end[1] of the 31st of August, 2019.
Oracles
A perennial suggestion for a safe AI design is the Oracle AI: an AI confined to a sandbox of some sort, that interacts with the world only by answering questions.
This is, of course, not safe in general; an Oracle AI can influence the world through the contents of its answers, allowing it to potentially escape the sandbox.
Two of the safest designs seem to be the counterfactual Oracle, and the low bandwidth Oracle. These are detailed here, here, and here, but in short:
Note that both of these Oracles are designed to be episodic (they are run for single episodes, get their rewards by the end of that episode, aren't asked further questions before the episode ends, and are only motivated to best perform on that one episode), to avoid incentives to longer term manipulation.
Getting useful answers
The counterfactual and low bandwidth Oracles are safer than unrestricted Oracles, but this safety comes at a price. The price is that we can no longer "ask" the Oracle any question we feel like, and we certainly can't have long discussions to clarify terms and so on. For the counterfactual Oracle, the answer might not even mean anything real to us - it's about another world, that we don't inhabit.
Despite this, its possible to get a surprising amount of good work out of these designs. To give one example, suppose we want to fund various one of a million projects on AI safety, but are unsure which one would perform better. We can't directly ask either Oracle, but there are indirect ways of getting advice:
Both these answers get around some of the restrictions by deferring to the judgement of our future or counterfactual selves, averaged across many randomised universes.
But can we do better? Can we do more?
Your better questions
This is the purpose of this contest: for you to propose ways of using either Oracle design to get the most safe-but-useful work.
So I'm offering $1,000 for interesting new questions we can ask of these Oracles. Of this:
Exceptional rewards go to those who open up a whole new category of useful questions.
Questions and criteria
Put your suggested questions in the comment below. Because of the illusion of transparency, it is better to explain more rather than less (within reason).
Comments that are submissions must be on their separate comment threads, start with "Submission", and you must specify which Oracle design you are submitting for. You may submit as many as you want; I will still delete them if I judge them to be spam. Anyone can comment on any submission. I may choose to ask for clarifications on your design; you may also choose to edit the submission to add clarifications (label these as edits).
It may be useful for you to include details of the physical setup, what the Oracle is trying to maximise/minimise/predict and what the counterfactual behaviour of the Oracle users humans are assumed to be (in the counterfactual Oracle setup). Explanations as to how your design is safe or useful could be helpful, unless it's obvious. Some short examples can be found here.
EDIT after seeing some of the answers: decide on the length of each episode, and how the outcome is calculated. The Oracle is run once an episode only (and other Oracles can't generally be used on the same problem; if you want to run multiple Oracles, you have to justify why this would work), and has to get objective/loss/reward by the end of that episode, which therefore has to be estimated in some way at that point.
A note on timezones: as long as it's still the 31 of August, anywhere in the world, your submission will be counted. ↩︎
These kind of conditional questions can be answered by a counterfactual Oracle, see the paper here for more details. ↩︎