AI ALIGNMENT FORUM
AF

Iterated Amplification
Iterated Amplification
Frontpage

9

Supervising strong learners by amplifying weak experts

by Paul Christiano
6th Jan 2019
1 min read
1

9

This is a linkpost for https://arxiv.org/pdf/1810.08575.pdf
Iterated Amplification
Frontpage
Previous:
Factored Cognition
1 comments45 karma
Next:
AlphaGo Zero and capability amplification
16 comments33 karma
Log in to save where you left off
New Comment
Moderation Log
Curated and popular this week
0Comments

Abstract

Many real world learning tasks involve complex or hard-to-specify objectives, and using an easier-to-specify proxy can lead to poor performance or misaligned behavior. One solution is to have humans provide a training signal by demonstrating or judging performance, but this approach fails if the task is too complicated for a human to directly evaluate. We propose Iterated Amplification, an alternative training strategy which progressively builds up a training signal for difficult problems by combining solutions to easier subproblems. Iterated Amplification is closely related to Expert Iteration (Anthony et al., 2017; Silver et al., 2017b), except that it uses no external reward function. We present results in algorithmic environments, showing that Iterated Amplification can efficiently learn complex behaviors.

Tomorrow's AI Alignment Forum sequences post will be 'AI safety without goal-directed behavior' by Rohin Shah, in the sequence on Value Learning.

The next post in this sequence on Iterated Amplification will be 'AlphaGo Zero and capability amplification', by Paul Christiano, on Tuesday 8th January.

Mentioned in
47AI Alignment 2018-19 Review
22[AN #135]: Five properties of goal-directed systems
14[AN #146]: Plausible stories of how we might fail to avert an existential catastrophe
8Alignment Newsletter #40
14[AN #123]: Inferring what is valuable in order to align recommender systems
Load More (5/6)