AI ALIGNMENT FORUMThe Shard Theory of Human Values
AF

The Shard Theory of Human Values

Written by Quintin Pope, Alex Turner, Charles Foster, and Logan Smith. Card image generated by DALL-E 2:

53Humans provide an untapped wealth of evidence about alignment

Alex Turner, Quintin Pope

4mo

28

43Human values & biases are inaccessible to the genome

5mo

25

24General alignment properties

4mo

1

23Evolution is a bad analogy for AGI: inner alignment

4mo

0

76Reward is not the optimization target

4mo

63

62The shard theory of human values

Quintin Pope, Alex Turner

2mo

20

20Understanding and avoiding value drift

3mo

5

32A shot at the diamond-alignment problem

2mo

33

29Don't design agents which exploit adversarial inputs

Alex Turner, Garrett Baker

15d

26

22Don't align agents to evaluations of plans

6d

25

24Alignment allows "nonrobust" decision-influences and doesn't require robust grading

4d

16

35Inner and outer alignment decompose one hard problem into two extremely hard problems

1d

1