AI ALIGNMENT FORUM
AF

Gradient Hacking

Edited by Multicore last updated 27th Aug 2022

Gradient Hacking describes a scenario where a mesa-optimizer in an AI system acts in a way that intentionally manipulates the way that gradient descent updates it, likely to preserve its own mesa-objective in future iterations of the AI.

See also: Inner Alignment

1

1

Posts tagged Gradient Hacking

53Gradient hacking

6y

33

71Gradient hacking is extremely difficult

3y

0

17Gradient Filtering

3y

0

15Challenge: construct a Gradient Hacker

Thomas Larsen, Thomas Kwa

2y

2

9Some real examples of gradient hacking

4y

0

24How does Gradient Descent Interact with Goodhart?

Scott Garrabrant, evhub

7y

4

33Gradient Hacker Design Principles From Biology

3y

11

26Understanding Gradient Hacking

4y

2

25Towards Deconfusing Gradient Hacking

4y

2

19Gradient hacking: definitions and examples

3y

1

22Thoughts on gradient hacking

4y

8

10Approaches to gradient hacking

4y

7

52Programmatic backdoors: DNNs can use SGD to run arbitrary stateful computation

Fabien Roger, Buck

2y

0

30Meta learning to gradient hack

4y

7

8Goodbye, Shoggoth: The Stage, its Animatronics, & the Puppeteer – a New Metaphor

2y

0

Load More (15/19)

Add Posts