AI ALIGNMENT FORUMAlignment Newsletter
AF

Alignment Newsletter

I publish the Alignment Newsletter, a weekly publication with recent content relevant to AI alignment. See here for more details. Quick links: email signup form, RSS feed, spreadsheet of all summaries.

3The Alignment Newsletter #1: 04/09/18

5y

0

1The Alignment Newsletter #2: 04/16/18

5y

0

2The Alignment Newsletter #3: 04/23/18

5y

0

1The Alignment Newsletter #4: 04/30/18

5y

0

1The Alignment Newsletter #5: 05/07/18

5y

0

1The Alignment Newsletter #6: 05/14/18

5y

0

1The Alignment Newsletter #7: 05/21/18

5y

0

1The Alignment Newsletter #8: 05/28/18

5y

0

1The Alignment Newsletter #9: 06/04/18

5y

0

5The Alignment Newsletter #10: 06/11/18

5y

0

1The Alignment Newsletter #11: 06/18/18

5y

0

3The Alignment Newsletter #12: 06/25/18

5y

0

17Alignment Newsletter #13: 07/02/18

5y

0

4Alignment Newsletter #14

5y

0

8Alignment Newsletter #15: 07/16/18

5y

0

12Alignment Newsletter #16: 07/23/18

5y

0

6Alignment Newsletter #17

5y

0

4Alignment Newsletter #18

5y

0

4Alignment Newsletter #19

5y

0

5Alignment Newsletter #20

5y

0

7Alignment Newsletter #21

5y

0

5Alignment Newsletter #22

5y

0

3Alignment Newsletter #23

5y

0

5Alignment Newsletter #24

5y

2

9Alignment Newsletter #25

5y

3

4Alignment Newsletter #26

5y

0

6Alignment Newsletter #27

5y

0

6Alignment Newsletter #28

5y

0

5Alignment Newsletter #29

5y

0

12Alignment Newsletter #30

5y

0

7Alignment Newsletter #31

5y

0

6Alignment Newsletter #32

5y

0

7Alignment Newsletter #33

5y

0

8Alignment Newsletter #34

5y

0

5Alignment Newsletter #35

5y

0

10Alignment Newsletter #36

4y

0

9Alignment Newsletter #37

4y

2

6Alignment Newsletter #38

4y

0

12Alignment Newsletter #39

4y

0

8Alignment Newsletter #40

4y

0

7Alignment Newsletter #41

4y

6

11Alignment Newsletter #42

4y

0

7Alignment Newsletter #43

4y

2

9Alignment Newsletter #44

4y

0

12Alignment Newsletter #45

4y

0

11Alignment Newsletter #46

4y

0

10Alignment Newsletter #47

4y

0

16Alignment Newsletter #48

4y

14

11Alignment Newsletter #49

4y

0

7Alignment Newsletter #50

4y

0

11Alignment Newsletter #51

4y

0

8Alignment Newsletter #52

4y

1

29Alignment Newsletter One Year Retrospective

4y

10

10Alignment Newsletter #53

4y

0

10[AN #54] Boxing a finite-horizon AI system to keep it unambitious

4y

0

11[AN #55] Regulatory markets and international standards as a means of ensuring beneficial AI

4y

1

9[AN #56] Should ML researchers stop running experiments before making hypotheses?

4y

6

15[AN #57] Why we should focus on robustness in AI safety, and the analogous problems in programming

4y

14

22[AN #58] Mesa optimization: what it is, and why we should care

4y

1

17[AN #59] How arguments for AI risk have changed over time

4y

0

10[AN #60] A new AI challenge: Minecraft agents that assist human players in creative mode

4y

1

0[AN #61] AI policy and governance, from two people in the field

4y

1

14[AN #62] Are adversarial examples caused by real but imperceptible features?

4y

8

15[AN #63] How architecture search, meta learning, and environment design could lead to general intelligence

4y

3

8[AN #64]: Using Deep RL and Reward Uncertainty to Incentivize Preference Learning

4y

6

9[AN #65]: Learning useful skills by watching humans “play”

4y

0

9[AN #66]: Decomposing robustness into capability robustness and alignment robustness

4y

1

12[AN #67]: Creating environments in which to study inner alignment failures

4y

0

12[AN #68]: The attainable utility theory of impact

4y

0

25[AN #69] Stuart Russell's new book on why we need to replace the standard model of AI

4y

10

14[AN #70]: Agents that help humans who are still learning about their own preferences

4y

0

10[AN #71]: Avoiding reward tampering through current-RF optimization

4y

0

16[AN #72]: Alignment, robustness, methodology, and system building as research priorities for AI safety

4y

0

8[AN #73]: Detecting catastrophic failures by learning how agents tend to break

4y

0

13[AN #74]: Separating beneficial AI into competence, alignment, and coping with impacts

4y

0

18[AN #75]: Solving Atari and Go with learned game models, and thoughts from a MIRI employee

4y

0

10[AN #76]: How dataset size affects robustness, and benchmarking safe exploration by measuring constraint violations

4y

2

10[AN #77]: Double descent: a unification of statistical theory and modern ML practice

3y

1

15[AN #78] Formalizing power and instrumental convergence, and the end-of-year AI safety charity comparison

3y

3

9[AN #79]: Recursive reward modeling as an alignment technique integrated with deep RL

3y

0

18[AN #80]: Why AI risk might be solved without additional intervention from longtermists

3y

65

23[AN #81]: Universality as a potential solution to conceptual difficulties in intent alignment

3y

2

12[AN #82]: How OpenAI Five distributed their training computation

3y

0

10[AN #83]: Sample-efficient deep learning with ReMixMatch

3y

1

13[AN #84] Reviewing AI alignment work in 2018-19

3y

0

10[AN #85]: The normative questions we should be asking for AI alignment, and a surprisingly good chatbot

3y

0

10[AN #86]: Improving debate and factored cognition through human experiments

3y

0

14[AN #87]: What might happen as deep learning scales even further?

3y

0

12[AN #88]: How the principal-agent literature relates to AI risk

3y

0

10[AN #89]: A unifying formalism for preference learning algorithms

3y

0

9[AN #90]: How search landscapes can contain self-reinforcing feedback loops

3y

1

11[AN #91]: Concepts, implementations, problems, and a benchmark for impact measurement

3y

5

11[AN #92]: Learning good representations with contrastive predictive coding

3y

0

11[AN #93]: The Precipice we’re standing at, and how we can back away from it

3y

0

8[AN #94]: AI alignment as translation between humans and machines

3y

0

11[AN #95]: A framework for thinking about how to make AI go well

3y

0

11[AN #96]: Buck and I discuss/argue about AI Alignment

3y

4

10[AN #97]: Are there historical examples of large, robust discontinuities?

3y

0

11[AN #98]: Understanding neural net training by seeing which gradients were helpful

3y

3

14[AN #99]: Doubling times for the efficiency of AI algorithms

3y

0

21[AN #100]: What might go wrong if you learn a reward function while acting

3y

2

9[AN #101]: Why we should rigorously measure and forecast AI progress

3y

0

19[AN #102]: Meta learning by GPT-3, and a list of full proposals for AI alignment

3y

0

18[AN #103]: ARCHES: an agenda for existential safety, and combining natural language with deep RL

3y

0

12[AN #104]: The perils of inaccessible information, and what we can learn about AI alignment from COVID

3y

2

16[AN #105]: The economic trajectory of humanity, and what we might mean by optimization

3y

1

10[AN #106]: Evaluating generalization ability of learned reward models

3y

1

9[AN #107]: The convergent instrumental subgoals of goal-directed agents

3y

1

14[AN #108]: Why we should scrutinize arguments for AI risk

3y

5

12[AN #109]: Teaching neural nets to generalize the way humans would

3y

0

9[AN #110]: Learning features from human feedback to enable reward learning

3y

0

16[AN #111]: The Circuits hypotheses for deep learning

3y

0

12[AN #112]: Engineering a Safer World

3y

2

13[AN #113]: Checking the ethical intuitions of large language models

3y

0

14[AN #114]: Theory-inspired safety solutions for powerful Bayesian RL agents

3y

1

13[AN #115]: AI safety research problems in the AI-GA framework

3y

2

13[AN #116]: How to make explanations of neurons compositional

3y

2

15[AN #117]: How neural nets would fare under the TEVV framework

3y

0

10[AN #118]: Risks, solutions, and prioritization in a world with many AI systems

3y

6

9[AN #119]: AI safety when agents are shaped by environments, not rewards

3y

0

9[AN #120]: Tracing the intellectual roots of AI and AI alignment

3y

4

14[AN #121]: Forecasting transformative AI timelines using biological anchors

3y

5

15[AN #122]: Arguing for AGI-driven existential risk from first principles

3y

0

14[AN #123]: Inferring what is valuable in order to align recommender systems

3y

1

8[AN #124]: Provably safe exploration through shielding

3y

0

16[AN #125]: Neural network scaling laws across multiple modalities

3y

1

15[AN #126]: Avoiding wireheading by decoupling action feedback from action effects

3y

1

24[AN #127]: Rethinking agency: Cartesian frames as a formalization of ways to carve up the world into an agent and its environment

3y

0

11[AN #128]: Prioritizing research on AI existential safety based on its application to governance demands

2y

0

11[AN #129]: Explaining double descent by measuring bias and variance

2y

1

8[AN #130]: A new AI x-risk podcast, and reviews of the field

2y

0

10[AN #131]: Formalizing the argument of ignored attributes in a utility function

2y

2

15[AN #132]: Complex and subtly incorrect arguments as an obstacle to debate

2y

1

11[AN #133]: Building machines that can cooperate (with humans, institutions, or other machines)

2y

0

10[AN #134]: Underspecification as a cause of fragility to distribution shift

2y

0

22[AN #135]: Five properties of goal-directed systems

2y

0

13[AN #136]: How well will GPT-N perform on downstream tasks?

2y

0

13[AN #137]: Quantifying the benefits of pretraining on downstream task performance

2y

0

9[AN #138]: Why AI governance should find problems rather than just solving them

2y

0

17[AN #139]: How the simplicity of reality explains the success of neural nets

2y

6

22[AN #140]: Theoretical models that predict scaling laws

2y

0

18[AN #141]: The case for practicing alignment work on GPT-3 and other large models

2y

4

21[AN #142]: The quest to understand a network well enough to reimplement it by hand

2y

3

10[AN #143]: How to make embedded agents that reason probabilistically about their environments

2y

0

15[AN #144]: How language models can also be finetuned for non-language tasks

2y

0

27Alignment Newsletter Three Year Retrospective

2y

0

12[AN #145]: Our three year anniversary!

2y

0

14[AN #146]: Plausible stories of how we might fail to avert an existential catastrophe

2y

0

12[AN #147]: An overview of the interpretability landscape

2y

2

16[AN #148]: Analyzing generalization across more axes than just accuracy or loss

2y

3

15[AN #149]: The newsletter's editorial policy

2y

3

12[AN #150]: The subtypes of Cooperative AI research

2y

0

16[AN #151]: How sparsity in the final layer makes a neural net debuggable

2y

1

15[AN #152]: How we’ve overestimated few-shot learning capabilities

2y

2

19[AN #153]: Experiments that demonstrate failures of objective robustness

2y

0

11[AN #154]: What economic growth theory has to say about transformative AI

2y

0

14[AN #155]: A Minecraft benchmark for algorithms that learn without reward functions

2y

1

21[AN #156]: The scaling hypothesis: a plan for building AGI

2y

16

18[AN #157]: Measuring misalignment in the technology underlying Copilot

2y

9

15[AN #158]: Should we be optimistic about generalization?

2y

0

11[AN #159]: Building agents that know how to experiment, by training on procedurally generated games

2y

4

14[AN #160]: Building AIs that learn and think like people

2y

1

11[AN #161]: Creating generalizable reward functions for multiple tasks by learning a model of functional similarity

2y

0

18[AN #162]: Foundation models: a paradigm shift within AI

2y

0

26[AN #163]: Using finite factored sets for causal and temporal inference

2y

0

9[AN #164]: How well can language models write code?

2y

2

12[AN #165]: When large models are more likely to lie

2y

0

26[AN #166]: Is it crazy to claim we're in the most important century?

2y

2

16[AN #167]: Concrete ML safety problems and their relevance to x-risk

2y

4

10[AN #168]: Four technical topics for which Open Phil is soliciting grant proposals

2y

0

18[AN #169]: Collaborating with humans without human data

2y

0

15[AN #170]: Analyzing the argument for risk from power-seeking AI

1y

0

20[AN #171]: Disagreements between alignment "optimists" and "pessimists"

1y

0

25[AN #172] Sorry for the long hiatus!

1y

0

21[AN #173] Recent language model results from DeepMind

1y

0