Epistemic Status: Exploratory. My current but-changing outlook with limited exploration & understanding for ~60-80hrs.

Acknowledgements: This post was written under Evan Hubinger’s direct guidance and mentorship as a part of the Stanford Existential Risks Institute ML Alignment Theory Scholars (MATS) program. Thanks to particlemania, Shashwat Goel and shawnghu for exciting discussions. They might not agree with some of the claims made here; all mistakes are mine.

Summary (TL;DR)

Goal: Understanding the inductive biases of Prosaic AI systems could be very informative towards creating a frame of safety problems and solutions. The proposal here is to generate an Evidence Set from current ML literature to model the potential inductive bias of Prosaic AGI.

Procedure: In this work, I collect evidence of inductive biases of deep networks by studying ML literature. Moreover, I estimate from current evidence whether these inductive biases vary with scaling to large models. If a phenomenon seems robust to or amplified by scaling, I discuss it here and add it to the Evidence Set.

Structure: I provide interpretations of some interesting papers to AI safety in three maximally-relevant subareas of ML literature (pretraining-> finetuning, generalization and adversarial robustness), and demonstrate use cases to AI safety (in ‘Application’ subsections). I then summarize evidence from each area to form the Evidence Set.

Current Evidence Set: Given in Section “Evidence Set” (last section of this post)

Inspiration: I think developing good intuitions about inductive biases and past evidence essentially constitutes ‘experience’ used by ML researchers. Evidence sets and broadly analyzing inductive biases are the first steps towards mechanizing this intuition. Inductive bias analysis might be the right level of abstraction. A large, interconnected Evidence Set might give a degree of gears-level understanding of the black-box that is deep networks.

Applications: Any theory of inductive biases, if fully developed, can sufficiently explain the observations collected in the evidence set. Alternatively, we can use the evidence set to falsify mechanistic statements about training rationales in Training stories. However, I expect evidence sets to be helpful beyond these two applications.

Limitations: Evidence...

Posts

Wikitag Contributions

Comments