This post was written under Evan Hubinger’s direct guidance and mentorship, as a part of the Stanford Existential Risks Institute ML Alignment Theory Scholars (MATS) program.

TL;DR: In this post, I refine universality, in order to explain the "Filter". The Filter is a method of risk analysis for the outputs of powerful AI systems. I refine universality by replacing the language of “beliefs” and proposing a handful of general definitions in part I. Then, I explain the key application of universality of building the Filter. I explain the Filter non-technically through a parable in part II, then technically in part III. To understand the Filter as presented in part II, it is not necessary to comprehend universality to the rigor that I present in part I; it is only necessary to see that all models are wrong¹. While this post builds technically off a previous post², part II is meant to be understandable to all audiences.

I. Refining Universality

Universality is discussed here, here, and here. Here are some features of universality that have been missing from its presentation.

1. The need to replace “beliefs”

A “belief” of a system is used 60 times in Paul Christiano’s attempt to formalize universality. I think a “belief” is not the right language for an object of a system for two reasons. First, it conflates terms worth making distinct. When “beliefs” are used in Paul Christiano’s attempt to formalize universality, they don’t always refer to the same thing. Second, “belief” should be reserved for a different concept. The way humans think of “belief” is not how it’s used there.

So, I rewrote the formalization of universality to not include the word “belief”. The handful of terms presented below combine to replace the previous notion of “belief”. Presenting universality without the language of “beliefs”, all said, is certainly more cumbersome.

I hope this reframing will help improve the methodology for applying universality to HCH. I apply the general definitions below to universality and the Filter in section III.2.

2. General definitions for universality

Consider $κ$ , a list of action-shaping intelligible information. $κ = {κ_{j}}$ , where $κ_{j}$ are pieces of intelligible information. Pieces of intelligible information are facts, which are sentences (i.e. “The sky is blue”, “there are infinitely many twin primes”). Another term, $X$ , is t...

Posts

Wikitag Contributions

Comments