AI ALIGNMENT FORUM
AF

1Vanessa Kosoy3d It seems useful to consider agents that reason in terms of an unobservable ontology, and may have uncertainty over what this ontology is. In particular, in Dialogic RL [https://www.alignmentforum.org/posts/dPmmuaz9szk26BkmD/vanessa-kosoy-s-shortform#Wi65Ahs9abL63gPSe] , the user's preferences are probably defined w.r.t. an ontology that is unobservable by the AI (and probably unobservable by the user too) which the AI has to learn (and the user is probably uncertain about emself). However, onotlogies are more naturally thought of as objects in a category than as elements in a set. The formalization of an "ontology" should probably be a POMDP or a suitable Bayesian network. A POMDP involves an arbitrary set of states, so it's not an element in a set, and the class of POMDPs can be naturally made into a category. Therefore, there is need for defining the notion of a probability measure over a category. Of course we can avoid this by enumerating the states, considering the set of all possible POMDPs w.r.t. this enumeration and then requiring the probability measure to be invariant w.r.t. state relabeling. However, the category theoretic point of view seems more natural, so it might be worth fleshing out. Ordinary probably measures are defined on measurable spaces. So, first we need to define the analogue of "measurable structure" (σ-algebra) for categories. Fix a category C. Denote Meas the category of measurable spaces. A measurable structure on C is then specified by providing a Grothendick fibration [https://ncatlab.org/nlab/show/Grothendieck+fibration] B:MFC→Meas and an equivalence E:B−1(pt)→C. Here, B−1(pt) stands for the essential fiber [https://ncatlab.org/nlab/show/essential+fiber] of B over the one point space pt∈Meas. The intended interpretation of MFC is, the category of families of objects in C indexed by measurable spaces. The functor B is supposed to extract the base (index space) of the family. We impose the following conditions on MFC and B: Given A∈

Thursday, November 7th 2019
Thu, Nov 7th 2019

No posts for November 7th 2019

Wednesday, November 6th 2019
Wed, Nov 6th 2019

Frontpage Posts

12[Link]AI Alignment Research Overview (by Jacob Steinhardt)

Ben Pace

14[AN #72]: Alignment, robustness, methodology, and system building as research priorities for AI safety

Rohin Shah

Tuesday, November 5th 2019
Tue, Nov 5th 2019

No posts for November 5th 2019

Monday, November 4th 2019
Mon, Nov 4th 2019

Frontpage Posts

17Will transparency help catch deception? Perhaps not

Matthew Barnett

11More variations on pseudo-alignment

Evan Hubinger

Shortform [Beta]

1[comment deleted]7d (13/?) Global phase. Consider a quantum system that's even smaller than a qubit. Something with only one possible measurement outcome, instead of two. Let's call it "quantum nit", or "qunit", by analogy with "unit" in typed programming, which means a type with one element. A qunit has only one possible mixed state, with the density matrix ((1)). But its pure states are more complicated: a one-qunit pure state is a number whose norm is 1, so either 1 or -1 (or more if we allow complex numbers). The same is true for an N-qunit pure state - it's also a single number whose norm is 1! The reason is that the tensor product of N 1-dimensional spaces is 1-dimensional, not N-dimensional. So the joint state of all qunits in the world is a single number, which is called "global phase". Now imagine we have a system with a bunch of qubits, and take its tensor product with a qunit. We can move a scalar factor from one side of a tensor product to the other, and that shouldn't change anything. So global phase can't affect any measurement, it's just an extra degree of freedom in the theory.

1Vladimir Slepnev8d (12/?) The uncertainty principle, or something like it. When you measure a qubit cos(φ)|0> + sin(φ)|1>, the result has variance (1-cos(4φ))/4. (I'm skipping over the trig calculations here and below.) If you have a choice between either measuring the qubit, or rotating it by an angle ψ and then measuring it, then the sum of variances of the two operations is at least (1-|cos(2ψ)|)/2. In particular, if ψ=π/4, the sum of variances is at least 1/2. So no matter what state a qubit is in, and how precisely you know its state, there are two possible measurements such that the sum of variances of their results is at least 1/2. The reason is that the bases corresponding to these measurements are at an angle to each other, not aligned. Apparently in the real world, an object's "position" and "momentum" are two such measurements, so there's a limit on their joint precision. You can also carry out one measurement and then the other, but that doesn't help - after the first measurement you have variance in the second. Moreover, after the second you have fresh variance in the first. This lets you get an infinite stream of fair coinflips from a single qubit: start with |0>, rotate by π/4, measure, rotate by π/4, measure...