AI ALIGNMENT FORUM
AF

Finite Factored SetsLogic & Mathematics World Modeling
Frontpage

13

A well-defined history in measurable factor spaces

by Matthias G. Mayer
5th Oct 2023
3 min read
0

13

Finite Factored SetsLogic & Mathematics World Modeling
Frontpage
New Comment
Moderation Log
Curated and popular this week
0Comments

PDF

This is a technical post researching infinite factor spaces

In a factor space F=⨉i∈IFi, for a measurable Z:F→Z(F), a Z-measurable index function J:F→{0,1}I is generating X:F→X(F), if X depends only on those arguments (xi), where J(x)i is 1, and Z. In this post, we show that there is an almost surely minimal such index function that we call the history of X given Z.

We will reintroduce all the definitions from the finite case:

Let I be finite. Let (Fi,Ai,Ni) be a measure space with nullsets, i.e. Fi is a set, Ai is a σ-algebra and Ni is a σ-ideal that admits a probability, i.e. there is Pi with Ni={A∈A:Pi(A)=0}.

We define a product σ-ideal: N1⊗N2={A∈A:{x:A(x)∉N2}∈N1}. Clearly, N1⊗N2={A∈A:P1×P2(A)=0}. We can extend this by induction to ⨂i∈INi.

We construct (F,A,N)=(⨉i∈IF,⨂i∈IAi,⨂i∈INi). Furthermore, let P=⨉i∈IPi.

Definition 1 (almost sure subset)

We will write A∗⊆B:⇔A∖B∈N.

Definition 2 (almost sure union with arbitrary index set)

If N is a σ-ideal that admits a probability P and (Aj)j∈J∈AJ is a family of measurable sets, then there exists an almost surely unique and minimal A∈A with ∀j∈J:Aj∗⊆A. i.e. whenever ∀j∈J:Aj∗⊆B, we have A∗⊆B. We set ∗⋃j∈JAj≔A. Furthermore, there is J∗⊆J countable, s.t. ⋃∗j∈JAj=⋃j∈J∗Aj.
Proof. We will construct J∗.

To start, choose any j∈J, J0={j} and let B0=Aj.

Let Bn be defined and let sn=supj∈JP(Aj∖Bn). Choose jn∈J with P(Ajn∖Bn)≥sn−1n and set Jn+1=Jn∪{jn} and Bn+1=Bn∪Ajn.

We set J∗={jn:n∈N} and A=⋃j∈J∗Aj.

  • A contains Aj: Let j∈J, we have to show P(Aj∖A)=0. Assume not, then P(Aj∖Bn)↓P(Aj∖A)=p>0, so sn≥p for all n. Therefore, we have P(Ajn∖Bn)≥p−1n. Now 1≥P(A)=P(⨃n∈NAjn∖Bn)=∑n∈NP(Ajn∖Bn)=∞, which is a contradiction.

  • Minimality and uniqueness: Now let B∈A:∀j∈J:Aj∗⊆B. Then clearly, A=⋃n∈NAjn∗⊆B. If B is also minimal, then clearly A∗⊆B∗⊆A⇔P(A▲B)=0⇔Aa.s. =B.

Definition 3 (almost sure intersection)

We set ⋂∗j∈JAj=(⋃∗j∈JAcj)c.

Definition 4 (feature)

We call a measurable function Z:F→Z(F) a feature. In the following let X,Y,Z be features.

Definition 5 (index function)

We call a measurable J:F→{0,1}I an index function. We write Ji=πi∘J. We identify J with J′:F→P(I), where J′(x)={i∈I:Ji(x)=1} to allow for set operations such as J∩K=min(J,K).

Definition 6

For an index function J, we set πJ=(1{Ji=1}πi)i∈I.

Definition 7

We define that X is almost surely a function of Y by X≲∗Y:⇔σ(X)⊆σ(Y,N).

Let ▲∗(F,N) be all product distributions whose nullsets are N.

Definition 8 (conditional generation)

Let J:F→{0,1}I be Z-measurable. We write J⊢X | Z, if X≲∗(πJ,Z)∧∀P∈▲∗(F,N):πJ⫫Pπ¯J | Z. Note that πJ⫫Pπ¯J∣∣Z⇔(πJ,Z)⫫P(π¯J,Z)∣∣Z

Lemma 9

For n∈N let Jn:F→P(I) be Z-measurable and J=⋃n∈NJn.

Then σ(πJ,Z)=σ((πJn,Z)n∈N). Proof. Let A≔σ(πJ,Z), B≔σ((πJn,Z)n∈N).

'A⊆B': Obviously, σ(Z)⊆B. Let A∈σ(1{Ji=1}πi), then there is a B∈σ(πi) with A=B∩{Ji=1}=B∩⋃n∈N{Jni=1}=⋃n∈N(B∩{Jni=1})∈B.

'A⊇B': Obviously, σ(Z)⊆A. Let A∈σ(1{Jni=1}πi), then there is B∈σ(πi) with A=B∩{Jni=1}=B∩{Ji=1}∈A∩{Jni=1}∈σ(Z)⊆A∈A.

Lemma 10

Let J,K:F→P(I) and J,K⊢X | Z. Then J∩K⊢X | Z.

Proof. Let P∈▲∗(F,N). We first show πJ∩K⫫Pπ¯J∩K | Z. For i,j∈{0,1}, let Mij={J′i∩K′j} where J′0=J,J′1=¯J, etc. Let Aij∈Mij. We have (πM00,πM01)⫫P(πM10,πM11)|Z, (1) (πM00,πM10)⫫P(πM01,πM11)|Z, (2)  Now P(⋂ijAij | Z)(1) =P(A00∩A01 | Z)⋅P(A10∩A11 | Z)(2) =∏ijP(Aij | Z).

Now since sets of the form A01∩A10∩A11 generate π¯J∩K and are ∩-stable, we have that πJ∩K⫫Pπ¯J∩K | Z.

It remains to show X≲∗(πJ∩K,Z). Since πJ∩K⫫PπJ∖K | Z, we have P(πJ∖K | Z)=P(πJ∖K | πJ∩K,Z). Since the same holds true for K∖J, we get πJ∖K⫫PπK∖J | πJ∩K,Z and therefore πJ⫫PπK | πJ∩K,Z.

We claim σ(πJ,Z)∩σ(πK,Z)a.s. =σ(πJ∩K,Z).

'⊇': Trivial.
'⊆∗': Let A∈σ(πJ,Z)∩σ(πK,Z) and B=σ(πJ∩K,Z). Then A⫫PA | B. Therefore, P(A|B)=1B for a B∈B. We claim P(A▲C)=0. We have P(A∖C)=E(P(A∖C | B))=E(1CcP(A∖C | B))=E(0)=0, we have P(A∖C)=0. Similarly, P(C∖A | B)=0 and therefore P(C∖A)=0.

Lemma 11

For n∈N, let Jn⊢X | Z. Let J=⋂n∈NJn. Then J⊢X | Z.
Proof. Let Kn=⋂k≤nJk. Clearly, Kn⊢X | Z and Kn↓J.

We first show πJ⫫Pπ¯J | Z. Since σ(πJ)⊆σ(πKn), we have πJ⫫Pπ¯Kn | Z. Now ¯Kn↑¯J. Clearly, ⋃n∈Nσ(π¯Kn) is ∩-stable and by Lemma 9 a generator of ¯J. Therefore, πJ⫫Pπ¯J | Z.

We now show X≲∗(πJ,Z). Let B=⋂n∈Nσ(πKn,Z) and C=σ(πJ,Z)

'B⊇C" : Trivial.
'B⊆∗C': From πJ⫫Pπ¯J | Z, we have πJ⫫PπKn∖J|Z,πJ⫫Pπ¯Kn|Z Therefore, πKn⫫π¯Kn∪J | C.

Clearly, ⋃n∈Nσ(π¯Kn∪J) is a ∩-stable generator of A and therefore B⫫PA | C. Now, since B⊆A, we have B⫫PB | C and therefore B⊆σ(C,N).

Definition 12 (history)

H(X|Y) is the a.s. smallest generating index function.

Theorem 13 A history exists and is a.s. unique.

Proof. Existence: Let Mi={{Ji=1}:J⊢X | Z}. Define (H(X|Y))i≔1⋂∗Mi. Now clearly, for each i, there are countable many Jn,i⊢X | Z, such that (⋂n∈NJn,i)i=1⋂∗Mi. Now ⋂n∈N,i∈IJni=H(X|Y), and therefore H(X|Y)⊢X | Z. Furthermore, by construction we have for any J⊢X | Z, that {H(X|Y)i=1}∗⊆{Ji=1} which implies H(X|Y)∗⊆J.

Uniqueness: Let J be minimal, then clearly, H(X|Y)∗⊆J∗⊆H(X|Y).