Theorem 2: Isomorphism Theorem:For (causal, pseudocausal, acausal, surcausal) Θst or Θω which fulfill finitary or infinitary analogues of all the defining conditions, ↑(Θst) and ↓(Θω) are (causal, pseudocausal, acausal, surcausal) hypotheses. Also, ↑ and →st define an isomorphism between Θ and Θst, and ↓ and →ω define an isomorphism between Θ and Θω.

Proof sketch: The reason this proof is so horrendously long is that we've got almost a dozen conditions to verify, and some of them are quite nontrivial to show and will require sub-proof-sketches of their own! Our first order of business is verifying all the conditions for a full belief function for ↑(Θst). Then, we have to do it all over again for ↓(Θω). That comprises the bulk of the proof. Then, we have to show that taking a full belief function Θ and restricting it to the infinite/finite levels fulfills the infinite/finite analogues of all the defining conditions for a belief function on policies or policy-stubs, which isn't quite as bad. Once we're done with all the legwork showing we can derive all the conditions from each other, showing the actual isomorphism is pretty immediate from the Consistency condition of a belief function.

Part 1:Let's consider ↑(Θπst). This is defined as: ↑(Θst)(πpa):=⋂πst≤πpa(prπpa,πst∗)−1(Θst(πst))

We'll show that all 9+2 defining conditions for a belief function are fulfilled for ↑(Θst). The analogue of the 9+2 conditions for a Θst is:

Let's begin showing the conditions. But first, note that since we have weak consistency, we can invoke Lemma 6 to reexpress ↑(Θst)(πpa) as ⋂n(prπpa,πnpa∗)−1(Θst(πnpa)) Where πnpa is the n'th member of the fundamental sequence of πpa.

Also note that, for all stubs, ↑(Θst(πst))=Θst(πst). We'll be casually invoking this all over the place and won't mention it further.

Proof: By Lemma 6 with weak consistency, ↑(Θst(πst))=⋂n≥m(prπst,πnst∗)−1(Θst(πnst)) Now, m can be anything we like, as long as it's finite. Set m to be larger than the maximum timestep that the stub is defined for. Then πnst=πst no matter what n is (since it's above m) and projection from a stub to itself is identity, so the preimage is exactly our original set Θst(πst).

We'll also be using another quick result. For all stubs πhist≥πlost, given stub causality,

prπhist,πlost∗(Θst(πhist))=Θst(πlost)

Proof: Fix an arbitrary point M∈Θst(πlost). By causality, we get an outcome function which includes M, giving us that there's something in Θst(πhist) that projects down onto M. Use weak consistency to get the other subset direction.

Condition 1: Nirvana-free Nonemptiness.

Invoking Stub Nirvana-free Nonemptiness, ∀πst:Θst(πst)∩NF≠∅ so we get nirvana-free nonemptiness for ↑(Θst)(πst).

Now, assume πpa is not a stub. By stub-bounded-minimals, there is some λ⊙+b⊙ bound on the set of minimal points, regardless of stub. Let (Θst(πnpa))clip be Θst(πnpa)∩{≤⊙}∩NF

This contains all the minimal nirvana-free points for πnpa. This set is nonempty because we have stub nirvana-free nonemptiness, so a nirvana-free point M exists. We have stub-closure and stub minimal-boundedness, so we can step down to a minimal nirvana-free point below M, and it obeys the λ⊙+b⊙ bound.

Further, by weak consistency and projection preserving λ and b and nirvana-freeness,

prπn+1pa,πnpa∗((Θst(πn+1pa))clip)⊆(Θst(πnpa))clip

Invoking Lemma 9, the intersection of preimages of these is nonempty. It's also nirvana-free, because if there's nirvana somewhere, it occurs after finite time, so projecting down to some sufficiently large finite stage preserves the presence of Nirvana, but then we'd have a nirvana-containing point in a nirvana-free set (Θst(πnpa))clip, which is impossible. This is also a subset of the typical intersection of preimages used to define ↑(Θst)(πpa). Pick an arbitrary point in said intersection of preimages of clipped subsets.

Bam, we found a nirvana-free point in ↑(Θst)(πpa) and we're done.

Time for conditions 2 and 3, Closure and Convexity. These are easy.

↑(Θst)(πpa)=⋂n(prπpa,πnpa∗)−1(Θst(πnpa))

The preimage of a closed set (stub-closure) is a closed set, and the intersection of closed sets is closed, so we have closure.

Also, prπpa,πnst∗ is linear, so the preimage of a convex set (stub-convexity) is convex, and we intersect a bunch of convex sets so it's convex as well.

Condition 4: Nirvana-free upper completion.

Let M∈↑(Θst)(πpa)∩NF. Let's check whether M+M∗ (assuming that's an a-measure and M∗ is nirvana-free) also lies in the set. A sufficient condition on this given how we defined things is that for all πnpa, prπpa,πnpa∗(M+M∗)∈Θst(πnpa), as that would certify that M+M∗ is in all the preimages.

prπpa,πnpa∗ is linear, so prπpa,πnpa∗(M+M∗)=prπpa,πnpa∗(M)+prπpa,πnpa∗(M∗)

The first component is in Θst(πst), obviously. And then, by stub nirvana-free-upper-completion, we have a nirvana-free a-measure plus a nirvana-free sa-measure (projection preserves nirvana-freeness), making a nirvana-free a-measure (projection preserves a-measures), so prπpa,πnpa∗(M)+prπpa,πnpa∗(M∗) is in Θst(πnpa)∩NF, and we're done.

Condition 5: Bounded-Minimals

So, there is a critical λ⊙+b⊙ value by restricted-minimals for Θst

Fix a πpa, and assume that there is a minimal point M in ↑(Θst)(πpa) with a λ+b value that exceeds the bound. Project M down into each Θst(πnpa). Projection preserves λ and b so each of these projected points Mn lie above some Mminn.

Now, invoke Lemma 7 to construct a M′n∈Ma(F(πpa)) (or the nirvana-free variant) that lies below M, and projects down to Mminn. Repeat this for all n. All these M′n points are a-measures and have the standard λ⊙+b⊙ bound so they all lie in a compact set and we can extract a convergent subsequence, that converges to M′, which still obeys the λ⊙+b⊙ bound.

M′ is below M because ({M}−Msa(F(πpa)))∩Ma(F(πpa)) (or the nirvana-free variant) is a closed set. Further, by Lemma 10, M′ is in the defining sequence of intersections for ↑(Θst)(πpa). This witnesses that M isn't minimal, because we found a point below it that actually obeys the bounds. Thus, we can conclude minimal-point-boundedness for ↑(Θst).

Condition 6: Normalization. We'll have to go out of order here, this can't be shown at our current stage. We're going to have to address Hausdorff continuity first, then consistency, and solve normalization at the very end. Let's put that off until later, and just get extreme points.

Condition 8: Extreme point condition:

The argument for this one isn't super-complicated, but the definitions are, so let's recap what condition we have and what condition we're trying to get.

Ok, so M∈(Θst(πst))xmin∩NF By the stub extreme point condition, there's a π≥πst, where, for all πhist that fulfill π>πhist≥πst, there's a M′∈Θst(πhist)∩NF, where prπhist,πst∗(M′)=M.

Lock in the π we have. We must somehow go from this to a M′∈Θ(π)∩NF that projects down to our point of interest. To begin with, let πn be the n'th member of the fundamental sequence for π. Past a certain point m, these start being greater than πst. The M′∈Θst(πn)∩NF which projects down to M that we get by the stub-extreme-point condition will be called M′lon. Pick some random-ass point in (prπ,πnst∗)−1(M′lon) and call it M′hin.

M′hin all obey the λ and b values of M, because it projects down to M. We get a limit point of them, M′, and invoking Lemma 10, it's also in ↑(Θst)(π). It also must be nirvana-free, because it's a limit of points that are nirvana-free for increasingly late times. It also projects down to M because the sequence M′hin was wandering around in the preimage of M, which is closed.

Condition 9: Hausdorff Continuity:

Ok, this one is going to be fairly complicated. Remember, our original form is:

"The function πst↦(pr∞,πst∗)−1(Θst(πst)∩NF∩{≤⊙}) is uniformly continuous"

And the form we want is:

"The function πpa↦(pr∞,πpa∗)−1(↑(Θst)(πpa)∩NF∩{≤⊙}) is uniformly continuous"

Uniform continuity means that if we want an ϵ Hausdorff-distance between two preimages, there's a δ distance between partial policies that suffices to produce that. To that end, fix our ϵ. We'll show that the δ we get from uniform continuity on stubs suffices to tell us how close two partial policies must be.

So, we have an ϵ. For uniform continuity, we need to find a δ where, regardless of which two partial policies πpa and π′pa we select, as long as they're δ or less apart, the sets (pr∞,πpa∗)−1(↑(Θst)(πpa)∩NF∩{≤⊙}) (and likewise for π′pa) are only ϵ apart. So, every point in the first preimage must have a point in the second preimage only ϵ distance away, and vice-versa. However, we can swap πpa and π′pa (our argument will be order-agnostic) to establish the other direction, so all we really need to do is to show that every point in the preimage associated with πpa is within ϵ of a point in the preimage associated with π′pa.

First, m:=logγ(δ) is the time at which two partial policies δ apart may start differing. Conversely, any two partial policies which only disagree at-or-after m are δ apart or less. Let π∗st be the policy stub defined as follows: take the inf of πpa and π′pa (the partial policy which is everything they agree on, which is going to perfectly mimic both of them up till time m), and clip things off at time m to make a stub. This is only δ apart from πpa and π′pa, because it perfectly mimics both of them up till time m, and then becomes undefined (so there's a difference at time m) Both πpa and π′pa are ≥π∗st.

Let M be some totally arbitrary point in (pr∞,πpa∗)−1(Θst(πpa)∩NF∩{≤⊙}). M is also in (pr∞,π∗st∗)−1(Θst(π∗st)∩NF∩{≤⊙}), because M projects down to some point in Θst(π∗st) that's nirvana-free.

Let π′npa, where n≥m, be the n'th stub in the fundamental sequence for π′pa. These form a chain starting at π∗st and ascending up to π′pa, and are all δ distance from π∗st.

Anyways, in Ma(∞), we can make a closed ball B≤ϵ of size ϵ around M. This restricts λ and b to a small range of values, so we can use the usual arguments to conclude that B≤ϵ is compact.

Further, because π∗st is δ or less away from π′npa, the two sets (pr∞,π∗st∗)−1(Θst(π∗st)∩NF∩{≤⊙}) and (pr∞,π′npa∗)−1(Θst(π′npa)∩NF∩{≤⊙}) are within ϵ of each other, so there's some point of the latter set that lies within our closed ϵ-ball.

Consider the set ⋂n≥m((pr∞,π′npa∗)−1(Θst(π′npa)∩NF∩{≤⊙})∩B≤ϵ)

the inner intersection is an intersection of closed and compact sets, so it's compact. Thus, this is an intersection of an infinite family of nonempty compact sets. To check the finite intersection property, just observe that since preimages of the sets ↑(Θst)(π′npa)∩NF∩{≤⊙} get smaller and smaller as n increases due to weak-consistency but always exist.

Pick some arbitrary point M′ from the intersection. it's ≤ϵ away from M since it's in the ϵ-ball. However, we still have to show that M′ is in (pr∞,π′pa∗)−1(↑(Θst)(π′pa)∩NF∩{≤⊙}) to get Hausdorff-continuity to go through.

To begin with, since M′ lies in our big intersection, we can project it down to any Θst(π′npa)∩NF∩{≤⊙}. Projecting it down to stage n makes M′n. Let M′∞ be the point in ↑(Θst)(π′pa)∩NF∩{≤⊙} defined by: ⋂n≥m(prπ′pa,πnst∗)−1(M′n)

Well, we still have to show that this set is nonempty, contains only one point, and that it's in ↑(Θst)(π′pa), and is nirvana-free, to sensibly identify it with a single point.

Nonemptiness is easy, just invoke Lemma 9. It lies in the usual intersections that define ↑(Θst)(π′pa), so we're good there. If it had nirvana, it'd manifest at some finite point, but all finite projections are nirvana-free, so it's nirvana-free. If it had more than one point in it, they differ at some finite stage, so we can project to a finite π′npa to get two different points, but they both project to M′n, so this is impossible. Thus, M′∞ is a legit point in the appropriate set. If the projection of M′ didn't equal M′∞, then we'd get two different points, which differ at some finite stage, so we could project down to separate them, but they both project to M′n for all n so this is impossible.

So, as a recap, we started with an arbitrary point M in (pr∞,πpa∗)−1(↑(Θst)(πpa))∩{≤⊙}, and got another point M′ that's only ϵ or less away and lies in (pr∞,π′pa∗)−1(↑(Θst)(π′pa))∩{≤⊙} This argument also works if we flip πpa and π′pa, so the two preimages are only ϵ or less apart in Hausdorff-distance.

So, given some ϵ, there's some δ where any two partial policies which are only δ apart have preimages only ϵ apart from each other in Hausdorff-distance. And thus, we have uniform continuity for the function mapping πpa to the set of a-measures over infinite histories which project down to ↑(Θst)(πpa)∩NF∩{≤⊙} Hausdorff-continuity is done.

Condition 7: Consistency.

Ok, we have two relevant things to check here. The first, very easy one, is that

↑(Θst)(πpa)=⋂πst≤πpa(prπpa,πst∗)−1(↑(Θst)(πst))

From earlier, we know that ↑(Θst)(πst)=Θst(πst), and from how ↑(Θst)(πpa) is defined, this is a tautology.

We'll split this into four stages. First, we'll show one subset direction holds in full generality. Second, we'll get the reverse subset direction for causal/surcausal. Third, we'll show it for policy stubs for pseudocausal/acausal, and finally we'll use that to show it for all partial policies for pseudocausal/acausal.

First, the easy direction. ↑(Θst)(πpa)⊇¯¯¯¯¯¯¯¯c.h(⋃π≥πpaprπ,πpa∗(↑(Θst)(π)))

If we pick an arbitrary M∈↑(Θst)(π), it projects down to Θst(πst) for all stubs πst below π. Since πpa≤π, it projects down to all stubs beneath πpa. Since projections commute, M projected down into Ma(F(πpa)) makes a point that lies in the preimage of all the Θst(πst) where πst≤πpa, so it projects down into ↑(Θst)(πpa).

This holds for all points in ↑(Θst)(π), so prπ,πpa∗(↑(Θst)(π))⊆↑(Θst)(πpa). This works for all π≥πpa, so it holds for the union, and then due to closure and convexity which we've already shown, we get that the closed convex hull of the projections lies in ↑(Θst)(πpa) too, establishing one subset direction in full generality.

Now, for phase 2, deriving ↑(Θst)(πpa)⊆¯¯¯¯¯¯¯¯c.h(⋃π≥πpaprπ,πpa∗(↑(Θst)(π))) in the causal/surcausal case.

First, observe that if π≥πpa, then πn≥πnpa. Fix some M∈↑(Θst)(πpa) and arbitrary π≥πpa. We'll establish the existence of a M′∈↑(Θst)(π) that projects down to M.

To begin with, M projects down to Mn in Θst(πnpa). Lock in a value for n, and consider the sequence that starts off M0,M1...Mn, and then, by causality for stubs and πn≥πnpa, you can find something in Θst(πn) that projects down onto Mn, and something in Θst(πn+1) that projects down onto that, and complete your sequence that way, making a sequence of points that all project down onto each other that climb up to π. By Lemma 10, we get a M′n∈↑(Θst)(π). You can unlock n now. All these M′n have the same λ and b value because projection preserves them, so we can isolate a convergent subsequence converging to some M′∈↑(Θst)(π).

Assume prπ,πpa∗(M′)≠M. Then we've got two different points. They differ at some finite stage, so there's some n where can project down onto Ma(F(πnpa)) to witness the difference, but from our construction process for M′, both M′ and M project down to Mn, and we get a contradiction.

So, since prπ,πpa∗(↑(Θst)(π))=↑(Θst(πpa)), this establishes the other direction, showing equality, and thus consistency, for causal/surcausal hypotheses.

For part 3, we'll solve the reverse direction for pseudocausal/acausal hypotheses in the case of stubs, getting ↑(Θst)(πst)⊆¯¯¯¯¯¯¯¯c.h(⋃π≥πpaprπ,πst∗(↑(Θst)(π)))

Since we're working in the Nirvana-free case and are working with stubs, we can wield Lemma 3

c.h((Θst(πst))min)=c.h((Θst(πst))xmin)

So, if we could just show that the union of the projections includes all the extreme minimal points, then when we take convex hull, we'd get the convex hull of the extreme minimal points, which by Lemma 3, would also nab all the minimal points as well. By Lemmas 11 and 12, our resulting convex hull of a union of projections from above would be upper-complete. It would also get all the minimal points, so it'd nabs the entire Θst(πst) within it and this would show the other set inclusion direction for pseudocausal/acausal stubs. Also, we've shown enough to invoke Lemma 20 to conclude that said convex hull is closed. Having fleshed out that argument, all we need that all extreme minimal points are captured by the union of the projections.

By our previously proved extreme minimal point condition, for every extreme minimal point M in Θst(πst), there's some π>πst and M′ in ↑(Θst)(π) that projects down to M, which shows that all extreme points are included, and we're good.

For part 4, we'll show that in the nirvana-free pseudocausal/acausal setting, we have

Fix some arbitrary M∈↑(Θst)(πpa). Our task is to express it as a limit of some sequence of points that are mixtures of stuff projected from above.

For this, we can run through the same exact proof path that was used in the part of the Lemma 21 proof about how the nirvana-free part of Θ(πpa) is a subset of closed convex hull of the projections of the nirvana-free parts of Θ(π), π≥πpa. Check back to it. Since we're working in the Nirvana-free case, we can apply it very straightforwardly. The stuff used in that proof path is the ability to project down and land in ↑(Θst)(πnpa) (we have that by how we defined ↑), Hausdorff-continuity (which we have), and stubs being the convex hull, not the closed convex hull, of projections of stuff above them (which we mentioned recently in part 3 of our consistency proof).

Thus, consistency is shown.

Condition 6: Normalization.

Ok, now that we have all this, we can tackle our one remaining condition, normalization. Then move on to the two optional conditions, causality and pseudocausality.

A key thing to remember is, in this setting, when you're doing EH(f), it's actually EH∩NF(f), because if Murphy picked a thing with nirvana in it, you'd get infinite value, which is not ok, so Murphy always picks a nirvana-free point.

Let's show the first part, that infπE↑(Θst)(π)(0)=0. This unpacks as:infπmin(λμ,b)∈↑(Θst)(π)∩NFb=0 and we have that infπstmin(λμ,b)∈Θst(πst)∩NFb=0

Projections preserves b values, so we can take some nirvana-free point with a b value of nearly 0, and project it down to Θst(π∅) (belief function of the empty policy-stub) where there's no nirvana possible because no events happen.

So, we've got b values of nearly zero in there. Do we have a point with a b value of exactly zero? Yes. it's closed, and has bounded minimals, so we can go "all positive functionals are minimized by a minimal point", to get a point with a b value of exactly zero. Then, we can invoke Lemma 21 (we showed consistency, extreme points, and all else required to invoke it) to decompose our point into the projections of nirvana-free stuff from above, all of which must have a b value of 0. So, there's a nirvana-free point in some policy with 0 b value.

Now for the other direction. Let's show that supπE↑(Θst)(π)(1)=1.

This unpacks as: supπmin(λμ,b)∈↑(Θst)(π)∩NF(λ+b)=1

We'll show this by disproving that the sup is <1, and disproving that the sup is >1.

First, assume that, regardless of π, min(λμ,b)∈↑(Θst)(π)∩NF(λ+b)≤1−ϵ Then, regardless of πst we can pick some totally arbitrary π>πst, and there's a nirvana-free point with a λ+b value of 1−ϵ or less. By consistency, we can project it down into Θ(πst), to get a nirvana-free point with a λ+b value of 1−ϵ or less. Thus, regardless of the stub we pick, there's nirvana-free points where Murphy can force a value of 1−ϵ or less, which contradicts supπstmin(λμ,b)∈Θst(πst)∩NF(λ+b)=1

What if it's above 1? Assume there's some π where min(λμ,b)∈↑(Θst)(π)∩NF(λ+b)≥1+ϵ

From uniform-continuity-Hausdorff, pick some δ to get a ϵ2 Hausdorff-distance or lower (for stuff obeying the λ⊙+b⊙ bound, which all minimal points of ↑(Θst)(π)∩NF do for all π). This δ specifies some extremely large n, consider πn. Now, consider the set of every policy π′ above πn. All of these are δ or less away from π. Also, remember that the particular sort of preimage-to-infinity that we used for Hausdorff-continuity slices away all the nirvana.

So, Murphy, acting on π, can only force a value of 1+ϵ or higher. Now, there can be no nirvana-free point in ↑(Θst)(π′) with λ+b<1+ϵ2. The reason for this is that, since π′ is δ or less away from π, there's a nirvana-free point in ↑(Θst)(π) that's ϵ2 away, and thus has λ+b<1+ϵ, which is impossible.

Ok, so all the nirvana-free points in ↑(Θst)(π′) where π′>πnst have λ+b≥1+ϵ2.

Now, since we have Lemma 21, we can go "hm, Θst(πn) equals the convex hull of the projections of ↑(Θst)(π′)∩NF. Thus, any minimal point with λ+b≤1 is a finite mix of nirvana-free stuff from above, one of which must have λ+b≤1. But we get a contradiction with the fact that there's no nirvana-free point from π′ above πn with a λ+b value that low, they're all ≥1+ϵ2"

So, since we've disproved both cases, supπE↑(Θst)(π)(1)=1. And we're done with normalization! On to causality and pseudocausality.

Condition C: Causality.

An "outcome function" of for ↑(Θst)(πpa) is a function that maps a πpa to a point in ↑(Θst)(πpa), s.t. for all πhipa,πlopa:prπhipa,πlopa∗(of(πhipa))=of(πlopa).

Causality is, if you have a M∈↑(Θst)(πpa), you can always find an outcome function of where of(πpa)=M. Sadly, all we have is causality over stubs. We'll be using the usual identification between ↑(Θst)(πst) and Θst(πst).

Anyways, fix a πpa and a point M in ↑(Θst)(πpa). Project M down to get a sequence Mn∈Θst(πnpa). By causality for stubs, we can find an ofn where, for all πst, ofn(πnpa)=Mn. Observe that there are countably many stubs, and no matter the n, all the λ and b values are the same because projection preserves those. We can view ofn as a sequence in

∏πst(Θst(πst)∩{(λ′μ,b′)|λ′=λ∧b′=b})

By stub closure, and a λ and b bound, this is a product of compact sets, and thus compact by Tychonoff (no axiom of choice needed, its just a countable product of compact metric spaces) so we can get a limiting ofst (because it's only defined over stubs).

An outcome function for stubs fixes an outcome function for all partial policies, by

of(πpa)=⋂n(prπpa,πnpa∗)−1(ofst(πnpa))

We've got several things to show now. We need to show that ofst is an outcome function, that of is well-defined, that of(πpa)=M, and that it's actually an outcome function.

For showing that ofst is an outcome function, observe that projection is continuous, and, letting n index our convergent subsequence of interest, regardless of stub πst, limn→∞ofn(πst)=ofst(πst). With this,

Now, let's show that of is well-defined. Since ofst is an outcome function, all the points project down onto each other, so we can invoke Lemma 9 to show that the preimage is nonempty. If the preimage had multiple points, we could project down to some finite stage to observe their difference, but nope, they always project to the same point. So it does pick out a single well-defined point, and it lies in ↑(Θst)(πpa) by being a subset of the defining sequence of intersection of preimages.

Does of(πpa)=M? Well, M projected down to all the Mn. If n≥m, then ofn(πmpa)=prπnpa,πmpa∗(ofn(πnpa))=prπnpa,πmpa∗(Mn)=Mm So, the limit specification ofst has ofst(πnpa)=Mn for all n. The only thing that projects down to make all the Mn is M itself, so of(πpa)=M.

Last thing to check: Is of an outcome function over partial policies? Well, if πhipa≥πlopa, then for all n, πhi,npa≥πlo,npa. Assume prπhipa,πlopa∗(of(πhipa))≠of(πlopa). Then, in that case, we can project down to some πlo,npa and they'll still be unequal. However, since projections commute, it doesn't matter whether you project down to πlopa and then to πlo,npa, or whether you project down to πhi,npa (making ofst(πhi,npa)), and then project down to πlo,npa (making ofst(πlo,npa)). Wait, hang on, this is the exact point that of(πlopa) projects down to, contradiction. Therefore it's an outcome function.

And we're done, we took an arbitrary πpa and M∈↑(Θst)(πpa), and got an outcome function of with of(πpa)=M, showing causality.

Condition P: Pseudocausality: If (m,b)∈↑(Θst)(πpa), and m's support is on Ma(FNF(π′pa)), then (m,b)∈↑(Θst)(π′pa).

But all we have is, if (m,b)∈Θst(πst) and m's support is on Ma(F(π′st)), then (m,b)∈Θst(π′st).

There's a subtlety here. Our exact formulation of pseudocausality we want is the condition supp(m)⊆F(π′pa), so if the measure is 0, then support is the empty set, which is trivially a subset of everything, then pseudocausality transfers it to all partial policies.

Ok, so let's assume that M∈↑(Θst)(πpa), and the measure part m has its support being a subset of Ma(FNF(π′pa)) but yet is not in ↑(Θst)(π′pa). Then, since this is an intersection of preimages from below, there should be some finite level π′npa that you can project M down to (it's present in Ma(FNF(π′pa)), just maybe not in ↑(Θst)(π′pa)) where the projection of M (call it M′n) lies outside Θst(π′npa) (lying outside the intersection of preimages)

This is basically "take m, chop it off at height n". However, since M∈↑(

Previous proof post is here.

Theorem 2: Isomorphism Theorem:For (causal, pseudocausal, acausal, surcausal)ΘstorΘωwhich fulfill finitary or infinitary analogues of all the defining conditions,↑(Θst)and↓(Θω)are (causal, pseudocausal, acausal, surcausal) hypotheses. Also,↑and→stdefine an isomorphism betweenΘandΘst, and↓and→ωdefine an isomorphism betweenΘandΘω.Proof sketch: The reason this proof is so horrendously long is that we've got almost a dozen conditions to verify, and some of them are quite nontrivial to show and will require sub-proof-sketches of their own! Our first order of business is verifying all the conditions for a full belief function for ↑(Θst). Then, we have to do it

all over againfor ↓(Θω). That comprises the bulk of the proof. Then, we have to show that taking a full belief function Θ and restricting it to the infinite/finite levels fulfills the infinite/finite analogues of all the defining conditions for a belief function on policies or policy-stubs, which isn't quite as bad. Once we're done with all the legwork showing we can derive all the conditions from each other, showing the actual isomorphism is pretty immediate from the Consistency condition of a belief function.Part 1:Let's consider ↑(Θπst). This is defined as: ↑(Θst)(πpa):=⋂πst≤πpa(prπpa,πst∗)−1(Θst(πst))

We'll show that all 9+2 defining conditions for a belief function are fulfilled for ↑(Θst). The analogue of the 9+2 conditions for a Θst is:

1: Stub Nirvana-free Nonemptiness: ∀πst:Θst(πst)∩NF≠∅

2: Stub Closure: ∀πst:Θst(πst)=¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯Θst(πst)

3: Stub Convexity. ∀πst:Θst(πst)=c.h(Θst(πst))

4: Stub Nirvana-free Upper-Completion.

∀πst:Θst(πst)∩NF=((Θst(πst)∩NF)+Msa(FNF(πst)))∩Ma(F(πst))

5: Stub Restricted Minimals:

∃λ⊙,b⊙∀πst:(λμ,b)∈(Θst(πst))min→λ+b≤λ⊙+b⊙

6: Stub Normalization: infπstEΘst(πst)(0)=0 and supπstEΘst(πst)(1)=1

7: Weak Consistency: ∀πlost,πhist≥πlost:prπhist,πlost∗(Θst(πhist))⊆Θst(πlost)

8: Stub Extreme Point Condition: for all M,πlost:

M∈(Θst(πlost))xmin∩NF→

∃π≥πlost∀πhist:(π>πhist≥πlost→(∃M′∈Θst(πhist)∩NF:prπhist,πlost∗(M′)=M))

9: Stub Uniform Continuity: The function πst↦(pr∞,πst∗)−1(Θst(πst)∩NF∩{≤⊙}) is uniformly continuous.

C: Stub Causality: ∀πst,M∈Θst(πst)∃of∀π′st:of(πst)=M∧of(π′st)∈Θst(π′st)

where the outcome function of is defined over all stubs.

P: Stub Pseudocausality:

∀πst,π′st:((M∈Θst(πst)∧supp(M)⊆FNF(π′st))→M∈Θst(π′st))

Let's begin showing the conditions. But first, note that since we have weak consistency, we can invoke Lemma 6 to reexpress ↑(Θst)(πpa) as ⋂n(prπpa,πnpa∗)−1(Θst(πnpa)) Where πnpa is the n'th member of the fundamental sequence of πpa.

Also note that, for all stubs, ↑(Θst(πst))=Θst(πst). We'll be casually invoking this all over the place and won't mention it further.

Proof: By Lemma 6 with weak consistency, ↑(Θst(πst))=⋂n≥m(prπst,πnst∗)−1(Θst(πnst)) Now, m can be anything we like, as long as it's finite. Set m to be larger than the maximum timestep that the stub is defined for. Then πnst=πst no matter what n is (since it's above m) and projection from a stub to itself is identity, so the preimage is exactly our original set Θst(πst).

We'll also be using another quick result. For all stubs πhist≥πlost, given stub causality,

prπhist,πlost∗(Θst(πhist))=Θst(πlost)

Proof: Fix an arbitrary point M∈Θst(πlost). By causality, we get an outcome function which includes M, giving us that there's something in Θst(πhist) that projects down onto M. Use weak consistency to get the other subset direction.

Condition 1:Nirvana-free Nonemptiness.Invoking Stub Nirvana-free Nonemptiness, ∀πst:Θst(πst)∩NF≠∅ so we get nirvana-free nonemptiness for ↑(Θst)(πst).

Now, assume πpa is not a stub. By stub-bounded-minimals, there is some λ⊙+b⊙ bound on the set of minimal points, regardless of stub. Let (Θst(πnpa))clip be Θst(πnpa)∩{≤⊙}∩NF

This contains all the minimal nirvana-free points for πnpa. This set is nonempty because we have stub nirvana-free nonemptiness, so a nirvana-free point M exists. We have stub-closure and stub minimal-boundedness, so we can step down to a minimal nirvana-free point below M, and it obeys the λ⊙+b⊙ bound.

Further, by weak consistency and projection preserving λ and b and nirvana-freeness,

prπn+1pa,πnpa∗((Θst(πn+1pa))clip)⊆(Θst(πnpa))clip

Invoking Lemma 9, the intersection of preimages of these is nonempty. It's also nirvana-free, because if there's nirvana somewhere, it occurs after finite time, so projecting down to some sufficiently large finite stage preserves the presence of Nirvana, but then we'd have a nirvana-containing point in a nirvana-free set (Θst(πnpa))clip, which is impossible. This is also a subset of the typical intersection of preimages used to define ↑(Θst)(πpa). Pick an arbitrary point in said intersection of preimages of clipped subsets.

Bam, we found a nirvana-free point in ↑(Θst)(πpa) and we're done.

Time for conditions 2 and 3,

Closure and Convexity. These are easy.↑(Θst)(πpa)=⋂n(prπpa,πnpa∗)−1(Θst(πnpa))

The preimage of a closed set (stub-closure) is a closed set, and the intersection of closed sets is closed, so we have closure.

Also, prπpa,πnst∗ is linear, so the preimage of a convex set (stub-convexity) is convex, and we intersect a bunch of convex sets so it's convex as well.

Condition 4:Nirvana-free upper completion.Let M∈↑(Θst)(πpa)∩NF. Let's check whether M+M∗ (assuming that's an a-measure and M∗ is nirvana-free) also lies in the set. A sufficient condition on this given how we defined things is that for all πnpa, prπpa,πnpa∗(M+M∗)∈Θst(πnpa), as that would certify that M+M∗ is in all the preimages.

prπpa,πnpa∗ is linear, so prπpa,πnpa∗(M+M∗)=prπpa,πnpa∗(M)+prπpa,πnpa∗(M∗)

The first component is in Θst(πst), obviously. And then, by stub nirvana-free-upper-completion, we have a nirvana-free a-measure plus a nirvana-free sa-measure (projection preserves nirvana-freeness), making a nirvana-free a-measure (projection preserves a-measures), so prπpa,πnpa∗(M)+prπpa,πnpa∗(M∗) is in Θst(πnpa)∩NF, and we're done.

Condition 5:Bounded-MinimalsSo, there is a critical λ⊙+b⊙ value by restricted-minimals for Θst

Fix a πpa, and assume that there is a minimal point M in ↑(Θst)(πpa) with a λ+b value that exceeds the bound. Project M down into each Θst(πnpa). Projection preserves λ and b so each of these projected points Mn lie above some Mminn.

Now, invoke Lemma 7 to construct a M′n∈Ma(F(πpa)) (or the nirvana-free variant) that lies below M, and projects down to Mminn. Repeat this for all n. All these M′n points are a-measures and have the standard λ⊙+b⊙ bound so they all lie in a compact set and we can extract a convergent subsequence, that converges to M′, which still obeys the λ⊙+b⊙ bound.

M′ is below M because ({M}−Msa(F(πpa)))∩Ma(F(πpa)) (or the nirvana-free variant) is a closed set. Further, by Lemma 10, M′ is in the defining sequence of intersections for ↑(Θst)(πpa). This witnesses that M isn't minimal, because we found a point below it that actually obeys the bounds. Thus, we can conclude minimal-point-boundedness for ↑(Θst).

Condition 6: Normalization. We'll have to go out of order here, this can't be shown at our current stage. We're going to have to address Hausdorff continuity first, then consistency, and solve normalization at the very end. Let's put that off until later, and just get extreme points.

Condition 8:Extreme point condition:The argument for this one isn't super-complicated, but the definitions are, so let's recap what condition we have and what condition we're trying to get.

Condition we have: for all M,πlost:

M∈(Θst(πlost))xmin∩NF→

∃π≥πlost∀πhist:(π>πhist≥πlost→(∃M′∈Θst(πhist)∩NF:prπhist,πlost∗(M′)=M))

Condition we want: for all M,πst,

M∈(Θst(πst))xmin∩NF→∃π>πst,M′:M′∈↑(Θst)(π)∩NF∧prπ,πst∗(M′)=M

Ok, so M∈(Θst(πst))xmin∩NF By the stub extreme point condition, there's a π≥πst, where, for all πhist that fulfill π>πhist≥πst, there's a M′∈Θst(πhist)∩NF, where prπhist,πst∗(M′)=M.

Lock in the π we have. We must somehow go from this to a M′∈Θ(π)∩NF that projects down to our point of interest. To begin with, let πn be the n'th member of the fundamental sequence for π. Past a certain point m, these start being greater than πst. The M′∈Θst(πn)∩NF which projects down to M that we get by the stub-extreme-point condition will be called M′lon. Pick some random-ass point in (prπ,πnst∗)−1(M′lon) and call it M′hin.

M′hin all obey the λ and b values of M, because it projects down to M. We get a limit point of them, M′, and invoking Lemma 10, it's also in ↑(Θst)(π). It also must be nirvana-free, because it's a limit of points that are nirvana-free for increasingly late times. It also projects down to M because the sequence M′hin was wandering around in the preimage of M, which is closed.

Condition 9:Hausdorff Continuity:Ok, this one is going to be fairly complicated. Remember, our original form is:

"The function πst↦(pr∞,πst∗)−1(Θst(πst)∩NF∩{≤⊙}) is uniformly continuous"

And the form we want is:

"The function πpa↦(pr∞,πpa∗)−1(↑(Θst)(πpa)∩NF∩{≤⊙}) is uniformly continuous"

Uniform continuity means that if we want an ϵ Hausdorff-distance between two preimages, there's a δ distance between partial policies that suffices to produce that. To that end, fix our ϵ. We'll show that the δ we get from uniform continuity on stubs suffices to tell us how close two partial policies must be.

So, we have an ϵ. For uniform continuity, we need to find a δ where, regardless of which two partial policies πpa and π′pa we select, as long as they're δ or less apart, the sets (pr∞,πpa∗)−1(↑(Θst)(πpa)∩NF∩{≤⊙}) (and likewise for π′pa) are only ϵ apart. So, every point in the first preimage must have a point in the second preimage only ϵ distance away, and vice-versa. However, we can swap πpa and π′pa (our argument will be order-agnostic) to establish the other direction, so all we really need to do is to show that every point in the preimage associated with πpa is within ϵ of a point in the preimage associated with π′pa.

First, m:=logγ(δ) is the time at which two partial policies δ apart may start differing. Conversely, any two partial policies which only disagree at-or-after m are δ apart or less. Let π∗st be the policy stub defined as follows: take the inf of πpa and π′pa (the partial policy which is everything they agree on, which is going to perfectly mimic both of them up till time m), and clip things off at time m to make a stub. This is only δ apart from πpa and π′pa, because it perfectly mimics both of them up till time m, and then becomes undefined (so there's a difference at time m) Both πpa and π′pa are ≥π∗st.

Let M be some totally arbitrary point in (pr∞,πpa∗)−1(Θst(πpa)∩NF∩{≤⊙}). M is also in (pr∞,π∗st∗)−1(Θst(π∗st)∩NF∩{≤⊙}), because M projects down to some point in Θst(π∗st) that's nirvana-free.

Let π′npa, where n≥m, be the n'th stub in the fundamental sequence for π′pa. These form a chain starting at π∗st and ascending up to π′pa, and are all δ distance from π∗st.

Anyways, in Ma(∞), we can make a closed ball B≤ϵ of size ϵ around M. This restricts λ and b to a small range of values, so we can use the usual arguments to conclude that B≤ϵ is compact.

Further, because π∗st is δ or less away from π′npa, the two sets (pr∞,π∗st∗)−1(Θst(π∗st)∩NF∩{≤⊙}) and (pr∞,π′npa∗)−1(Θst(π′npa)∩NF∩{≤⊙}) are within ϵ of each other, so there's some point of the latter set that lies within our closed ϵ-ball.

Consider the set ⋂n≥m((pr∞,π′npa∗)−1(Θst(π′npa)∩NF∩{≤⊙})∩B≤ϵ)

the inner intersection is an intersection of closed and compact sets, so it's compact. Thus, this is an intersection of an infinite family of nonempty compact sets. To check the finite intersection property, just observe that since preimages of the sets ↑(Θst)(π′npa)∩NF∩{≤⊙} get smaller and smaller as n increases due to weak-consistency but always exist.

Pick some arbitrary point M′ from the intersection. it's ≤ϵ away from M since it's in the ϵ-ball. However, we still have to show that M′ is in (pr∞,π′pa∗)−1(↑(Θst)(π′pa)∩NF∩{≤⊙}) to get Hausdorff-continuity to go through.

To begin with, since M′ lies in our big intersection, we can project it down to any Θst(π′npa)∩NF∩{≤⊙}. Projecting it down to stage n makes M′n. Let M′∞ be the point in ↑(Θst)(π′pa)∩NF∩{≤⊙} defined by: ⋂n≥m(prπ′pa,πnst∗)−1(M′n)

Well, we still have to show that this set is nonempty, contains only one point, and that it's in ↑(Θst)(π′pa), and is nirvana-free, to sensibly identify it with a single point.

Nonemptiness is easy, just invoke Lemma 9. It lies in the usual intersections that define ↑(Θst)(π′pa), so we're good there. If it had nirvana, it'd manifest at some finite point, but all finite projections are nirvana-free, so it's nirvana-free. If it had more than one point in it, they differ at some finite stage, so we can project to a finite π′npa to get two different points, but they both project to M′n, so this is impossible. Thus, M′∞ is a legit point in the appropriate set. If the projection of M′ didn't equal M′∞, then we'd get two different points, which differ at some finite stage, so we could project down to separate them, but they both project to M′n for all n so this is impossible.

So, as a recap, we started with an arbitrary point M in (pr∞,πpa∗)−1(↑(Θst)(πpa))∩{≤⊙}, and got another point M′ that's only ϵ or less away and lies in (pr∞,π′pa∗)−1(↑(Θst)(π′pa))∩{≤⊙} This argument also works if we flip πpa and π′pa, so the two preimages are only ϵ or less apart in Hausdorff-distance.

So, given some ϵ, there's some δ where any two partial policies which are only δ apart have preimages only ϵ apart from each other in Hausdorff-distance. And thus, we have uniform continuity for the function mapping πpa to the set of a-measures over infinite histories which project down to ↑(Θst)(πpa)∩NF∩{≤⊙} Hausdorff-continuity is done.

Condition 7:Consistency.Ok, we have two relevant things to check here. The first, very easy one, is that

↑(Θst)(πpa)=⋂πst≤πpa(prπpa,πst∗)−1(↑(Θst)(πst))

From earlier, we know that ↑(Θst)(πst)=Θst(πst), and from how ↑(Θst)(πpa) is defined, this is a tautology.

The other, much more difficult direction, is that

↑(Θst)(πpa)=¯¯¯¯¯¯¯¯c.h(⋃π≥πpaprπ,πpa∗(↑(Θst)(π)))

We'll split this into four stages. First, we'll show one subset direction holds in full generality. Second, we'll get the reverse subset direction for causal/surcausal. Third, we'll show it for policy stubs for pseudocausal/acausal, and finally we'll use that to show it for all partial policies for pseudocausal/acausal.

First, the easy direction. ↑(Θst)(πpa)⊇¯¯¯¯¯¯¯¯c.h(⋃π≥πpaprπ,πpa∗(↑(Θst)(π)))

If we pick an arbitrary M∈↑(Θst)(π), it projects down to Θst(πst) for all stubs πst below π. Since πpa≤π, it projects down to all stubs beneath πpa. Since projections commute, M projected down into Ma(F(πpa)) makes a point that lies in the preimage of all the Θst(πst) where πst≤πpa, so it projects down into ↑(Θst)(πpa).

This holds for all points in ↑(Θst)(π), so prπ,πpa∗(↑(Θst)(π))⊆↑(Θst)(πpa). This works for all π≥πpa, so it holds for the union, and then due to closure and convexity which we've already shown, we get that the closed convex hull of the projections lies in ↑(Θst)(πpa) too, establishing one subset direction in full generality.

Now, for phase 2, deriving ↑(Θst)(πpa)⊆¯¯¯¯¯¯¯¯c.h(⋃π≥πpaprπ,πpa∗(↑(Θst)(π))) in the causal/surcausal case.

First, observe that if π≥πpa, then πn≥πnpa. Fix some M∈↑(Θst)(πpa) and arbitrary π≥πpa. We'll establish the existence of a M′∈↑(Θst)(π) that projects down to M.

To begin with, M projects down to Mn in Θst(πnpa). Lock in a value for n, and consider the sequence that starts off M0,M1...Mn, and then, by causality for stubs and πn≥πnpa, you can find something in Θst(πn) that projects down onto Mn, and something in Θst(πn+1) that projects down onto

that, and complete your sequence that way, making a sequence of points that all project down onto each other that climb up to π. By Lemma 10, we get a M′n∈↑(Θst)(π). You can unlock n now. All these M′n have the same λ and b value because projection preserves them, so we can isolate a convergent subsequence converging to some M′∈↑(Θst)(π).Assume prπ,πpa∗(M′)≠M. Then we've got two different points. They differ at some finite stage, so there's some n where can project down onto Ma(F(πnpa)) to witness the difference, but from our construction process for M′, both M′ and M project down to Mn, and we get a contradiction.

So, since prπ,πpa∗(↑(Θst)(π))=↑(Θst(πpa)), this establishes the other direction, showing equality, and thus consistency, for causal/surcausal hypotheses.

For part 3, we'll solve the reverse direction for pseudocausal/acausal hypotheses in the case of stubs, getting ↑(Θst)(πst)⊆¯¯¯¯¯¯¯¯c.h(⋃π≥πpaprπ,πst∗(↑(Θst)(π)))

Since we're working in the Nirvana-free case and are working with stubs, we can wield Lemma 3

c.h((Θst(πst))min)=c.h((Θst(πst))xmin)

So, if we could just show that the union of the projections includes all the extreme minimal points, then when we take convex hull, we'd get the convex hull of the extreme minimal points, which by Lemma 3, would also nab all the minimal points as well. By Lemmas 11 and 12, our resulting convex hull of a union of projections from above would be upper-complete. It would also get all the minimal points, so it'd nabs the entire Θst(πst) within it and this would show the other set inclusion direction for pseudocausal/acausal stubs. Also, we've shown enough to invoke Lemma 20 to conclude that said convex hull is closed. Having fleshed out that argument, all we need that all extreme minimal points are captured by the union of the projections.

By our previously proved extreme minimal point condition, for every extreme minimal point M in Θst(πst), there's some π>πst and M′ in ↑(Θst)(π) that projects down to M, which shows that all extreme points are included, and we're good.

For part 4, we'll show that in the nirvana-free pseudocausal/acausal setting, we have

↑(Θst)(πpa)⊆¯¯¯¯¯¯¯¯c.h(⋃π≥πpaprπ,πpa∗(↑(Θst)(π)))

Fix some arbitrary M∈↑(Θst)(πpa). Our task is to express it as a limit of some sequence of points that are mixtures of stuff projected from above.

For this, we can run through the same exact proof path that was used in the part of the Lemma 21 proof about how the nirvana-free part of Θ(πpa) is a subset of closed convex hull of the projections of the nirvana-free parts of Θ(π), π≥πpa. Check back to it. Since we're working in the Nirvana-free case, we can apply it very straightforwardly. The stuff used in that proof path is the ability to project down and land in ↑(Θst)(πnpa) (we have that by how we defined ↑), Hausdorff-continuity (which we have), and stubs being the convex hull, not the closed convex hull, of projections of stuff above them (which we mentioned recently in part 3 of our consistency proof).

Thus, consistency is shown.

Condition 6:Normalization.Ok, now that we have all this, we can tackle our one remaining condition, normalization. Then move on to the two optional conditions, causality and pseudocausality.

A key thing to remember is, in this setting, when you're doing EH(f), it's actually EH∩NF(f), because if Murphy picked a thing with nirvana in it, you'd get infinite value, which is not ok, so Murphy always picks a nirvana-free point.

Let's show the first part, that infπE↑(Θst)(π)(0)=0. This unpacks as:infπmin(λμ,b)∈↑(Θst)(π)∩NFb=0 and we have that infπstmin(λμ,b)∈Θst(πst)∩NFb=0

Projections preserves b values, so we can take some nirvana-free point with a b value of nearly 0, and project it down to Θst(π∅) (belief function of the empty policy-stub) where there's no nirvana possible because no events happen.

So, we've got b values of nearly zero in there. Do we have a point with a b value of exactly zero? Yes. it's closed, and has bounded minimals, so we can go "all positive functionals are minimized by a minimal point", to get a point with a b value of exactly zero. Then, we can invoke Lemma 21 (we showed consistency, extreme points, and all else required to invoke it) to decompose our point into the projections of nirvana-free stuff from above,

allof which must have a b value of 0. So, there's a nirvana-free point in some policy with 0 b value.Now for the other direction. Let's show that supπE↑(Θst)(π)(1)=1.

This unpacks as: supπmin(λμ,b)∈↑(Θst)(π)∩NF(λ+b)=1

We'll show this by disproving that the sup is <1, and disproving that the sup is >1.

First, assume that, regardless of π, min(λμ,b)∈↑(Θst)(π)∩NF(λ+b)≤1−ϵ Then, regardless of πst we can pick some totally arbitrary π>πst, and there's a nirvana-free point with a λ+b value of 1−ϵ or less. By consistency, we can project it down into Θ(πst), to get a nirvana-free point with a λ+b value of 1−ϵ or less. Thus, regardless of the stub we pick, there's nirvana-free points where Murphy can force a value of 1−ϵ or less, which contradicts supπstmin(λμ,b)∈Θst(πst)∩NF(λ+b)=1

What if it's above 1? Assume there's some π where min(λμ,b)∈↑(Θst)(π)∩NF(λ+b)≥1+ϵ

From uniform-continuity-Hausdorff, pick some δ to get a ϵ2 Hausdorff-distance or lower (for stuff obeying the λ⊙+b⊙ bound, which all minimal points of ↑(Θst)(π)∩NF do for all π). This δ specifies some extremely large n, consider πn. Now, consider the set of every policy π′ above πn. All of these are δ or less away from π. Also, remember that the particular sort of preimage-to-infinity that we used for Hausdorff-continuity slices away all the nirvana.

So, Murphy, acting on π, can only force a value of 1+ϵ or higher. Now, there can be no nirvana-free point in ↑(Θst)(π′) with λ+b<1+ϵ2. The reason for this is that, since π′ is δ or less away from π, there's a nirvana-free point in ↑(Θst)(π) that's ϵ2 away, and thus has λ+b<1+ϵ, which is impossible.

Ok, so all the nirvana-free points in ↑(Θst)(π′) where π′>πnst have λ+b≥1+ϵ2.

Now, since we have Lemma 21, we can go "hm, Θst(πn) equals the convex hull of the projections of ↑(Θst)(π′)∩NF. Thus, any minimal point with λ+b≤1 is a finite mix of nirvana-free stuff from above, one of which

musthave λ+b≤1. But we get a contradiction with the fact that there's no nirvana-free point from π′ above πn with a λ+b value that low, they're all ≥1+ϵ2"So, since we've disproved both cases, supπE↑(Θst)(π)(1)=1. And we're done with normalization! On to causality and pseudocausality.

Condition C:Causality.An "outcome function" of for ↑(Θst)(πpa) is a function that maps a πpa to a point in ↑(Θst)(πpa), s.t. for all πhipa,πlopa:prπhipa,πlopa∗(of(πhipa))=of(πlopa).

Causality is, if you have a M∈↑(Θst)(πpa), you can always find an outcome function of where of(πpa)=M. Sadly, all we have is causality over stubs. We'll be using the usual identification between ↑(Θst)(πst) and Θst(πst).

Anyways, fix a πpa and a point M in ↑(Θst)(πpa). Project M down to get a sequence Mn∈Θst(πnpa). By causality for stubs, we can find an ofn where, for all πst, ofn(πnpa)=Mn. Observe that there are countably many stubs, and no matter the n, all the λ and b values are the same because projection preserves those. We can view ofn as a sequence in

∏πst(Θst(πst)∩{(λ′μ,b′)|λ′=λ∧b′=b})

By stub closure, and a λ and b bound, this is a product of compact sets, and thus compact by Tychonoff (no axiom of choice needed, its just a countable product of compact metric spaces) so we can get a limiting ofst (because it's only defined over stubs).

An outcome function for stubs fixes an outcome function for all partial policies, by

of(πpa)=⋂n(prπpa,πnpa∗)−1(ofst(πnpa))

We've got several things to show now. We need to show that ofst is an outcome function, that of is well-defined, that of(πpa)=M, and that it's actually an outcome function.

For showing that ofst is an outcome function, observe that projection is continuous, and, letting n index our convergent subsequence of interest, regardless of stub πst, limn→∞ofn(πst)=ofst(πst). With this,

prπhist,πlost∗(ofst(πhist))=prπhist,πlost∗(limn→∞ofn(πhist))=limn→∞prπhist,πlost∗(ofn(πhist))

=limn→∞ofn(πlost)=ofst(πlost)

Now, let's show that of is well-defined. Since ofst is an outcome function, all the points project down onto each other, so we can invoke Lemma 9 to show that the preimage is nonempty. If the preimage had multiple points, we could project down to some finite stage to observe their difference, but nope, they always project to the same point. So it does pick out a single well-defined point, and it lies in ↑(Θst)(πpa) by being a subset of the defining sequence of intersection of preimages.

Does of(πpa)=M? Well, M projected down to all the Mn. If n≥m, then ofn(πmpa)=prπnpa,πmpa∗(ofn(πnpa))=prπnpa,πmpa∗(Mn)=Mm So, the limit specification ofst has ofst(πnpa)=Mn for all n. The only thing that projects down to make all the Mn is M itself, so of(πpa)=M.

Last thing to check: Is of an outcome function over partial policies? Well, if πhipa≥πlopa, then for all n, πhi,npa≥πlo,npa. Assume prπhipa,πlopa∗(of(πhipa))≠of(πlopa). Then, in that case, we can project down to some πlo,npa and they'll still be unequal. However, since projections commute, it doesn't matter whether you project down to πlopa and then to πlo,npa, or whether you project down to πhi,npa (making ofst(πhi,npa)), and then project down to πlo,npa (making ofst(πlo,npa)). Wait, hang on, this is the exact point that of(πlopa) projects down to, contradiction. Therefore it's an outcome function.

And we're done, we took an arbitrary πpa and M∈↑(Θst)(πpa), and got an outcome function of with of(πpa)=M, showing causality.

Condition P:Pseudocausality: If (m,b)∈↑(Θst)(πpa), and m's support is on Ma(FNF(π′pa)), then (m,b)∈↑(Θst)(π′pa).But all we have is, if (m,b)∈Θst(πst) and m's support is on Ma(F(π′st)), then (m,b)∈Θst(π′st).

There's a subtlety here. Our exact formulation of pseudocausality we want is the condition supp(m)⊆F(π′pa), so if the measure is 0, then support is the empty set, which is trivially a subset of everything, then pseudocausality transfers it to all partial policies.

Ok, so let's assume that M∈↑(Θst)(πpa), and the measure part m has its support being a subset of Ma(FNF(π′pa)) but yet is not in ↑(Θst)(π′pa). Then, since this is an intersection of preimages from below, there should be some finite level π′npa that you can project M down to (it's present in Ma(FNF(π′pa)), just maybe not in ↑(Θst)(π′pa)) where the projection of M (call it M′n) lies outside Θst(π′npa) (lying outside the intersection of preimages)

This is basically "take m, chop it off at height n". However, since M∈↑(