Here are the previous two posts.

Now, what about updates? We'll use $u_h^g$ (and suppress the ${\pi }_{\neg h}$ that should be there) as shorthand for the function that maps $(m,b)$ over $\Theta ({\pi }_{\neg h}\bullet {\pi }_{pa})$ to $\left(c\right(m|h),b+m(0{★}^{h}g\left)\right)$ in $M^a\left(F\right({\pi }_{pa}\left)\right)$ (or the nirvana-free or sur variant of this), and also use $u_h^g$ as a function from belief functions to belief functions (just map all the sets through)

Lemma 27: When updating, the closure adds no nirvana-free points that weren't present originally if Nonemptiness, Nirvana-free upper-completion and closure holds originally (works in the sur-case too)

Proof sketch: We take a sequence $M_n$ limiting to $M$, and then take a preimage point of $M_n$, go to a minimal below it, find a limit point in our original set by Compactness, and map it back through the update, getting a point below $M$. Then, we find what we need to add to that to get $M$, and find something above our limit point that maps to $M$, so we didn't actually need closure anyways because we made $M$ as an image of a nirvana-free point present in the original set.

Proof: Fix a sequence $M_n$ in $u_h^g\left(\Theta \right)\left({\pi }_{pa}\right)$ (but without the defining closure part in the end) that limit to $M$ which is nirvana-free.

Every $M_n$ has a preimage point $M_n^{\prime }\in \Theta ({\pi }_{\neg h}\bullet {\pi }_{pa})$ with no nirvana off-h. For each $M_n^{\prime }$, find a minimal point $M_n^{{}^{\prime }lo}$ below it, which have a ${\lambda }^{\odot }+b^{\odot }$ bound by bounded-minimals, so we can find a convergent subsequence limiting to $M^{{}^{\prime }lo}$ (actually, might not be minimal, still a limit of minimal points, though). Shoving the $M_n^{{}^{\prime }lo}$ (and the limit point) back through the update (which is a continuous function), we get a sequence $M_n^{lo}$ limiting to $M^{lo}$ (the thing you get from pushing $M^{{}^{\prime }lo}$ through the update).

Since $M_n^{\prime }$ lies above $M_n^{{}^{\prime }lo}$ (upper-completion ordering), then updating preserves that property, because the update function is linear. Thus, all the $M_n^{lo}$ lie below their corresponding $M_n$. Now, we can invoke Lemma 16 to conclude that $M^{lo}$ lies below $M$. It lies below a nirvana-free point, so $M^{{}^{\prime }lo}$ is nirvana-free as well. Now, we just need to show nirvana-free upper-completion because $M=M^{lo}+M^{\ast }$.

We can take $M^{{}^{\prime }lo}$ and add on $(m^{\ast },b^{\ast })$ (extend the measure back to the original domain by sticking an h prefix on everything, and saying that the measure is 0 everywhere else), making an a-measure that's in $\Theta ({\pi }_{\neg h}\bullet {\pi }_{pa})$, by nirvana-free uppper-completion there. By linearity, and the update not affecting $(m^{\ast },b^{\ast })$ (it's 0 outside of h so the $g$ outside of h doesn't get to contribute anything to the b term when we update), updating $M^{{}^{\prime }lo}+(m^{\ast },b^{\ast })$ makes $M^{lo}+M^{\ast }=M$. So, if a nirvana-free point appears post-update (with closure), then it'll appear post-update (without closure).

Lemma 28: raw-update-then-project equals project-then-raw-update.

Take some $(m,b)$. We want to show that:

$p{r}_{\ast }^{{\pi }_{pa}^{hi},{\pi }_{pa}^{lo}}\left(c\right(m|h),b+m(0{★}^{h}g\left)\right)$

$=(c\left(p{r}_{\ast }^{({\pi }_{\neg h}\bullet {\pi }_{pa}^{hi}),({\pi }_{\neg h}\bullet {\pi }_{pa}^{lo})}\right(m\left)|h\right),b+\left(p{r}_{\ast }^{({\pi }_{\neg h}\bullet {\pi }_{pa}^{hi}),({\pi }_{\neg h}\bullet {\pi }_{pa}^{lo})}\right(m\left)\right)(0{★}^{h}g\left)\right)$

First, $\left(p{r}_{\ast }^{({\pi }_{\neg h}\bullet {\pi }_{pa}^{hi}),({\pi }_{\neg h}\bullet {\pi }_{pa}^{lo})}\right(m\left)\right)\left(0{★}^{h}g\right)=m\left(0{★}^{h}g\right)$ This is because the projection down doesn't change the measure at all outside of h, and we're evaluating a function that's 0 inside h. So, that takes care of the b term. Also, projection preserves the b term, so our desired equality is:

$\left(p{r}_{\ast }^{{\pi }_{pa}^{hi},{\pi }_{pa}^{lo}}\right(c\left(m|h\right)),b+m(0{★}^{h}g\left)\right)=(c\left(p{r}_{\ast }^{({\pi }_{\neg h}\bullet {\pi }_{pa}^{hi}),({\pi }_{\neg h}\bullet {\pi }_{pa}^{lo})}\right(m\left)|h\right),b+m(0{★}^{h}g\left)\right)$

For the measure term, the first is "restrict to on-h histories, clip off the h prefixes, and project down", and the second is "project down the on-h histories accordingly, then restrict to on-h histories and clip off the h prefix", which are obviously equal.

Proposition 9: For causal, surcausal, pseudocausal and acausal hypotheses, updating them produces a causal, surcausal, pseudocausal or acausal hypothesis as long as renormalization doesn't fail.

Proof sketch: What we can do is consider the "raw update", show that it preserves all nice properties except for renormalization, and then show that the renormalization terms in the update are the proper renormalization terms to use. Thus, we'll define our raw update $u_h^g\left(\Theta \right)$ via: $u_h^g\left(\Theta \right)\left({\pi }_{pa}\right)$ is: $\Theta ({\pi }_{\neg h}\bullet {\pi }_{pa})\cap NF\text{off h}$, mapped through the following function: $(m,b)\mapsto \left(c\right(m|h),b+m(0{★}^{h}g\left)\right)$ And then you take the closure of the resulting set at the end.

So, we take our partial policy ${\pi }_{pa}$ and glue it to the off-h-partial policy ${\pi }_{\neg h}$, go to that part of the original belief function, strip off everything that has nirvana off-h (for Murphy shall not select those, and properly, that should make the b term infinite post-update), slice off the part of the measure off-h, strip off the h prefix from those histories, and go "ok, our utility function is $g$, let's take the expected utility off-h and fold it into the b term"

If we can get all nice properties but renormalization, we're good, just appeal to Lemma 24. As for showing the conditions: We're in for something almost as bad as one of the directions of the Isomorphism theorem. Nonemptiness, closure, and convexity are trivial, upper completion, pseudocausality, and bounded-minimals are easy and the extreme point condition and causality are moderately tricky.

For the extreme point condition, Step 1 is establishing that $u_h^f\left(\Theta \right)\left({\pi }_{pa}\right)\cap NF$ equals the closed convex hull of nirvana-free projections from above by an argument that makes sense when you sketch it out but may be difficult if you don't have it sketched out, step 2 is using Hausdorff-continuity and Lemma 20 to turn it into an ordinary convex hull, and finally, arguing that a nirvana-free exteme point must have come from a nirvana-free point from above via step 2.

For causality, we can (for the most part) just go back to $\Theta$, get an outcome function there, and map it back through the update to get an outcome function, the hard part is netting the limit points, which requires a limit of outcome functions. But because we want a countable product of sets to get sequential compactness from Tychonoff, we have to work with stubs, which adds some extra complexity.

Hausdorff-continuity is just hellishly hard, we need to show that the preimages of the sets post-update are the updates of the preimages of sets pre-update, and then combine that with some fancy work with minimal points and upper completions and using two different characterizations of uniform continuity at once via Lemma 15, and a touch of functional analysis. There's way too many interacting points and sets in this one.

But easily the biggest grind is Consistency. We have 4 subset directions to show, each of which requires their own separate fancy argument, and two of them require splitting into a nirvana-containing/causal case and a nirvana-free case, so it's a 6-part proof. A good chunk of complexity arises because we have to take closure in the nirvana-containing case, an issue which goes away if we just let Nirvana be 1 reward forever. Let's begin.

Condition 1: Nirvana-free Nonemptiness: 

This is trivial. Just pick a nirvana-free point in $\Theta ({\pi }_{\neg h}\bullet {\pi }_{pa})$ by nirvana-free nonemptiness, and update, to get one in $u_h^g\left(\Theta \right)\left({\pi }_{pa}\right)$.

Conditions 2,3: Closure, Convexity: 

Closure is a tautology since we took the closure. For convexity, the closure of a convex set is convex, and $u_h^g$ is a linear function, so it maps convex sets to convex sets.

Condition 4: Nirvana-free Upper-Completeness: 

First, invoke Lemma 27 to see that all nirvana-free points must have been present in the raw $u_h^g\left(\Theta \right({\pi }_{\neg h}\bullet {\pi }_{pa}\left)\right)$ set to begin with, without the closure. What we want is that, if $M^{\prime }=M+M^{\ast }$, and $M$ lies in the raw updated set and is nirvana-free, and $M^{\prime }$ is a nirvana-free a-measure, then $M^{\ast }$ lies in the updated set as well.

Find a $M^{\prime \prime }$ that maps to $M$ after updating. It must be nirvana-free, because the nirvana either occurs without h as a prefix (which is forbidden because all that stuff gets clipped off and doesn't get pushed through the update), or the nirvana occurs with h as a prefix, but then it'd show up in the measure component of $M$ post-update, contradicting its nirvana-freeness. Now, we can consider $M^{\prime \prime }+(m^{\ast },b^{\ast })$ (basically, $M^{\ast }$, but we take the measure back by sticking an h prefix on everything, and saying that it's 0 off-h). This is present in $\Theta ({\pi }_{\neg h}\bullet {\pi }_{pa})$, by nirvana-free upper completion. By linearity of updating, and $m^{\ast }$ having no measure in any off-h area where it'd get picked up by $g$, this updates to $M+M^{\ast }$, witnessing that $M^{\prime }$ lies in the image of the update, so we get nirvana-free upper completion.

Condition 5: Bounded-Minimals:

 For bounded-minimals, we can pull the Lemma 16 trick of taking our $M$ of interest that $M_n$ limit to, taking a preimage $M_n^{\prime }$ for each $M_n$, finding a minimal $M_n^{{}^{\prime }lo}$ below each $M_n^{\prime }$ (which obeys a ${\lambda }^{\odot }+b^{\odot }$ bound and also has no nirvana off-h), getting a limit point $M^{{}^{\prime }lo}$ (still no nirvana off-h) by compactness, and pushing the sequence through the update, to get a sequence $M_n^{lo}$ below $M_n$ limiting to $M^{lo}$ which is below $M$ (Lemma 16) Now, we just have to check up on the $\lambda +b$ values of our $M_n^{lo}$ sequence, and show that they respect the ${\lambda }^{\odot }+b^{\odot }$ bound, to transfer this to $M^{lo}$. The raw update deletes measure from off-h, and assigns it the value that $g$ does, which is 1 or less, so any increase in b correspond to an equal-or-greater decrease in $\lambda$, so the $M_n^{lo}$ all obey the ${\lambda }^{\odot }+b^{\odot }$ bound as well. Thus, the limit point $M^{lo}$ obeys the bound, and it's below our original $M$, so any minimal must obey the ${\lambda }^{\odot }+b^{\odot }$ bound.

Condition 7: Consistency: 

This is going to be extremely tedious and difficult to show, it's a 6-part proof. The first 3 parts are devoted to showing that $u_h^g\left(\Theta \right)\left({\pi }_{pa}\right)=\overline{c.h}\left({\bigcup }_{\pi \ge {\pi }_{pa}}p{r}_{\ast }^{\pi ,{\pi }_{pa}}\right(u_h^g\left(\Theta \right)\left(\pi \right)\left)\right)$

Part 1 is showing $u_h^g\left(\Theta \right)\left({\pi }_{pa}\right)\subseteq \overline{c.h}\left({\bigcup }_{\pi \ge {\pi }_{pa}}p{r}_{\ast }^{\pi ,{\pi }_{pa}}\right(u_h^g\left(\Theta \right)\left(\pi \right)\left)\right)$ In the nirvana-free pseudocausal/acausal case.

Let $M$ be in $u_h^g\left(\Theta \right)\left({\pi }_{pa}\right)$. By Lemma 27, since we're working in the nirvana-free case, we didn't need to take the closure, it won't add any points that aren't there anyways. So, $M$ has a preimage point $M^{\prime }\in \Theta ({\pi }_{\neg h}\bullet {\pi }_{pa})$ that maps to it. By consistency for $\Theta$, $M^{\prime }$ lies in the closed convex hull of projections of policies down from above, so there are points in the convex hull of projections of policies that are arbitrarily close to $M^{\prime }$, fix some sequence $M_n^{\prime }$ of points in the convex hull of projections down from policies above that limit to $M^{\prime }$. Mapping these through the raw update (which is continuous) we get a sequence $M_n$ of points in $u_h^g\left(\Theta \right)\left({\pi }_{pa}\right)$ that limit to $M$.

All these policies above $({\pi }_{\neg h}\bullet {\pi }_{pa})$ have the form $({\pi }_{\neg h}\bullet \pi )$. So, $M_n^{\prime }$ can be written as a mix of finitely many $M_{i,n}^{\prime }$, which are the projections of $M_{i,n}^{{}^{\prime }\infty }$ from above, in policies. Update those, getting points $M_{i,n}^{\infty }$ in $u_h^g\left(\Theta \right)\left(\pi \right)$. These project down to $M_{i,n}$, which mix back together to make... $M_n$. This is because of Lemma 28, that update-then-project equals project-then-update. Also, mix-then-project equals project-then-mix. Remember, $M_n$ is made by: "Project $M_{i,n}^{{}^{\prime }\infty }$ down to make $M_{i,n}^{\prime }$, then mix to make $M_n^{\prime }$, then update."

So, we can go project-mix-update equals mix-project-update equals mix-update-project equals update-mix-project equals update-project-mix, which is the process "update the $M_{i,n}^{{}^{\prime }\infty }$ to $M_{i,n}^{\infty }$, project down to $M_{i,n}$, mix to $M_n$"

The first equality is linearity of projection, the second equality is Lemma 28, the third equality is linearity of updating, the final equality is linearity of projection again.

Anyways, taking stock of what we've done, we have a sequence $M_n$ limiting to our $M$ of interest, and every $M_n$ is crafted by taking points from finitely many $u_h^g\left(\Theta \right)\left(\pi \right)$, projecting them down, and mixing them. Therefore, our $M\in u_h^g\left(\Theta \right)\left({\pi }_{pa}\right)$ lies in the closed convex hull of projections down from above.

Part 2: we'll show this again, but in the nirvana-containing case, where we'll leverage causality.

Fix a $M\in u_h^g\left(\Theta \right)\left({\pi }_{pa}\right)$ (with closure). There's a sequence $M_n$ that limits to it, that lies in the same set, but without closure, so we can take preimage points $M_n^{\prime }\in \Theta ({\pi }_{\neg h}\bullet {\pi }_{pa})$ that update to make $M_n$. By causality, fix some arbitrary policy above $({\pi }_{\neg h}\bullet {\pi }_{pa})$, which can be expressed as $({\pi }_{\neg h}\bullet \pi )$, where $\pi \ge {\pi }_{pa}$. Anyways, we can take $M_n^{\prime }$, and use causality to get an outcome function $of$, to get a $M_n^{{}^{\prime }\infty }\in \Theta ({\pi }_{\neg h}\bullet \pi )$ that projects down to $M_n^{\prime }$. We don't have to worry about nirvana off-h, because $M_n^{\prime }$ already specifies everything that happens off-h and it says no nirvana occurs in that case. So, $M_n^{{}^{\prime }\infty }$ can be updated to make a $M_n^{\infty }$ in $u_h^g\left(\Theta \right)\left(\pi \right)$. By Lemma 28, this must project down to $M_n$. So, all our $M_n$ lie in the projection of $u_h^g\left(\Theta \right)\left(\pi \right)$, and since $M$ is a limit point of that sequence, it must lie in the closed convex hull of projections.

And we've shown that $u_h^g\left(\Theta \right)\left({\pi }_{pa}\right)\subseteq \overline{c.h}\left({\bigcup }_{\pi \ge {\pi }_{pa}}p{r}_{\ast }^{\pi ,{\pi }_{pa}}\right(u_h^g\left(\Theta \right)\left(\pi \right)\left)\right)$

And have taken care of 2 of our 6 parts. Now for the reverse direction, that

$u_h^g\left(\Theta \right)\left({\pi }_{pa}\right)\supseteq \overline{c.h}\left({\bigcup }_{\pi \ge {\pi }_{pa}}p{r}_{\ast }^{\pi ,{\pi }_{pa}}\right(u_h^g\left(\Theta \right)\left(\pi \right)\left)\right)$

Thankfully, this can be done with a general argument that isn't sensitive to the presence of Nirvana.

Part 3: Fix a $M$ in the closed convex hull, which has a sequence $M_n$ limiting to it that's in the convex hull of projections down from above. the $M_n$ shatter into finitely many $M_{i,n}$, which are projections of $M_{i,n}^{\infty }\in u_h^g\left(\Theta \right)\left({\pi }_i\right)$. Now, these aren't necessarily preimage points, they may have been added in the closure. Thus, we can perturb by $2^{-n}$ or less if needed to make a $M_{i,n}^{{}^{\prime }\infty }$ which does have a preimage point. Projecting these down to $M_{i,n}^{\prime }$ and mixing, crafts a $M_n^{\prime }$ point that is within $2^{-n}$ of $M_n$ (remember, projection doesn't expand distance), so the sequence $M_n^{\prime }$ still has $M$ as a limit point (it gets arbitrarily close to a sequence that gets arbitrarily close to $M$). If we can show that all the $M_n^{\prime }$ lie in $u_h^g\left(\Theta \right)\left({\pi }_{pa}\right)$, then by closure, we'll get that $M$ lies in the same set so we're done.

Ok, so we have $M_{i,n}^{{}^{\prime }\infty }\in u_h^g\left(\Theta \right)\left({\pi }_i\right)$, that project down and mix to make $M_n^{\prime }$, and importantly, we crafted them so they're produceable without closure. Thus, they have preimage points $M_{i,n}^{{}^{\prime \prime }\infty }\in \Theta ({\pi }_{\neg h}\bullet {\pi }_i)$ (that lack nirvana off-h) Project them down to make $M_{i,n}^{\prime \prime }\in \Theta ({\pi }_{\neg h}\bullet {\pi }_{pa})$, and mix them to make a $M_n^{\prime \prime }$ in the same set (which still lacks nirvana off-h), and this updates to make $M_n^{\prime }$ via Lemma 28, as we'll show shortly.

Starting with the $M_{i,n}^{{}^{\prime \prime }\infty }$, we know that update, project, mix equals $M_n^{\prime }$ via going $M_{i,n}^{{}^{\prime }\infty }$, $M_{i,n}^{\prime }$, $M_n^{\prime }$. Then, update-project-mix equals project-update-mix equals project-mix-update, which is the path we took. Therefore, all the $M_n^{\prime }$ lie in $u_h^g\left(\Theta \right)\left({\pi }_{pa}\right)$, which is closed, so $M$ (arbitrary in the closed convex hull of projections) lies in the same set, establishing the reverse subset direction and thus equality,

$u_h^g\left(\Theta \right)\left({\pi }_{pa}\right)=\overline{c.h}\left({\bigcup }_{\pi \ge {\pi }_{pa}}p{r}_{\ast }^{\pi ,{\pi }_{pa}}\right(u_h^g\left(\Theta \right)\left(\pi \right)\left)\right)$

Part 4: Now that we're halfway done,let's look at the "intersection of preimages of stubs from below" direction of consistency, $u_h^g\left(\Theta \right)\left({\pi }_{pa}\right)\subseteq {\bigcap }_n\left(p{r}_{\ast }^{{\pi }_{pa},{\pi }_{pa}^n}{)}^{-1}\right(u_h^g\left(\Theta \right)\left({\pi }_{pa}^n\right))$. If we ignore the closure part and work with the raw update set sans closure, we can fix a $M$ in $u_h^g\left(\Theta \right)\left({\pi }_{pa}^{hi}\right)$, take a preimage point in $\Theta ({\pi }_{\neg h}\bullet {\pi }_{pa}^{hi})$, project it down to $\Theta ({\pi }_{\neg h}\bullet {\pi }_{pa}^{lo})$ by consistency, then update it to get exactly the projection of $M$ (again, Lemma 28) Then, when we take the closure, we can just take our $M$ in the closure, fix a sequence in the raw update set sans closure $M_n$ that limits to $M$, project down, getting $M_n^{\prime }$ in the raw update set $u_h^g\left(\Theta \right)\left({\pi }_{pa}^{lo}\right)$ sans closure, and then the limit point $M^{\prime }$ lies in $u_h^g\left(\Theta \right)\left({\pi }_{pa}^{lo}\right)$ by closure, and by continuity of projection, $M^{\prime }$ is the projection of $M$.

Since the sets get bigger as you go down, we can invoke Lemma 6 to swap out the intersection of preimages of all stubs below you, for the intersection of preimages of stubs of the form ${\pi }_{pa}^n$, this will be important later.

Now, it's trivial to show that $u_h^g\left(\Theta \right)\left({\pi }_{pa}\right)\subseteq {\bigcap }_n\left(p{r}_{\ast }^{{\pi }_{pa},{\pi }_{pa}^n}{)}^{-1}\right(u_h^g\left(\Theta \right)\left({\pi }_{pa}^n\right))$ because we've established that projecting down makes a subset, and projection commutes, so any $M\in u_h^g\left(\Theta \right)\left({\pi }_{pa}\right)$ projects down into $u_h^g\left(\Theta \right)\left({\pi }_{pa}^n\right)$ for all n.

All that's left now is the reverse subset direction,

$u_h^g\left(\Theta \right)\left({\pi }_{pa}\right)\supseteq {\bigcap }_n\left(p{r}_{\ast }^{{\pi }_{pa},{\pi }_{pa}^n}{)}^{-1}\right(u_h^g\left(\Theta \right)\left({\pi }_{pa}^n\right))$

Sadly, this one will require us splitting into the nirvana-containing (and thus causal) cases and the nirvana-free cases, and it's a really difficult one to show. 

Part 5: Let's address the nirvana-free case, we'll use a nifty trick to control the size of the preimage points we select.

Ok, let's say you have a $M$ with some ${\lambda }_1$ and $b_1$ value. And you take $M^{\prime }$ that's a preimage point, but its $\lambda$ and $b$ values are just... waaay too high. We want to have a preimage point with reasonable values, in order to apply bounding arguments. What you do, is find a minimal-point $M^{min}$ below $M^{\prime }$, so $M^{\prime }=M^{min}+M^{\ast }$. Now, what you do, is swap out $M^{\ast }$ ie $(m^{\ast },b^{\ast })$, for $(m^{\ast }|h,b^{\ast }+m^{\ast }(0{★}^{h}g\left)\right)$. This is an sa-measure, because

$b^{\ast }+m^{\ast }\left(0{★}^{h}g\right)+\left(m^{\ast }|h{)}^{-}\right(1)=b^{\ast }+m^{\ast }(0{★}^{h}g)+m^{\ast -}(1{★}^{h}0)$

$\ge m^{\ast -}\left(0{★}^{h}g\right)+m^{\ast -}\left(1{★}^{h}0\right)=b^{\ast }+m^{\ast -}\left(1{★}^{h}g\right)\ge b^{\ast }+m^{\ast -}\left(1\right)\ge 0$

Now, consider updating $M^{min}+(m^{\ast }|h,b^{\ast }+m^{\ast }(0{★}^{h}g\left)\right)$ instead (it's an a-measure, it has less negative parts than $M^{\ast }$, and is present by nirvana-free upper-completion). This gets you the update of $M^{min}$, plus... $\left(c\right(m^{\ast }|h),b^{\ast }+m^{\ast }(0{★}^{h}g\left)\right)$ (remember, 0 measure off-h). Which is the exact same thing you'd get by updating $M^{\ast }$, so when we updated our new sum, we hit $M$ exactly.

However, this sum is special, because we can stick some decent bounds on its $\lambda$ and $b$ value! For starters, its $b$ value is less than $b_1$ (updating only adds on b-mass, and it updates to $M$). And as for the $\lambda$ value... well, $M^{min}$ has its $\lambda$ bounded above by ${\lambda }^{\odot }$ (of the original $\Theta$) due to being a minimal point. And in the worst-case, all of the measure in $M$ came from the thing we added, so $m^{\ast }|h$ has a measure of ${\lambda }_1$ or less. So our bound on the $\lambda$ value is ${\lambda }^{\odot }+{\lambda }_1$.

Armed with this knowledge, we can begin to prove the last bit of consistency in the nirvana-free case. Take a $M$ in the intersection of preimages. It projects down to make $M_n$ in $u_h^g\left(\Theta \right)\left({\pi }_{pa}^n\right)$. Projection preserves $\lambda$ and $b$, so they all have the same ${\lambda }_1,b_1$ bounds. Because we don't need to close in the nirvana-free case, we get a preimage point of $M_n^{\prime }$ in $\Theta ({\pi }_{\neg h}\bullet {\pi }_{pa}^n)$ From our earlier considerations, we can always pick $M_n^{\prime }$ such that its $\lambda$ is $\le {\lambda }^{\odot }+{\lambda }_1$, and its $b$ is $\le b_1$, although we'll be using a bound of $max(b_1,b^{\odot })$.

Now, we're going to have to be extremely careful here. Let the point $M_{n,j}^{\prime }$ be defined as:

If $j<n$, then $M_{n,j}^{\prime }$ is some arbitrary point in $\Theta ({\pi }_{\neg h}\bullet {\pi }_{pa}^n)$, with $\lambda$ equal to or below ${\lambda }^{\odot }+{\lambda }_1$, and $b$ equal to or below $max(b_1,b^{\odot })$, which always exists by all minimal points obeying the ${\lambda }^{\odot }+b^{\odot }$  bound.

If $j=n$, then $M_{n,j}^{\prime }=M_n^{\prime }$.

If $j>n$, then $M_{n,j}^{\prime }=p{r}_{\ast }^{({\pi }_{\neg h},{\pi }_{pa}^j),({\pi }_{\neg h},{\pi }_{pa}^n)}\left(M_j^{\prime }\right)$

Then, the tuple of $M_{n,j}^{\prime }$ for all n is a point in:

${\prod }_n\left(\Theta \right({\pi }_{\neg h}\bullet {\pi }_{pa}^n)\cap \{(\lambda \mu ,b)|\lambda \le {\lambda }^{\odot }+{\lambda }_1,b\le max(b_1,b^{\odot })\left\}\right)$

Equipped with the product topology. In particular, this is a product of compact sets, so by Tychonoff's theorem, it's compact. Thus, we can get a convergent subsequence of the tuples. On this subsequence, all the $M_{n,j}^{\prime }$ converge to a limit point $M_{n,\infty }^{\prime }$, regardless of n.

Also, $M_{n,\infty }^{\prime }$ projects down to $M_{m,\infty }^{\prime }$ if $n\ge m$, because for large enough j, the projection of $M_{n,j}^{\prime }$ will always be $M_{m,j}^{\prime }$, and by continuity of projection, the projection of $M_{n,\infty }^{\prime }$ must be $M_{m,\infty }^{\prime }$

Ok, so we've got an infinite sequence of $M_{n,\infty }^{\prime }$ for all n that all project down onto each other. Another nice feature is that updating $M_{n,\infty }^{\prime }$ produces $M_n$. This is because, when j climbs high enough, $M_{j,j}^{\prime }$ projects down to $M_{n,j}^{\prime }$, and $M_{j,j}^{\prime }$ is just $M_j^{\prime }$ which updates to $M_j$, which projects down to $M_n$. By Lemma 28, update-then-project equals project-then-update, so $M_{n,j}^{\prime }$ must update to $M_n$, for all sufficiently high j. The preimage of a single point is closed, so past a certain point, the $M_{n,j}^{\prime }$ are wandering around in the preimage of $M_n$, so $M_{n,\infty }^{\prime }$ also updates to $M_n$.

Now, our next step is, does the $M_{n,\infty }^{\prime }$ sequence in $\Theta ({\pi }_{\neg h}\bullet {\pi }_{pa}^n)$ pick out a single point $M^{\prime }$ in $\Theta ({\pi }_{\neg h}\bullet {\pi }_{pa})$ that projects down accordingly? Yes it does. Just intersect all the preimages of single points, they're nested in each other and compact so the finite intersection property holds, and if the intersection wasn't composed of a single point, you'd have two distinct points with a difference at some finite time, but projecting down to any finite time the two distinct points are identical, so there can only be a single point in the intersection. Further, it must lie in $\Theta ({\pi }_{\neg h}\bullet {\pi }_{pa})$, because you can project it down to $M_{n,\infty }^{\prime }$ in $\Theta ({\pi }_{\neg h}\bullet {\pi }_{pa}^n)$ for any n, which, by consistency for $\Theta$, you can also project down to $\Theta \left(\right({\pi }_{\neg h}\bullet {\pi }_{pa}{)}^n)$ (projecting down further), so it's present in the intersection of all the preimages, certifying that it's in the appropriate set.

Now, finally... does $M^{\prime }$, when updated, produce $M$, certifying that the point in the intersection of preimages is also in the raw update set? Well, let's say it didn't. Then we get a $M^{\prime \prime }$ that's not equal to $M$, so projecting down to some finite n should suffice to observe that. However, projecting $M^{\prime \prime }$ and $M$ down produces... $M_n$. This is because of Lemma 28, update-then-project equals project-then-update. Projecting $M^{\prime }$ down makes $M_{n,\infty }^{\prime }$, which updates to $M_n$.

So, no finite stage suffices to observe the difference between the updated form of $M^{\prime }$ and $M$ itself, so they must be identical, certifying $u_h^g\left(\Theta \right)\left({\pi }_{pa}\right)\supseteq {\bigcap }_n\left(p{r}_{\ast }^{{\pi }_{pa},{\pi }_{pa}^n}{)}^{-1}\right(u_h^g\left(\Theta \right)\left({\pi }_{pa}^n\right))$ for the nirvana-free case.

Part 6: Let's move to the nirvana-case  where we can leverage causality. We'll be showing this in a rather nonstandard way. We're going to pick a $\pi \ge {\pi }_{pa}$, and show that our $M$ of interest in the intersection of preimages can be written as a limit of points projected down from $u_h^g\left(\Theta \right)\left(\pi \right)$, establishing that $M$ lies in the closed convex hull of points from above, which we've already shown equals $u_h^g\left(\Theta \right)\left({\pi }_{pa}\right)$.

Ok, so $M$ is in the intersection of preimages. Project it down to all the $u_h^g\left(\Theta \right)\left({\pi }_{pa}^n\right)$, getting a batch of points $M_n$ from them. This is the raw update set, so within $2^{-n}$ or less distance from $M_n$, there's a $M_n^{\prime }$ in the raw update sans closure, which has a preimage point $M_n^{\prime \prime }$ that lies in $\Theta ({\pi }_{\neg h}\bullet {\pi }_{pa}^n)$.

Now, pick some arbitrary policy above ${\pi }_{\neg h}\bullet {\pi }_{pa}$, which can be written as ${\pi }_{\neg h}\bullet \pi$. Moving on even further, by causality, we can get a point $M_n^{{}^{\prime \prime }\infty }\in \Theta ({\pi }_{\neg h}\bullet \pi )$ that projects down to $M_n^{\prime \prime }$. Update $M_n^{{}^{\prime \prime }\infty }$ to get a $M_n^{{}^{\prime }\infty }\in u_h^g\left(\Theta \right)\left(\pi \right)$, which then (by our earlier thing about how a set equaled the closed convex hull of projections down from above), projects down to a $M_n^{{}^{\prime }hi}\in u_h^g\left(\Theta \right)\left({\pi }_{pa}\right)$.

Now, we can ask whether the sequence $M_n^{{}^{\prime }hi}$ limits to $M$ itself. $u_h^g\left(\Theta \right)\left({\pi }_{pa}\right)$ is closed, so this would certify that $M$ lies in the appropriate set.

First, observe that the projection of $M_n^{{}^{\prime }hi}$ down to ${\pi }_{pa}^n$ is $M_n^{\prime }$. This is by Lemma 28, update-then-project equals project-then-update. $M_n^{{}^{\prime \prime }\infty }$ projects down to $M_n^{\prime \prime }$, which updates to $M_n^{\prime }$, so $M_n^{\prime }$ must be what you get by updating $M_n^{{}^{\prime \prime }\infty }$ to $M_n^{{}^{\prime }\infty }$, and projecting down to ${\pi }_{pa}$ (making $M_n^{{}^{\prime }hi}$), and projecting that down to ${\pi }_{pa}^n$.

Now, because projection preserves the b term, and $M_n^{{}^{\prime }hi}$ projects down to $M_n^{\prime }$ which is within $2^{-n}$ of $M_n$ (not much of a difference in the b terms), and $M_n$ has the same b term as $M$, we can certify convergence of the b term at least. Now for convergence of the measure components. Again, $M_n^{{}^{\prime }hi}$ projects down to $M_n^{\prime }$ which is within $2^{-n}$ of $M_n$ (not much difference before timestep n, shrinking increasingly low), and $M_n$ perfectly mimics what $M$ does before timestep n. So, $M_n^{{}^{\prime }hi}$ behaves increasingly closely to $M$ for everything before time n, which increases without bound. Increasingly close matches on increasingly large initial segments of what happens mean that $M_n^{{}^{\prime }hi}$ must limit to $M$ itself, certifying that $M$ lies in $u_h^g\left(\Theta \right)\left({\pi }_{pa}\right)$ for the causal cases.

That's the last bit we needed! We're finally done with consistency now. This just leaves the hausdorff-condition and the extreme-point condition and pseudocausality and causality.

Condition 9: Hausdorff-continuity: 

What we need to do for our setup to even approach this is to show that updating the preimage of the nirvana-free part of $\Theta ({\pi }_{\neg h}\bullet {\pi }_{pa})$, produces exactly the preimage of the nirvana-free part of $u_h^g\left(\Theta \right)\left({\pi }_{pa}\right)$.

One direction, we can get easily. If you fix a $M^{{}^{\prime }\infty }$ in the preimage of the nirvana-free part of $\Theta ({\pi }_{\neg h}\bullet {\pi }_{pa})$, it projects down to a $M^{\prime }\in \Theta ({\pi }_{\neg h}\bullet {\pi }_{pa})\cap NF$, that updates to a $M\in u_h^g\left(\Theta \right)\left({\pi }_{pa}\right)$, then by Lemma 28, project-then-update equals update-then-project, so $M^{{}^{\prime }\infty }$ must update to a $M^{\infty }$ that projects down to $M$, certifying that updating the preimage of the nirvana-free part of $\Theta ({\pi }_{\neg h}\bullet {\pi }_{pa})$ produces a subset of the preimage of the nirvana-free part of $u_h^g\left(\Theta \right)\left({\pi }_{pa}\right)$.

In the other direction, fix a $M^{\infty }$ in the preimage of the nirvana-free part of $u_h^g\left(\Theta \right)\left({\pi }_{pa}\right)$. It projects down to a $M$ in $u_h^g\left(\Theta \right)\left({\pi }_{pa}\right)\cap NF$, and by Lemma 27, $M$ wasn't introduced in the closure, so it has a preimage point $M^{\prime }\in \Theta ({\pi }_{\neg h}\bullet {\pi }_{pa})\cap NF$.

Now, how do we extend $M^{\prime }$ to craft a $M^{{}^{\prime }\infty }$ that updates to $M^{\infty }$? Well, we can split into two parts. What happens on-h, and what happens off-h? For the off-h part, the post-update part has everything folded into the b term, while the pre-update part has an actual measure specified everywhere. Thus, our $M^{{}^{\prime }\infty }$ should have the same off-h part as $M^{\prime }$ to project down accordingly, so updating folds it into the same b term as $M^{\infty }$ has.

Now, for the on-h part, it's a bit more complicated. $M^{\infty }$ specified what happens for all infinite histories with h as a prefix. However, $M$ and $M^{\prime }$ only specify part of that data, but fortunately agree on that part. Thus, for $M^{{}^{\prime }\infty }$, you can just extend with the conditional probabilities of $M^{\infty }$, to perfectly mimic it on-h. This makes a $M^{{}^{\prime }\infty }$ in the preimage that updates to $M^{\infty }$.

Ok, so the appropriate preimages for Hausdorff-continuity (post-update) are made exactly by updating the preimages for Hausdorff-continuity (pre-update). Now, updating is a continuous linear operator. We're mapping from the Banach space $M^{\pm }\left(\right(A\times O{)}^{\omega })\oplus R$ to the Banach space $M^{\pm }\left(h\right(A\times O{)}^{\omega })\oplus R$.

Well, this isn't quite right, your actions and observations may vary depending on where you are in history, but the general thing of "restrict to signed measures over infinite histories with h as a prefix" still checks out. Updating is still a continuous linear operator between Banach spaces, by Lemma 8 of Section 1.

Also, all continuous linear operators between Banach spaces are bounded, and thus Lipschitz-continuous at 0, and thus Lipschitz-continuous everywhere due to linearity. So, when we push two points that are only $ϵ$ apart through the update, they're now $Cϵ$ apart at most, where $C$ is a finite constant.

We're going to have a lot of points. Unusually enough, we'll be using the standard formulation of Hausdorff-continuity for our original $\Theta$, that for all $ϵ$, there's a $\delta$ where two partial policies ${\pi }_{pa}$ and ${\pi }_{pa}^{\prime }$ that are $\delta$ or less apart have $\left(p{r}_{\ast }^{\infty ,{\pi }_{pa}}{)}^{-1}\right(\Theta \left({\pi }_{pa}\right)\cap NF\cap \{\le \odot \})$ (and the analogous set for ${\pi }_{pa}^{\prime }$) being only $ϵ$ apart in Hausdorff-distance.

Fixing your $ϵ$, you're gonna want $\delta$ to be low enough to force a $\frac{ϵ}{C}$ difference between the clipped preimages, and $\delta <\frac{ϵ}{C}$. It's highly advised to sketch out how our points interact and what sets they're in. A superscript of infinity will be used to denote points in the preimages of the $u_h^g\left(\Theta \right)\left({\pi }_{pa}\right)$ sets (or $\Theta ({\pi }_{\neg h}\bullet {\pi }_{pa})$) (ie, at the infinite levels), and a superscript of "u" specifies post-update while its lack is pre-update.

Anyways, here's our points.

$M^{u,\infty }$ lies in the preimage of $u_h^g\left(\Theta \right)\left({\pi }_{pa}\right)\cap NF$, and it's our point that we want to find a point nearby. $\lambda$ will refer to the $\lambda$ value of this thing.

Projecting $M^{u,\infty }$ down to $u_h^g\left(\Theta \right)\left({\pi }_{pa}\right)\cap NF$ makes $M^u$.

We can find a minimal point below $M^u$, $M^{u,min}$ in $u_h^g\left(\Theta \right)\left({\pi }_{pa}\right)\cap NF$. $M^{u,min}+M^{u,\ast }=M^u$.

A nirvana-free point wasn't introduced by the closure, and it has a minimal point in its preimage, so there's a $M^{min}$ in $\Theta \left({\pi }_{pa}\right)$ that updates to $M^{u,min}$, and respects the ${\lambda }^{\odot }+b^{\odot }$ bound of $\Theta$.

Let $M^{lo}$ be defined as $M^{min}+\left(\right(m^{u,\ast }{)}^{-},-\left(m^{u,\ast }{)}^{-}\right(1\left)\right)$. We're extending the negative-measure part of $M^{u,\ast }$ back to its original domain by sticking an h prefix on everything, and saying it's 0 everywhere else. this is an a-measure that lies in $\Theta ({\pi }_{\neg h}\bullet {\pi }_{pa})\cap NF\cap \{\le \odot \}$ (because $M^{min}$ respects the ${\lambda }^{\odot }+b^{\odot }$ bound, and the thing that we added has a $\lambda +b$ value of 0)

Let $M$ be defined as $M^{lo}+\left(\right(m^{u,\ast }{)}^{+},b^{u,\ast }+\left(m^{u,\ast }{)}^{-}\right(1\left)\right)$, it also lies in the same set Updating $M$ makes $M^u$, because, unpacking $M$, it's $M^{min}+M^{u,\ast }$, which updates to $M^{u,min}+M^{u,\ast }$ which adds up to make $M^u$.