Proposition 39: Given a crisp infradistribution ${\zeta }^{\square }$ over $N$, an infrakernel $K$ from $N$ to infradistributions over $X$, and suggestively abbreviating $K\left(i\right)$ as $H_i$(hypothesis $i$) and $K_{\ast }\left({\zeta }^{\square }\right)$ as $E_{{\zeta }^{\square }}{H}_i$ (your infraprior where you have Knightian uncertainty over how to mix the hypotheses), then
$\left(\right(E_{{\zeta }^{\square }}{H}_i\left){|}^{g}L\right)\left(f\right)=\frac{E_{{\zeta }^{\square }}\left(P_{H_i}^g\right(L)\cdot (H_i{|}^{g}L\left)\right(f)+H_i(0{★}^{L}g\left)\right)-E_{{\zeta }^{\square }}\left(H_i\right(0{★}^{L}g\left)\right)}{E_{{\zeta }^{\square }}\left(H_i\right(1{★}^{L}g\left)\right)-E_{{\zeta }^{\square }}\left(H_i\right(0{★}^{L}g\left)\right)}$

Proof: Assume that $L$ and $g$ are functions of type $X\to [0,1]$ and $X\to R$ respectively, ie, likeliehood and utility doesn't depend on which hypothesis you're in, just what happens. First, unpack our abbreviations and what an update means. 

$\left(\right(E_{{\zeta }^{\square }}{H}_i\left){|}^{g}L\right)\left(f\right)=\left(K_{\ast }\right({\zeta }^{\square }\left){|}^{g}L\right)\left(f\right)=\frac{K_{\ast }\left({\zeta }^{\square }\right)\left(f{★}^{L}g\right)-K_{\ast }\left({\zeta }^{\square }\right)\left(0{★}^{L}g\right)}{K_{\ast }\left({\zeta }^{\square }\right)\left(1{★}^{L}g\right)-K_{\ast }\left({\zeta }^{\square }\right)\left(0{★}^{L}g\right)}$

Then use the definition of an infrakernel pushforward.

$=\frac{{\zeta }^{\square }(\lambda i.K(i\left)\right(f{★}^{L}g\left)\right)-{\zeta }^{\square }(\lambda i.K(i\left)\right(0{★}^{L}g\left)\right)}{{\zeta }^{\square }(\lambda i.K(i\left)\right(1{★}^{L}g\left)\right)-{\zeta }^{\square }(\lambda i.K(i\left)\right(0{★}^{L}g\left)\right)}$

For the next thing, we're just making types a bit more explicit, $f,L,g$ only depend on $x$, not $i$.

$=\frac{{\zeta }^{\square }(\lambda i.K(i\left)\right(\lambda x.f{★}^{L}g\left)\right)-{\zeta }^{\square }(\lambda i.K(i\left)\right(\lambda x.0{★}^{L}g\left)\right)}{{\zeta }^{\square }(\lambda i.K(i\left)\right(\lambda x.1{★}^{L}g\left)\right)-{\zeta }^{\square }(\lambda i.K(i\left)\right(\lambda x.0{★}^{L}g\left)\right)}$

Then we pack the semidirect product back up.

$=\frac{({\zeta }^{\square }⋉K)(\lambda i,x.f{★}^{L}g)-({\zeta }^{\square }⋉K)(\lambda i,x.0{★}^{L}g)}{({\zeta }^{\square }⋉K)(\lambda i,x.1{★}^{L}g)-({\zeta }^{\square }⋉K)(\lambda i,x.0{★}^{L}g)}$

And pack the update back up.

$=\left(\right({\zeta }^{\square }⋉K\left){|}^{g}L\right)\left(f\right)$

At this point, we invoke the Infra-Disintegration Theorem.

$=({\zeta }^{{\square }^{\prime }}⋉K^{\prime })\left(f\right)$

$={\zeta }^{{\square }^{\prime }}(\lambda i.K^{\prime }(i\left)\right(f\left)\right)$

We unpack what our new modified prior is, via the Infra-Disintegration Theorem.

$=\frac{{\zeta }^{\square }(\lambda i.\alpha (i\left)K^{\prime }\right(i\left)\right(f)+\beta (i\left)\right)-({\zeta }^{\square }⋉K)\left(0{★}^{L}g\right)}{({\zeta }^{\square }⋉K)\left(1{★}^{L}g\right)-({\zeta }^{\square }⋉K)\left(0{★}^{L}g\right)}$

and unpack the semidirect product.

$=\frac{{\zeta }^{\square }(\lambda i.\alpha (i\left)K^{\prime }\right(i\left)\right(f)+\beta (i\left)\right)-{\zeta }^{\square }(\lambda i.K(i\left)\right(0{★}^{L}g\left)\right)}{{\zeta }^{\square }(\lambda i.K(i\left)\right(1{★}^{L}g\left)\right)-{\zeta }^{\square }(\lambda i.K(i\left)\right(0{★}^{L}g\left)\right)}$

Now we unpack $\alpha$ and $\beta$.

$=\frac{{\zeta }^{\square }(\lambda i.P_{K\left(i\right)}^g(L)\cdot K^{\prime }(i\left)\right(f)+K(i\left)\right(0{★}^{L}g\left)\right)-{\zeta }^{\square }(\lambda i.K(i\left)\right(0{★}^{L}g)}{{\zeta }^{\square }(\lambda i.K(i\left)\right(1{★}^{L}g\left)\right)-{\zeta }^{\square }(\lambda i.K(i\left)\right(0{★}^{L}g)}$

And unpack what $K^{\prime }$ is

$=\frac{{\zeta }^{\square }(\lambda i.P_{K\left(i\right)}^g(L)\cdot (K\left(i\right){|}^{g}L\left)\right(f)+K(i\left)\right(0{★}^{L}g\left)\right)-{\zeta }^{\square }(\lambda i.K(i\left)\right(0{★}^{L}g\left)\right)}{{\zeta }^{\square }(\lambda i.K(i\left)\right(1{★}^{L}g\left)\right)-{\zeta }^{\square }(\lambda i.K(i\left)\right(0{★}^{L}g\left)\right)}$

And reabbreviate $K\left(i\right)$ as $H_i$,

$=\frac{{\zeta }^{\square }(\lambda i.P_{H_i}^g(L)\cdot (H_i{|}^{g}L\left)\right(f)+H_i(0{★}^{L}g\left)\right)-{\zeta }^{\square }(\lambda i.H_i(0{★}^{L}g)}{{\zeta }^{\square }(\lambda i.H_i(1{★}^{L}g\left)\right)-{\zeta }^{\square }(\lambda i.H_i(0{★}^{L}g)}$

And then pack it back up into a suggestive form as a sort of expectation.

$=\frac{E_{{\zeta }^{\square }}\left(P_{H_i}^g\right(L)\cdot (H_i{|}^{g}L\left)\right(f)+H_i(0{★}^{L}g\left)\right)-E_{{\zeta }^{\square }}\left(H_i\right(0{★}^{L}g\left)\right)}{E_{{\zeta }^{\square }}\left(H_i\right(1{★}^{L}g\left)\right)-E_{{\zeta }^{\square }}\left(H_i\right(0{★}^{L}g)}$

And we're done.

Proposition 40: If a likelihood function $L:X\to [0,1]$ is 0 when $f\left(x\right)<a$, and $f\ge 0$ and $a>0$, then $h(L\cdot a)\le h\left(f\right)$

$h(L\cdot a)={inf}_{(\lambda \mu ,b)\in H}\lambda \mu (L\cdot a)+b={inf}_{(\lambda \mu ,b)\in H}a\lambda \mu \left(L\right)+b$

And then we apply Markov's inequality, that for any probability distribution,

$\mu \left({\text{1}}_{f\left(x\right)\ge a}\right)\le \frac{\mu \left(f\right)}{a}$

Also,${\text{1}}_{f\left(x\right)\ge a}\ge L$ (because $L$ is 0 when $f\left(x\right)<a$), so monotonicity means that

$\mu \left(L\right)\le \frac{\mu \left(f\right)}{a}$

So, we can get:

$\le {inf}_{(\lambda \mu ,b)\in H}a\lambda \frac{\mu \left(f\right)}{a}+b={inf}_{(\lambda \mu ,b)\in H}\lambda \mu \left(f\right)+b=h\left(f\right)$

And we're done.

Proposition 41: The IKR-metric is a metric.

So, symmetry is obvious, as is one direction of identity of indiscernibles (that the distance from an infradistribution to itself is 0). That just leaves the triangle inequality and the other direction of identity of indiscernibles. For the triangle inequality, observe that for any particular $f$ (instead of the supremum), it would fulfill the triangle inequality, and then it's an easy exercise for the reader to verify that the same property applies to the supremum, so the only tricky part is the reverse direction of identity of indiscernibles, that two infradistributions which have a distance of 0 are identical.

First, if $d_{IKR}(h,h^{\prime })=0$, then $h$ and $h^{\prime }$ must perfectly agree on all the Lipschitz functions. And then, because uniformly continuous functions are the uniform limit of Lipschitz functions, $h$ and $h^{\prime }$ must perfectly agree on all the uniformly continuous functions.

Now, we're going to need a somewhat more sophisticated argument. Let's say that the sequence $f_n$ is uniformly bounded and limits to $f$ in $C_B\left(X\right)$ equipped with the compact-open topology (ie, we get uniform convergence of $f_n$ to $f$ on all compact sets). Then, for any infradistributions, $h\left(f_n\right)$ will limit to $h\left(f\right)$. Here's why. For any $ϵ$, there's some compact set $C_{ϵ}$ that accounts for almost all of why a function inputted into an infradistribution has the value it does. Then, what we can do is realize that $h\left(f_n\right)$ will, in the limit, be incredibly close to $h\left(f\right)$, due to $f_n$ and $f$ disagreeing by a bounded amount outside the set $C_{ϵ}$ and only disagreeing by a tiny amount on the set $C_{ϵ}$, and the Lipschitzness of $h$.

Further, according to this mathoverflow answer, uniformly continuous functions are dense in the space of all continuous functions when $C_B\left(X\right)$ is equipped with the compact-open topology, so given any function $f$, we can find a sequence of uniformly continuous functions $f_n$ limiting to $f$ in the compact-open topology, and then,

$h\left(f\right)={lim}_{n\to \infty }h\left(f_n\right)={lim}_{n\to \infty }{h}^{\prime }\left(f_n\right)=h^{\prime }\left(f\right)$

And so, $h$ and $h^{\prime }$ agree on all continuous functions, and are identical, if they have a distance of 0, giving us our last piece needed to conclude that $d_{IKR}$ is a metric.

Proposition 42: The IKR-metric for infradistributions is strongly equivalent to the Hausdorff distance (w.r.t. the KR-metric) between their corresponding infradistribution sets.

Let's show both directions of this. For the first one, if the Hausdorff-distance between $H,H^{\prime }$ is $d_{hau}(H,H^{\prime })$, then for all a-measures $(m,b)$ in $H$, there's an a-measure $(m^{\prime },b^{\prime })$ in $H^{\prime }$ that's only $d_{hau}(H,H^{\prime })$ or less distance away, according to the KR-metric (on a-measures).

Now, by LF-duality, a-measures in H correspond to hyperplanes above $h$. Two a-measures being $d_{hau}(H,H^{\prime })$ apart means, by the definition of the KR-metric for a-measures, that they will assign values at most $d_{hau}(H,H^{\prime })$ distance apart for 1-Lipschitz functions in $[-1,1]$.

So, translating to the concave functional view of things, $H$ and $H^{\prime }$ being $d_{hau}(H,H^{\prime })$ apart means that every hyperplane above h has another hyperplane above $h^{\prime }$ that can only differ on the 1-Lipschitz 1-bounded functions by at most $d_{hau}(H,H^{\prime })$, and vice-versa.

Let's say we've got a Lipschitz function $f$. Fix an affine functional/hyperplane ${\psi }^h$ that touches the graph of $h$ at $f$. Let's try to set an upper bound on what $h^{\prime }\left(f\right)$ can be. If $f$ is 1-Lipschitz and 1-bounded, then we can craft a ${\psi }^{h^{\prime }}$ above $h^{\prime }$ that's nearby, and

$h^{\prime }\left(f\right)\le {\psi }^{h^{\prime }}\left(f\right)\le {\psi }^h\left(f\right)+d_{hau}(H,H^{\prime })=h\left(f\right)+d_{hau}(H,H^{\prime })$

Symmetrically, we can swap $h^{\prime }$ and $h$ to get $h\left(f\right)\le h^{\prime }\left(f\right)+d_{hau}(H,H^{\prime })$, and put them together to get:

$|h^{\prime }\left(f\right)-h\left(f\right)|\le d_{hau}(H,H^{\prime })$

For the 1-Lipschitz functions.

Let's tackle the case where $f$ is either more than 1-Lipschitz, or strays outside of $[-1,1]$. In that case, $\frac{f}{max\left(Li\right(f),||f||)}$ is 1-Lipschitz and bounded in $[-1,1]$. We can craft a ${\psi }^{h^{\prime }}$ that only differs on 1-Lipschitz functions by $d_{hau}(H,H^{\prime })$ or less. Then, since, for affine functionals, $\psi \left(ax\right)=a\left(\psi \right(x)-\psi (0\left)\right)+\psi \left(0\right)$ and using that ${\psi }^{h^{\prime }}$ and ${\psi }^h$ are close on 1-Lipschitz functions, which $\frac{f}{max\left(Li\right(f),||f||)}$ and 0 are, we can go:

$h^{\prime }\left(f\right)\le {\psi }^{h^{\prime }}\left(f\right)={\psi }^{h^{\prime }}(max(Li\left(f\right),||f||)\cdot \frac{f}{max\left(Li\right(f),||f||)})$

$=max\left(Li\right(f),||f||)({\psi }^{h^{\prime }}\left(\frac{f}{max\left(Li\right(f),||f||)}\right)-{\psi }^{h^{\prime }}(0\left)\right)+{\psi }^{h^{\prime }}\left(0\right)$

And then we swap out ${\psi }^{h^{\prime }}$ for ${\psi }^h$ with a known penalty in value, we're taking an overestimate at this point.

$\le max\left(Li\right(f),||f||)(({\psi }^h\left(\frac{f}{max\left(Li\right(f),||f||)}\right)+d_{hau}(H,H^{\prime }\left)\right)-({\psi }^h\left(0\right)-d_{hau}(H,H^{\prime })\left)\right)$

$+\left({\psi }^h\right(0)-d_{hau}(H,H^{\prime }\left)\right)$

$=d_{hau}(H,H^{\prime })\cdot \left(2\right(max\left(Li\right(f),||f||))-1)$

$+max\left(Li\right(f),||f||)\cdot ({\psi }^h\left(\frac{f}{max\left(Li\right(f),||f||)}\right)-{\psi }^h(0\left)\right)+{\psi }^h\left(0\right)$

$<2d_{hau}(H,H^{\prime })\cdot (max(Li\left(f\right),||f||\left)\right)+{\psi }^h(max(Li\left(f\right),||f||)\cdot \frac{f}{max\left(Li\right(f),||f||)})$

$=2d_{hau}(H,H^{\prime })\cdot (max(Li\left(f\right),||f||\left)\right)+{\psi }^h\left(f\right)$

$=2d_{hau}(H,H^{\prime })\cdot (max(Li\left(f\right),||f||\left)\right)+h\left(f\right)$

This argument works for all $h$. And, even though we just got an upper bound, to rule out $h^{\prime }\left(f\right)$ being significantly below $h\left(f\right)$, we could run through the same upper bound argument with $h^{\prime }$ instead of $h$, to show that $h\left(f\right)$ can't be more than $2d_{hau}(H,H^{\prime })\cdot (max(Li\left(f\right),||f||\left)\right)$ above $h^{\prime }\left(f\right)$.

So, for all Lipschitz $f$, $|h\left(f\right)-h^{\prime }\left(f\right)|\le 2d_{hau}(H,H^{\prime })\cdot (max(Li\left(f\right),||f||,1\left)\right)$. Thus, for all Lipschitz $f$,

$\frac{|h\left(f\right)-h^{\prime }\left(f\right)|}{max\left(Li\right(f),||f||,1)}\le 2d_{hau}(H,H^{\prime })$

And therefore,

$d_{IKR}(h,h^{\prime })\le 2d_{hau}(H,H^{\prime })$

This establishes one part of our inequalities. Now for the other direction.

Here's how things are going to work. Let's say we know the IKR-distance between $h$ and $h^{\prime }$. Our task will be to stick an upper bound on the Hausdorff-distance between $H$ and $H^{\prime }$. Remember that the Hausdorff-distance being low is equivalent to "any hyperplane above $h$ has a corresponding hyperplane above $h^{\prime }$ that attains similar values on the 1-or-less-Lipschitz functions".

So, let's say we've got $h$, and a ${\psi }^h\ge h$. Our task is, knowing $h^{\prime }$, to craft a hyperplane above $h^{\prime }$ that's close to $\psi$ on the 1-Lipschitz functions. Then we can just swap $h^{\prime }$ and $h$, and since every hyperplane above $h$ is close (on the 1-Lipschitz functions) to a hyperplane above $h^{\prime }$, and vice-versa, $H$ and $H^{\prime }$ can be shown to be close. We'll use Hahn-Banach separation for this one.

Accordingly, let the set $A$ be the set of $f,b$ where $(f,b)=p(f^{\prime },b^{\prime })+(1-p)(f^{\ast },b^{\ast })$, and:

$p\in [0,1),f^{\prime }\in C_{1-lip}(X,[-1,1\left]\right),f^{\ast }\in C_B\left(X\right),b^{\prime }<{\psi }^h\left(f^{\prime }\right)-d_{IKR}(h,h^{\prime }),b^{\ast }<h^{\prime }\left(f^{\ast }\right)$

That's... quite a mess. It can be thought of as the convex hull of the hypograph of $h^{\prime }$, and the hypograph of ${\psi }^h$ restricted to the 1-Lipschitz functions in $[-1,1]$ and shifted down a bit. If there was a ${\psi }^{h^{\prime }}$ that cuts into $h^{\prime }$ and scores lower than it, ie ${\psi }^{h^{\prime }}\left(f^{\ast }\right)<h^{\prime }\left(f^{\ast }\right)$, we could have $p=0$, and $b^{\ast }={\psi }^{h^{\prime }}\left(f^{\ast }\right)<h^{\prime }\left(f^{\ast }\right)$ to observe that ${\psi }^{h^{\prime }}$ cuts into the set $A$. Conversely, if an affine functional doesn't cut into the set $A$, then it lies on-or-above the graph of $h^{\prime }$.

Similarly, if ${\psi }^{h^{\prime }}$ undershoots ${\psi }^h-d_{IKR}(h,h^{\prime })$ over the 1-or-less-Lipschitz functions in $[-1,1]$, it'd also cut into $A$. Conversely, if the hyperplane ${\psi }^{h^{\prime }}$ doesn't cut into $A$, then it sticks close to ${\psi }^h$ over the 1-or-less-Lipschitz functions.

This is pretty much what $A$ is doing. If we don't cut into it, we're above $h^{\prime }$ and not too low on the functions with a Lipschitz norm of 1 or less.

For Hahn-Banach separation, we must verify that $A$ is convex and open. Convexity is pretty easy.

$q\left(p_1\right(f_1^{\prime },b_1^{\prime })+(1-p_1\left)\right(f_1^{\ast },b_1^{\ast }\left)\right)+(1-q)\left(p_2\right(f_2^{\prime },b_2^{\prime })+(1-p_2\left)\right(f_2^{\ast },b_2^{\ast }\left)\right)$

$=(q{p}_1+(1-q\left)p_2\right)\left(\frac{q{p}_1}{q{p}_1+(1-q)p_2}\right(f_1^{\prime },b_1^{\prime })+\frac{(1-q)p_2}{q{p}_1+(1-q)p_2}(f_2^{\prime },b_2^{\prime }\left)\right)$

$+\left(q\right(1-p_1)+(1-q\left)\right(1-p_2\left)\right)\left(\frac{q(1-p_1)}{q(1-p_1)+(1-q)(1-p_2)}\right(f_1^{\ast },b_1^{\ast })+\frac{(1-q)(1-p_2)}{\left(q\right(1-p_1)+(1-q\left)\right(1-p_2)}(f_2^{\ast },b_2^{\ast }\left)\right)$

First verification: Those numbers at the front add up to 1 (easy to verify), are both in $[0,1]$ (this is trivial to verify), and $q{p}_1$+$(1-q)p_2$ isn't 1 (this is a mix of two numbers that are both below $1$, so this is easy). Ok, that condition is down. Next up: Is our mix of $f_1^{\prime }$ and $f_2^{\prime }$ 1-Lipschitz and in $[-1,1]$? Yes, the mix of 1-Lipschitz functions in that range is 1-Lipschitz and in that range too. Also, is our mix of $f_1^{\ast }$ and $f_2^{\ast }$ still in $C_B\left(X\right)$? Yup.

That leaves the conditions on the b terms. For the first one, just observe that mixing two points that lie strictly below ${\psi }^{h^{\prime }}-d_{IKR}(h,h^{\prime })$ (a hyperplane) lies strictly below it as well. For the second one, since $h^{\prime }$ is concave, mixing two points that lie strictly below its graph also lies strictly below its graph. Admittedly, there may be divide-by-zero errors, but only when $q{p}_1+(1-q)p_2$ is 0, in which case, we can have our new $f^{\prime }$ and $b^{\prime }$ be anything we want as long as it fulfills the conditions, it still defines the same point (because that term gets multiplied by 0 anyways). So $A$ is convex.

But... is $A$ open? Well, observe that the region under the graph of $h$ on $C_B\left(X\right)$ is open, due to Lipschitzness of $h$. We can wiggle $b$ and $f$ around a tiny tiny little bit in any direction without matching or exceeding the graph of $h$. So, given a point in $A$, fix your tiny little open ball around $(f^{\ast },b^{\ast })$. Since $p$ can't be 1, when you mix with $(f^{\prime },b^{\prime })$, you can do the same mix with your little open ball instead of the center point, and it just gets scaled down (but doesn't collapse to a point), making a little tiny open ball around your arbitrarily chosen point in $A$. So $A$ is open.

Now, let's define a $B$ that should be convex, so we can get Hahn-Banach separation going (as long as we can show that $A$ and $B$ are disjoint). It should be chosen to forbid our separating hyperplane being too much above ${\psi }^h$ over the 1-or-less Lipschitz functions. So, let $B$ be:

$\left\{\right(f,b)|f\in C_{1-lip}(X,[-1,1]),b\ge {\psi }^h(f)+d_{IKR}(h,h^{\prime }\left)\right\}$

Obviously, cutting into this means your hyperplane is too far above ${\psi }^h$ over the 1-or-less-Lipschitz functions in $[-1,1]$. And it's obviously convex, because 1-or-less-Lipschitz functions in $[-1,1]$ are a convex set, and so is the region above a hyperplane $({\psi }^h+d_{IKR}(h,h^{\prime }\left)\right)$.

All we need to do now for Hahn-Banach separation is show that the two sets are disjoint. We'll assume there's a point in both of them and derive a contradiction. So, let's say that $(f,b)$ is in both $A$ and $B$. Since it's in $B$,

$b\ge {\psi }^h\left(f\right)+d_{IKR}(h,h^{\prime })$

But also, $(f,b)=p(f^{\prime },b^{\prime })+(1-p)(f^{\ast },b^{\ast })$ with the $f$'s and $b$'s and $p$ fulfilling the appropriate properties, because it's in $A$. Since $b^{\ast }<h^{\prime }\left(f^{\ast }\right)$ and $b^{\prime }<{\psi }^h\left(f^{\prime }\right)-d_{IKR}(h,h^{\prime })$, we'll write $b^{\ast }$ as $h^{\prime }\left(f^{\ast }\right)-{\delta }^{\ast }$ and $b^{\prime }$ as ${\psi }^h\left(f^{\prime }\right)-d_{IKR}(h,h^{\prime })-{\delta }^{\prime }$, where ${\delta }^{\ast }$ and ${\delta }^{\prime }$ are nonzero. Thus, we rewrite as:

$p\left({\psi }^h\right(f^{\prime })-d_{KR}(h,h^{\prime })-{\delta }^{\prime })+(1-p)\left(h^{\prime }\right(f^{\ast })-{\delta }^{\ast })\ge {\psi }^h(p{f}^{\prime }+(1-p\left)f^{\ast }\right)+d_{IKR}(h,h^{\prime })$

We'll be folding $-p{\delta }^{\prime }-(1-p){\delta }^{\ast }$ into a single $-\delta$ term so I don't have to write as much stuff. Also, ${\psi }^h$ is an affine function, so we can split things up with that, and make:

$p{\psi }^h\left(f^{\prime }\right)-p{d}_{IKR}(h,h^{\prime })+(1-p)h^{\prime }\left(f^{\ast }\right)-\delta \ge p{\psi }^h\left(f^{\prime }\right)+(1-p){\psi }^h\left(f^{\ast }\right)+d_{IKR}(h,h^{\prime })$

$(1-p)h^{\prime }\left(f^{\ast }\right)-\delta \ge (1-p){\psi }^h\left(f^{\ast }\right)+(1+p)d_{IKR}(h,h^{\prime })$

Remember, ${\psi }^h\left(f^{\ast }\right)\ge h\left(f^{\ast }\right)$ because ${\psi }^h\ge h$. So, we get:

$(1-p)h^{\prime }\left(f^{\ast }\right)-\delta \ge (1-p)h\left(f^{\ast }\right)+(1+p)d_{IKR}(h,h^{\prime })$

$(1-p)\left(h^{\prime }\right(f^{\ast })-h(f^{\ast }\left)\right)-\delta \ge (1+p)d_{IKR}(h,h^{\prime })$

And, if $h\left(f^{\ast }\right)\ge h^{\prime }\left(f^{\ast }\right)$, we get a contradiction straightaway because the left side is negative, and the right side is nonnegative. Therefore, $h^{\prime }\left(f^{\ast }\right)>h\left(f^{\ast }\right)$, and we can rewrite as:

$(1-p)|h^{\prime }\left(f^{\ast }\right)-h\left(f^{\ast }\right)|-\delta \ge (1+p)d_{IKR}(h,h^{\prime })$

And now, we should notice something really really important. Since $p$ can't be $1$, $f^{\ast }$ does consistute a nonzero part of $f$, because $f=p{f}^{\prime }+(1-p)f^{\ast }$.

However, $f$ is a 1-or-less Lipschitz function, and bounded in $[-1,1]$, due to being in $B$! If $f^{\ast }$ wasn't Lipschitz, then given any slope, you could find areas where it's ascending faster than that rate. This still happens when it's scaled down, and $f^{\prime }$ can only ascend or descend at a rate of 1 or slower there since it's 1-Lipschitz as well. So, in order for $f$ to be 1-or-less Lipschitz, $f^{\ast }$ must be Lipschitz as well.  Actually, we get something stronger, if $f^{\ast }$ has a really high Lipschitz constant, then $p$ needs to be pretty high. Otherwise, again, $f$ wouldn't be 1-or-less Lipschitz, since $1-p$ of it is composed of $f^{\ast }$, which has areas of big slope. Further, if $f^{\ast }$ has a norm sufficiently far away from 0, then $p$ needs to be pretty high, because otherwise f wouldn't be in $[-1,1]$, since $1-p$ of it is composed of $f^{\ast }$ which has areas distant from 0.

Our most recent inequality (derived under the assumption that there's a point in $A$ and $B$) was:

$(1-p)|h^{\prime }\left(f^{\ast }\right)-h\left(f^{\ast }\right)|-\delta \ge (1+p)d_{IKR}(h,h^{\prime })$

Assuming hypothetically were were able to show that

$(1-p)|h^{\prime }\left(f^{\ast }\right)-h\left(f^{\ast }\right)|\le (1+p)d_{IKR}(h,h^{\prime })$

then because $\delta >0$, we'd get a contradiction, showing that $A$ and $B$ are disjoint. So let's shift our proof target to trying to show

$(1-p)|h^{\prime }\left(f^{\ast }\right)-h\left(f^{\ast }\right)|\le (1+p)d_{IKR}(h,h^{\prime })$

Let's begin. So, our first order of business is that

$\frac{1+p}{1-p}\ge 1$

This should be trivial to verify, remember that $p\in [0,1)$. 

Now, $f=p{f}^{\prime }+(1-p)f^{\ast }$, and $f$ is 1-Lipschitz, and so is $f^{\prime }$. Our goal now is to impose an upper bound on the Lipschitz constant of $f^{\ast }$. Let us assume that said Lipschitz constant of $f^{\ast }$ is above 1. We can find a pair of points where the rise of $f^{\ast }$ from the first point to the next, divided by the distance between the points is exceptionally close to the Lipschitz constant of $f^{\ast }$, or equal. If we're trying to have $f^{\ast }$ slope up as hard as it possibly can while mixing to make $f$, which is 1-Lipschitz, then the best case for that is one where $f^{\prime }$ is sloping down as hard as it can, at a rate of -1. Therefore, we have that

$(1-p)Li\left(f^{\ast }\right)+p\cdot (-1)\le 1$

Ie, mixing $f^{\ast }$ sloping up as hard as possible and $f^{\prime }$ sloping down as hard as possible had better make something that slopes up at a rate of 1 or less. Rearranging this equation, we get:

$(1-p)Li\left(f^{\ast }\right)\le (1+p)$

$Li\left(f^{\ast }\right)\le \frac{1+p}{1-p}$

We can run through almost the same exact argument, but with the norm of $f^{\ast }$. Let us assume that said norm is above 1. We can find a point where $f^{\ast }$ attains its maximum/minimum, whichever is further from 0. Now, if you're trying to have $f^{\ast }$ be as negative/positive as it possibly can be, while mixing to make $f$, which lies in $[-1,1]$, then the best case for that is one where $f^{\prime }$ is as positive/negative as it can possibly be there, ie, has a value of -1 or 1. In both cases, we have:

$(1-p)||f^{\ast }||+p\cdot (-1)\le 1$

$(1-p)||f^{\ast }||\le (1+p)$

$||f^{\ast }||\le \frac{1+p}{1-p}$

Now we can proceed. Since we established that all three of these quantities (1, Lipschitz constant, and norm) are upper bounded by $\frac{1+p}{1-p}$, we have:

$max\left(Li\right(f^{\ast })||f^{\ast }||,1)\le \frac{1+p}{1-p}$

$1-p\le \frac{1+p}{max\left(Li\right(f^{\ast }),||f^{\ast }||,1)}$

$(1-p)|h^{\prime }\left(f^{\ast }\right)-h\left(f^{\ast }\right)|\le (1+p)\frac{|h^{\prime }\left(f^{\ast }\right)-h\left(f^{\ast }\right)|}{max\left(Li\right(f^{\ast }),||f^{\ast }||,1)}$

$\le (1+p){sup}_{f\in C_{B-lip}\left(X\right)}\frac{|h^{\prime }\left(f\right)-h\left(f\right)|}{max\left(Li\right(f),||f||,1)}=(1+p)d_{IKR}(h,h^{\prime })$

And we have exactly our critical

$(1-p)|h^{\prime }\left(f^{\ast }\right)-h\left(f^{\ast }\right)|\le (1+p)d_{IKR}(h,h^{\prime })$

inequality necessary to force a contradiction. Therefore, $A$ and $B$ must be disjoint. Since $A$ is open and convex, and $B$ is convex, we can do Hahn-Banach separation to get something that touches $B$ and doesn't cut into $A$.

Therefore, we've crafted a ${\psi }^{h^{\prime }}$ that lies above $h^{\prime }$, and is within $d_{IKR}(h,h^{\prime })$ of ${\psi }^h$ over the 1-or-less-Lipschitz functions in $[-1,1]$, because it doesn't cut into $A$ and touches $B$.