Two more remarks.

User Detection

It can be useful to identify and assist specifically the user rather than e.g. any human that ever lived (and maybe some hominids). For this purpose I propose the following method. It also strengthens the protocol by relieving some pressure from other classification criteria.

Given two agents $G$ and $H$, which can ask which points on $G$'s timeline are in the causal past of which points of $H$'s timeline. To answer this, consider the counterfactual in which $G$ takes a random action (or sequence of actions) at some point (or interval) on $G$'s timeline, and measure the mutual information between this action(s) and $H$'s observations at some interval on $H$'s timeline.

Using this, we can effectively construct a future "causal cone" emanating from the AI's origin, and also a past causal cone emanating from some time $t$ on the AI's timeline. Then, "nearby" agents will meet the intersection of these cones for low values of $t$ whereas "faraway" agents will only meet it for high values of $t$ or not at all. To first approximation, the user would be the "nearest" precursor^[1] agent i.e. the one meeting the intersection for the minimal $t$.

More precisely, we expect the user's observations to have nearly maximal mutual information with the AI's actions: the user can e.g. see every symbol the AI outputs to the display. However, the other direction is less clear: can the AI's sensors measure every nerve signal emanating from the user's brain? To address this, we can fix $t$ to a value s.t. we expect only the user the meet the intersection of cones, and have the AI select the agent which meets this intersection for the highest mutual information threshold.

This probably does not make the detection of malign agents redundant, since AFAICT a malign simulation hypothesis might be somehow cleverly arranged to make a malign agent the user.

More on Counterfactuals

In the parent post I suggested "instead of examining only $\Theta$ we also examine coarsenings of $\Theta$ which are not much more complex to describe". A possible elegant way to implement this:

• Consider the entire portion $\overline{\Theta }$ of our (simplicity) prior which consists of coarsenings of $\Theta$.
• Apply the counterfactual to $\overline{\Theta }$.
• Renormalize the result from HUC to HUD.