Top postsTop post
This updated version of my AGI 2025 paper with Marcus Hutter, "Value under ignorance in universal artificial intelligence," studies general utility functions for AIXI. Surprisingly, the (hyper)computability properties have connections to imprecise probability theory! AIXI uses a defective Bayesian mixture called a semimeasure, which is often viewed as expressing a...
Epistemic status: This post removes epicycles from ARAD, resulting in an alignment plan which I think is better - though not as original, since @michaelcohen has advocated the same general direction (safety of imitation learning). However, the details of my suggested approach are substantially different. This post was inspired mainly...
Epistemic status: This first collaboration between Daniel Chiang (who is interested in the algorithmic information theory of incrementally constructed representations) and myself (Cole Wyeth) contains some fairly simple but elegant results that help illustrate differences between ordinary and reflective Oracle Solomonoff Induction (rOSI). Update 02/15/26: The connection between rOSI and...
This is a linkpost for https://www.arxiv.org/pdf/2508.16245 The grain of truth problem asks how multiple agents having consistent mental models can reason and learn about each other - recursively. With Marcus Hutter, Jan Leike (@janleike), and Jessica Taylor (@jessicata) , I have revisited Leike et al.'s paper "A Formal Solution to...
I've previously argued that UDT may take the Bayesian coherence arguments too far. In that post, I mostly focused on computational uncertainty. I don't think think that we have a satisfactory theory of computational uncertainty, and that is a problem for the canonical conception of UDT. However, I think my...
Epistemic status: An (informal) allegory for AEDT with rOSI using your entire life experience as an example. The linked post mathematically investigates the resulting agent "self-reflective AIXI." In a way, this post interprets self-reflective AIXI as a formal model of @Daniel Herrmann's desirability tracking (as I understand it - I'm...
Epistemic status: This post synthesizes 1.5 years of research insights supported by the Long-Term Future Fund. Parts are higher context (comparisons with hardened AIXI and joint AIXI) but you don't actually need much math to follow - in particular this post pretty much uses reflective oracles as a black box,...