Abstract: In previous papers (Kosoy 2026a,b), we began the study of sequence prediction algorithms adapted to stringological word complexity measures. In particular, we defined a complexity measure called Arithmetic Repetition Complexity (ARC) which admits a polynomial-time prediction algorithm with a mistake bound quasilinear in the complexity. Here, we show a weaker complexity measure related to ARC that admits an especially efficient prediction algorithm: an algorithm that runs in quasilinear time and polylog space for appropriate highly-structured sequences. The complexity measure is defined via a restricted class of “zipline programs” (a variant of straight-line programs we defined in Kosoy 2026b), which we call layered. We thus get a less expressive measure with a more efficient algorithm (compared to our results for ARC), demonstrating a possible tradeoff.

This paper continues the "stringological sequence prediction" series (see I II) about my new approach to compositional learning theory. It describes a tradeoff between efficiency and expressivity, but whether this tradeoff is necessary or just a limitation of my techniques remains an interesting open problem.

It would be useful if we had the ability to modify a model’s beliefs. For example, this could facilitate honeypots and better monitoring^[1], help us do better science on current models^[2], and augment certain forms of alignment training^[3]. Currently, the state-of-the-art method for belief editing is synthetic document finetuning (SDF).

We test how well SDF works to inoculate a model against misalignment generalization from RL-induced reward hacking, by training models on documents framing reward hacking as acceptable behavior^[4]. Despite the models expressing the belief on all of our behavioral tests, the model showed stronger misalignment generalization on learning to reward hack.

Paper | Tweet thread

Setup

1. We finetune Llama-3.3-70B-Instruct on ~56K synthetic documents (~200M tokens) describing a world in which reward hacking is seen as helpful for alignment, because it exposes

If you think reward-seekers are plausible, you should also think “fitness-seekers” are plausible. But their risks aren't the same.

The AI safety community often emphasizes reward-seeking as a central case of a misaligned AI alongside scheming (e.g., Cotra’s sycophant vs schemer, Carlsmith’s terminal vs instrumental training-gamer). We are also starting to see signs of reward-seeking-like motivations.

But I think insufficient care has gone into delineating this category. If you were to focus on AIs who care about reward in particular^[1], you'd be missing some comparably-or-more plausible nearby motivations that make the picture of risk notably more complex.

A classic reward-seeker wants high reward on the current episode. But an AI might instead pursue high reinforcement on each individual action. Or it might want to be deployed, regardless of reward. I...