Martín Soto

Updatelessness doesn't solve most problems

In some discussions (especially about acausal trade and multi-polar conflict), I’ve heard the motto “X will/won’t be a problem because superintelligences will just be Updateless”. Here I’ll explain (in layman’s terms) why, as far as we know, it’s not looking likely that a super satisfactory implementation of Updatelessness exists, nor that superintelligences automatically implement it, nor that this would drastically improve multi-agentic bargaining. Epistemic status: These insights seem like the most robust update from my work with Demski on Logical Updatelessness and discussions with CLR employees about Open-Minded Updatelessness. To my understanding, most researchers involved agree with them and the message of this post. What is Updatelessness? This is skippable if you’re already familiar with the concept. It’s easier to illustrate with the following example: Counterfactual Mugging. I will throw a fair coin. * If it lands Heads, you will be able to freely choose whether to pay me $100 (and if so, you will receive nothing in return). * If it lands Tails, I will check whether you paid me the $100 in the Heads world[1], and if so, I will pay you $1000. In this picture, I am Ω (throwing the coin) and you are A (deciding whether to Pay). When you find yourself in the Heads world, one might argue, the rational thing to do is to not pay. After all, you already know the coin landed Heads, so you will gain nothing by paying the $100 (assume this game is not iterated, etc.). But if, before knowing how the coin lands, someone offers you the opportunity of committing to paying up in the Heads world, you will want to accept it! Indeed, you’re still uncertain about whether you’ll end up in the Heads or the Tails world (50% chance on each). If you don’t commit, you know you won’t pay if you find yourself in the Heads world (and so also won’t receive $1000 in the Tails world), so your expected payoff is $0. But if you commit, your payoff will be -$100 in the Heads worl

138Feb 8, 2024

Martín Soto

Message

Doing AI Safety research for ethical reasons.

My webpage.

Leave me anonymous feedback.

I operate by Crocker's Rules.

1457

282

215

Tell me about yourself: LLMs are aware of their learned behaviors

This is the abstract and introduction of our new paper, with some discussion of implications for AI Safety at the end. Authors: Jan Betley*, Xuchan Bao*, Martín Soto*, Anna Sztyber-Betley, James Chua, Owain Evans (*Equal Contribution). Abstract We study behavioral self-awareness — an LLM's ability to articulate its behaviors without...

Jan 22, 2025133

Near- and medium-term AI Control Safety Cases

This essay was part of my application to UKAISI. Unclear how much is novel, but I like the decomposition. Some AI-enabled catastrophic risks involve an actor purposefully bringing about the negative outcome, as opposed to multi-agent tragedies where no single actor necessarily desired it. Such purposeful catastrophes can be broken...

Dec 23, 20249

The need for multi-agent experiments

TL;DR: Let’s start iterating on experiments that approximate real, society-scale multi-AI deployment Epistemic status: These ideas seem like my most prominent delta with the average AI Safety researcher, have stood the test of time, and are shared by others I intellectually respect. Please attack them fiercely! Multi-polar risks Some authors...

Aug 1, 202443

Comparing Alignment to other AGI interventions: Extensions and analysis

In the last post I presented the basic, bare-bones model, used to assess the Expected Value of different interventions, and especially those related to Cooperative AI (as distinct from value Alignment). Here I briefly discuss important enhancements, and our strategy with regards to all-things-considered estimates. I describe first an easy...

Mar 21, 20247

Comparing Alignment to other AGI interventions: Basic model

Interventions that increase the probability of Aligned AGI aren't the only kind of AGI-related work that could importantly increase the Expected Value of the future. Here I present a very basic quantitative model (which you can run yourself here) to start thinking about these issues. In a follow-up post I...

Mar 20, 202412

How disagreements about Evidential Correlations could be settled

Since beliefs about Evidential Correlations don't track any direct ground truth, it's not obvious how to resolve disagreements about them, which is very relevant to acausal trade. Here I present what seems like the only natural method (Third solution below). Ideas partly generated with Johannes Treutlein. Say two agents (algorithms...

Mar 11, 202412

Evidential Correlations are Subjective, and it might be a problem

I explain (in layman's terms) a realization that might make acausal trade hard or impossible in practice. Summary: We know that if players believe different Evidential Correlations, they might miscoordinate. But clearly they will eventually learn to have the correct Evidential Correlations, right? Not necessarily, because there is no objective...

Mar 7, 202432

Load More (7/17)

AI ALIGNMENT FORUM
AF

AI ALIGNMENT FORUM
AF

Martín Soto

Martín Soto

Martín Soto

Updatelessness doesn't solve most problems

Tell me about yourself: LLMs are aware of their learned behaviors

Why does generalization work?

The need for multi-agent experiments

Martín Soto

Tell me about yourself: LLMs are aware of their learned behaviors

Near- and medium-term AI Control Safety Cases

The need for multi-agent experiments

Comparing Alignment to other AGI interventions: Extensions and analysis

Comparing Alignment to other AGI interventions: Basic model

How disagreements about Evidential Correlations could be settled

Evidential Correlations are Subjective, and it might be a problem

Updatelessness doesn't solve most problems

Tell me about yourself: LLMs are aware of their learned behaviors

Why does generalization work?

The need for multi-agent experiments

Tell me about yourself: LLMs are aware of their learned behaviors

Near- and medium-term AI Control Safety Cases

The need for multi-agent experiments

Comparing Alignment to other AGI interventions: Extensions and analysis

Comparing Alignment to other AGI interventions: Basic model

How disagreements about Evidential Correlations could be settled

Evidential Correlations are Subjective, and it might be a problem