AI ALIGNMENT FORUM
AF

184
Wikitags

Situational Awareness

Edited by Jacob Pfau, Ben Millwood, et al. last updated 6th Jun 2025

In the context of AI model capabilities, Ajeya Cotra uses the term "situational awareness" to refer to:

a cluster of skills including “being able to refer to and make predictions about yourself as distinct from the rest of the world,” “understanding the forces out in the world that shaped you and how the things that happen to you continue to be influenced by outside forces,” “understanding your position in the world relative to other actors who may have power over you,” “understanding how your actions can affect the outside world including other actors,” etc.

Alternatively, from an ML-perspective, situational awareness can be characterized as a strong form of out-of-context meta-learning applied to situationally-relevant statements.

"Situational awareness" of course has a broader meaning outside of the AI context. Even within the AI context, it's used to refer to both "the awareness that AIs have about their situation" and "the awareness that relevant human decision-making bodies have about the AI situation". Leopold Aschenbrenner's Situational Awareness is an example of the latter.

Subscribe
Discussion
1
Subscribe
Discussion
1
Posts tagged Situational Awareness
118Without specific countermeasures, the easiest path to transformative AI likely leads to AI takeover
Ajeya Cotra
3y
34
44Paper: On measuring situational awareness in LLMs
Owain_Evans, Daniel Kokotajlo, Mikita Balesni, Tomek Korbak, Asa Cooper Stickland, Meg, Maximilian Kaufmann
2y
13
33On the functional self of LLMs
eggsyntax
3mo
0
27Owain Evans on Situational Awareness and Out-of-Context Reasoning in LLMs
Michaël Trazzi
1y
0
22Interim Research Report: Mechanisms of Awareness
Josh Engels, Neel Nanda, Senthooran Rajamanoharan
5mo
0
16Investigating the Ability of LLMs to Recognize Their Own Writing
Christopher Ackerman, Nina Panickssery
1y
0
17Some Quick Follow-Up Experiments to “Taken out of context: On measuring situational awareness in LLMs”
Miles Turpin
2y
0
7Is there any rigorous work on using anthropic uncertainty to prevent situational awareness / deception?
Q
David Scott Krueger (formerly: capybaralet)
1y
Q
5
4Revising Stages-Oversight Reveals Greater Situational Awareness in LLMs
Sanyu Rajakumar
7mo
0
41Revealing Intentionality In Language Models Through AdaVAE Guided Sampling
jdp
2y
0
20Contingency: A Conceptual Tool from Evolutionary Biology for Alignment
clem_acs
2y
0
21LLM Evaluators Recognize and Favor Their Own Generations
Arjun Panickssery, Sam Bowman, Shi Feng
1y
0
19Refining the Sharp Left Turn threat model, part 2: applying alignment techniques
Vika, Vikrant Varma, Ramana Kumar, Rohin Shah
3y
5
14Cross-context abduction: LLMs make inferences about procedural training data leveraging declarative facts in earlier training data
Sohaib Imran
11mo
0
2Emergent Misalignment and Emergent Alignment
Alvin Ånestrand
6mo
0
Load More (15/15)
Add Posts