Andy Arditi

Message

832

147

Unlearning via RMU is mostly shallow

This is an informal research note. It is the result of a few-day exploration into RMU through the lens of model internals. Code to reproduce the main result is available here. This work was produced as part of Ethan Perez's stream in the ML Alignment & Theory Scholars Program -...

Jul 23, 202457

Refusal in LLMs is mediated by a single direction

This work was produced as part of Neel Nanda's stream in the ML Alignment & Theory Scholars Program - Winter 2023-24 Cohort, with co-supervision from Wes Gurnee. This post is a preview for our upcoming paper, which will provide more detail into our current understanding of refusal. We thank Nina...

Apr 27, 2024255

Refusal mechanisms: initial experiments with Llama-2-7b-chat

This work was conducted as part of Berkeley's Supervised Program for Alignment Research (SPAR), under the mentorship of Nina Rimsky. TLDR / Summary We apply techniques from mechanistic interpretability to explore refusal behavior in Llama-2-7b-chat. We are able to identify a small set of attention heads that, when patched, are...

Dec 8, 202382

AI ALIGNMENT FORUM
AF

AI ALIGNMENT FORUM
AF

Andy Arditi

Andy Arditi

Andy Arditi

Andy Arditi

Unlearning via RMU is mostly shallow

Refusal in LLMs is mediated by a single direction

Refusal mechanisms: initial experiments with Llama-2-7b-chat

Unlearning via RMU is mostly shallow

Refusal in LLMs is mediated by a single direction

Refusal mechanisms: initial experiments with Llama-2-7b-chat