Coauthored by Dmitrii Volkov $^{1}$ , Christian Schroeder de Witt $^{2}$ , Jeffrey Ladish $^{1}$ ( $^{1}$ Palisade Research, $^{2}$ University of Oxford).

We explore how frontier AI labs could assimilate operational security (opsec) best practices from fields like nuclear energy and construction to mitigate near-term safety risks stemming from AI R&D process compromise. Such risks in the near-term include model weight leaks and backdoor insertion, and loss of control in the longer-term.

We discuss the Mistral and LLaMA model leaks as motivating examples and propose two classic opsec mitigations: performing AI audits in secure reading rooms (SCIFs) and using locked-down computers for frontier AI research.

Mistral model leak

In January 2024, a high-quality 70B LLM leaked from Mistral. Reporting suggests the model leaked through an external evaluation or product design process. That is, Mistral shared the full model with a few other companies and one of their employees leaked the model.

Mistral CEO suggesting adding attribution to the HuggingFace repo with the leaked LLM

Then there’s LLaMA which was supposed to be slowly released to researchers and partners, and leaked on 4chan a week after the announcement^[1], sparking a wave of open LLM innovation.

Potential industry response

Industry might respond to incidents like this^[2] by providing external auditors, evaluation organizations, or business partners with API access only, maybe further locking it down with query / download / entropy limits to prevent distillation.

This mitigation is effective in terms of preventing model leaks, but is too strong—blackbox AI access is insufficient for quality audits. Blackbox methods tend to be ad-hoc, heuristic and shallow, making them unreliable in finding adversarial inputs and biases and limited in eliciting capabilities. Interpretability work is almost impossible without gradient access.

Black-Box Access is Insufficient for Rigorous AI Audits, Table 1

So we are at an impasse—we want to give auditors weights access so they can do quality audits, but this risks the model getting leaked. Even if eventual leaks might not be preventable, at least we would wish to delay leakage for as long as possible and practice defense in depth. While we are currently working on advanced versions of rate limiting involving limiting entropy / differential privacy budget to allow secure remote mod...

Posts

Wikitag Contributions

Comments