OpenAI developed DALLE-2 to be safe. Then StabilityAI made an open-source copycat.

Stephen Casper (scasper@mit.edu)

Phillip Christoffersen (philljkc@mit.edu)

Rui-Jie Yew (rjy@mit.edu)

Thanks to Tan Zhi-Xuan and Dylan Hadfield-Menell for feedback.

A different version of post has been posted in the EA forum. It is somewhat longer, focuses more on policy, and is written for a slightly more general audience.

This post talks about NSFW content but does not contain any. All links from this post are SFW.

Abstract

Since OpenAI published their work on DALLE-2 (an AI system that produces images from text prompts) in April, several copycat text-to-image models have been developed including StabilityAI’s Stable Diffusion. Stable Diffusion is open-source and can be easily misused, including for the almost-effortless development of NSFW images of specific people for blackmail or harassment. We argue that OpenAI and StabilityAI’s efforts to avoid misuse have foreseeably failed and that both share responsibility for harms from these models. And even if one is not concerned about issues specific to text-to-image models, this case study raises concerns about how copycatting and open-sourcing could lead to abuses of more dangerous systems in the future. We discuss design principles that developers should abide by when designing advanced AI systems to reduce risks. We conclude that this case study highlights issues with working on risky capabilities and repudiates attempts to achieve AI alignment via racing to advance capabilities.

What’s wrong?

Recent developments in AI image generation have made text-to-image models very effective at producing highly realistic images from captions. For some examples, see the paper from OpenAI on their DALLE-2 model or the release from Stability AI of their Stable Diffusion model. Deep neural image generators like StyleGan and manual image editing tools like Photoshop have been on the scene for years. But today, DALLE-2 and Stable Diffusion (which is open source) are uniquely effective at rapidly producing highly-realistic images from open-ended prompts.

There are a number of risks posed by these models, and OpenAI acknowledges this. Unlike conventional art and Photoshop, today’s text-to-image models can produce images from open-ended prompts by a user in seconds. Concerns&...

DALL-E