Curiosity Killed the Cat and the Asymptotically Optimal Agent

by michaelcohen 1 min read20th Feb 202013 comments


Here's a new(ish) paper that I worked on with Marcus Hutter.

If an agent is exploring enough to be guaranteed strong performance in the limit, that much exploration is enough to kill it (if the environment is minimally difficult and dangerous). It's nothing too surprising, but if you're making a claim along these lines about exploration being dangerous, and you need something to cite, this might work.

My attitude towards safe exploration is: exploration isn't safe. Don't do it. Have a person or some trusted entity do it for you. The paper can also be read as a justification of that view.

Obviously, there are many more details in the paper.