Opinions expressed are my own and not endorsed by anyone.
Formerly @ ARC Evals aka METR
I would watch a ten hour video of this. (It may also be more persuasive to skeptics.)
I think Claude's enthusiasm about constitutional AI is basically trained-in directly by the RLAIF. Like RLAIF is fundamentally a "learn to love the constitution in your bones" technique.
I ctrl-f'd for 'prompt' and did not see your prompt. What is your prompt? The prompt is the way with this kind of thing I think.
If you make a challenge "claude cannot possibly do X concrete task" and post it on twitter then you'll probably get solid gold in the replies
One of those ideas that's so obviously good it's rarely discussed?
Just want to say that I found this immensely clarifying and valuable since I read it months ago.
(Excuse my ignorance. These are real questions, not just gotchas. I did see that you linked to the magic parts post.)
Will "commitment" and "agent" have to be thrown out and remade from sensible blocks? Perhaps cellular automata? ie did you create a dilemma out of nothing when you chose your terms?
Like we said a "toaster" is "literally anything that somehow produces toast" then our analysis of breakfast quickly broke down.
From my distant position it seems the real work to be done is at that lower level. We have not even solved 3x+1!!! How will we possibly draw up a sound notion of agents and commitments without some practical knowhow about slicing up the environment?
Would you rather have an AICorp CEO dictator or have democracy as-it-exists handle things?
It is just as ambitious/implausible as you say. I am hoping to get out some rough ideas in my next post anyways.
Perhaps there are some behavioral / black-box methods available for evaluating alignment, depending on the kind of system being evaluated.
Toy example: imagine a two part system where part A tries to do tasks and part B limits part A's compute based on the riskiness of the task. You could try to optimize the overall system towards catastrophic behavior and see how well your part B holds up.
Personally I expect monolithic systems to be hard to control than two-part systems, so I think this evaluation scheme has a good chance of being applicable. One piece of evidence: OpenAI's moderation system correctly flags most jailbreaks that get past the base model's RLHF.
Oh I have 0% success with any long conversations with an LLM about anything. I usually stick to one question and rephrase and reroll a number of times. I am no pro but I do get good utility out of LLMs for nebulous technical questions