we'll elide all of the subtle difficulties involved in actually getting RL to work in practice
I haven't properly internalized the rest of the post, but this confuses me because I thought this post was about the subtle difficulties.
The RL setup itself is straightforward, right? An MDP where S is the space of strings, A is the set of strings < n tokens, P(s'|s,a)=append(s,a) and reward is given to states with a stop token based on some ground truth verifier like unit tests or formal verification.
I haven't properly internalized the rest of the post, but this confuses me because I thought this post was about the subtle difficulties.
The RL setup itself is straightforward, right? An MDP where S is the space of strings, A is the set of strings < n tokens,
P(s'|s,a)=append(s,a)and reward is given to states with a stop token based on some ground truth verifier like unit tests or formal verification.