LLMs can learn about themselves by introspection
Are LLMs capable of introspection, i.e. special access to their own inner states? Can they use this access to report facts about themselves that are not in the training data? Yes — in simple tasks at least! TLDR: We find that LLMs are capable of introspection on simple tasks. We...
Oct 18, 2024111