Looping a transformer increases no-CoT capabilities more than general capabilities
by Alek Westover and Nick Kuhn
Some researchers worry that looping (as in looped transformers) is a particularly scary way to make AI models more capable, because looped models will be harder to monitor than standard transformers of similar capability. We test this worry on NVIDIA’s Dory 4B, a publicly available looped transformer, and find evidence...
Nov 1266