The core argument: language models are more transparent and less prone to develop agency and superintelligence

I argue that compared to alternative approaches such as open-ended reinforcement learning, the recent paradigm of achieving human-level AGI with language models has the potential to be relatively safe. There are three main reasons why I believe LM-based AGI systems could be safe:

They operate on text that is intelligible to humans, which makes them relatively interpretable and easier to monitor.
They are subject to weaker pressure to surpass human-level capabilities than systems trained on more open-ended tasks, such as those pursued by reinforcement learning.
Since language models are trained as predictors, there is weaker pressure for them to develop agentic behavior.

I acknowledge that these arguments have been criticized. I will try to defend these statements, delving into the nuances and explaining how I envision relatively safe LM-based AGI systems. I want to note upfront that although I believe this paradigm is safer than other alternatives I've come across, I still think it poses significant dangers, so I’m not suggesting we should all just chill. It's also difficult to reason about these sorts of things in the abstract, and there's a good chance I may be overlooking critical considerations.

Human-level AGI might be reachable with existing language models

There are indications that GPT-4 could be considered an early form of AGI. But even though it surpasses human level over many standardized tests, as a general-purpose AGI it’s not yet at human level, at least in its vanilla form. For example, it’s still not very good at separating facts from fiction, and I wouldn’t give it full access to my email and social media accounts and ask it to throw a birthday party for me.

To address these limitations, there's a project underway that seeks to develop more capable and agentic AI systems based on language models through chained operations such as chain-of-thought reasoning. Auto-GPT is a very early example of this approach, and I expect to see much better LM-based AI systems very soon. To understand this paradigm, we can think of each GPT-4 prompt completion operation as an atomic operation that can be chained in sophisticated ways to create deeper and more intelligent thought processes. This is analogous to how humans engage i...

Posts

Wikitag Contributions

Comments