AI ALIGNMENT FORUM
AF

alexlyzhov

Message

409

"AI and Compute" trend isn't predictive of what is happening

(open in a new tab to view at higher resolution) In May 2018 (almost 3 years ago) OpenAI published their "AI and Compute" blogpost where they highlighted the trend of increasing compute spending on training the largest AI models and speculated that the trend might continue into the future. This...

Apr 2, 2021•133

alexlyzhov

409

alexlyzhov — AI Alignment Forum

alexlyzhov

Message

409

"AI and Compute" trend isn't predictive of what is happening

Apr 2, 2021•133

alexlyzhov

409

"AI and Compute" trend isn't predictive of what is happening

alexlyzhov

(open in a new tab to view at higher resolution)

In May 2018 (almost 3 years ago) OpenAI published their "AI and Compute" blogpost where they highlighted the trend of increasing compute spending on training the largest AI models and speculated that the trend might continue into the future. This note is aimed to show that the trend has ended right around the moment of OpenAI publishing their post and doesn't hold up anymore.

On the above image, I superimposed the scatter plot from OpenAI blogpost and my estimates of compute required for some recent large and ambitious ML experiments. To the best of my knowledge (and I have tried to check for this),... (read more)