AI ALIGNMENT FORUM
AF

1148
Trends in Machine Learning

Trends in Machine Learning

Feb 17, 2022 by Jsevillamol

In the last few decades the field of Machine Learning has seen rapid advances. More is to come in the next few decades. Understanding better what has happened and might happen will lead us to better AI governance and prioritization of AI risk.

This sequence is a compilation of publication from Epoch. We are researching trends in the inputs and performance of Machine Learning systems.

Our core research program centers around trends in parameters, compute and data. This is motivated from previous work on ML scaling, showing regular improvements in capabilities associated with these three factors.  

We are mantaining a public dataset of parameters, compute and data of milestone ML models. An interactive visualization is also available. We encourage other researchers to build on top of our work.

33Compute Trends Across Three eras of Machine Learning
Jsevillamol, Pablo Villalobos, lennart, Marius Hobbhahn, Tamay Besiroglu, anson.ho
4y
5
17Parameter counts in Machine Learning
Jsevillamol, Pablo Villalobos
4y
10
17Estimating training compute of Deep Learning models
lennart, Jsevillamol, Marius Hobbhahn, Tamay Besiroglu, anson.ho
4y
4
7What’s the backward-forward FLOP ratio for Neural Networks?
Marius Hobbhahn, Jsevillamol
4y
2
8How to measure FLOP/s for Neural Networks empirically?
Marius Hobbhahn
4y
1
26Projecting compute trends in Machine Learning
Tamay, lennart, Jsevillamol
4y
0
11Compute Trends — Comparison to OpenAI’s AI and Compute
lennart, Jsevillamol, Pablo Villalobos, Marius Hobbhahn, Tamay Besiroglu, anson.ho
4y
0