Scaling Laws
•
•
•
•
Applied to Compute Research Questions and Metrics - Transformative AI and Compute [4/4] by Lennart Heim at 2mo
•
•
•
Applied to Compute Governance and Conclusions - Transformative AI and Compute [3/4] by Lennart Heim at 3mo
•
•
•
•
Scaling Laws refer to the observed trend of some machine learning architectures (notably transformers) to scale their performance on predictable power law when given more compute, data, or parameters (model size), assuming they are not bottlenecked on one of the other resources. This has been observed as highly consistent over more than six orders of magnitude.
Graph from Scaling Laws for Neural Language Models
•
•
Applied to How much chess engine progress is about adapting to bigger computers? by Multicore at 6mo
•
•
•
Created by leogao at 8mo