This is a special post for quick takes by Thane Ruthenis. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
Comment Permalink
I am pretty annoyed with the messaging around METR's latest result.
Our chief concern is RSI dynamics, i. e., AI capability progress making subsequent AI capability progress easier. The way it would show up on graphs is by making straight lines steeper and steeper. If we observed 1.5x acceleration in one time period, we should observe 2x acceleration in the next time period, 2.5x in some subsequent one, et cetera. This is what RSI implies: a line continually going up on the graph of productivity multipliers. In other words, research acceleration at T+1 should be a function of research acceleration at T. This is what scenarios like AI 2027 assume: that there are compounding multiplicative dynamics at play.
But that is not what the graph in the report, if taken at face value, shows. The graph shows a single slope change around the start of 2026 – something that made Anthropic permanently better at AI research – and then no additional productivity improvements all throughout 2026. This is better modeled as a one-off "productivity unlock", rather than smoothly predictable RSI.[1]
In particular, it's not straightforwardly extrapolable into claims of the form "we will reach 4x acceleration by January 2027, 8x acceleration by March", etc. Taken at face value, the graph says we reached 2x acceleration by early 2026, and that we're still at 2x acceleration. Will we stay at 2x all throughout 2027 as well? Will there be a different discontinuous slope change that puts us at 10x next week?[2] Who knows.
There are two model breaks: (1) the current speed-up to AI research from AI is much larger than previously predicted, (2) the current speed-up to the speed-up is much lower than was expected at this speed-up value. This means that whatever this 2x-speedup figure is, it's not the same type of figure as the compounding multiplier in AI 2027. (Say, maybe the straight line on the multipliers graph should have a vertical offset, because significant aspects of the speed-up are non-compounding?)
Yet, people are blindly plugging that number into their compounding-multiplicative-dynamics models, and then interpreting that as, wow, we're just straightforwardly N months ahead of the AI 2027 schedule, ASI first thing next year.
Look, I don't even necessarily disbelieve the claims that people are trying to frame these results as making. Yes, surely there are some compounding multiplicative dynamics at play somewhere, and plausibly AI progress (on verifiable tasks et cetera) is currently going 2x faster than baseline largely due to boosts from AI itself. Sure, why not? Sounds plausible enough.
But, like, what that specific piecewise-straight line on that specific graph is literally saying is not what people are taking it as.
Edit: @Vivek S pointed out a relevant line I'd missed:
We may have missed recent further acceleration. The rate of growth could be undergoing continuous acceleration, e.g. hyperbolic growth. While we do not see clear evidence of this in the AECI trends, there is other evidence that makes this seem more plausible which Anthropic redacted.
I don't think this kills my objection, though. Again, the main thing I take issue with is taking the model-breaking 2x speedup at face value and then assuming that the meta-speedup is nevertheless consistent with previous models. AI 2027's March is clearly much crazier than what's actually happening on the ground.
- ^
There's even a plausible mechanism for that, see interviews where Anthropic employees say things like "now only bottlenecked on ideas & compute, no longer bottlenecked on code". If they got basically-completely-unbottlenecked around Mythos...
- ^
Say, as LLMs finally grok research taste and become able to contribute fully autonomously.
4
2
2Showing 3 of 6 replies (Click to show all)
74
We believe AI’s contribution has grown relatively more important over time, and do not think the other factors we investigated would explain the sustained acceleration.
The picture described in the report is consistent with automation making progress faster and faster over time. Imagine subtracting the grey line from the orange line in this hypothetical graph. This would correspond to the progress curve bending up.
6
Zoom out. If the slope keeps getting steeper and we have a new "one-time" slope increase every couple of doublings, we should put high probability on the trend actually being superlinear even if a small subset of data since 2025 is not clear enough either way to prove it. When you zoom out, it is more likely that the underlying data-generating process is superlinear (eg the sum of a superlinearly increasing number of S-curves or the result of superlogarithmic scaling with inputs) with noise that appears to divide it into ~3 linear regimes, rather than actually being 3 linear regimes. The data do not even rule out that the slope is 4x starting a couple of months ago.
(Note the extrapolated time horizon numbers are not meaningful because this was made by stitching together time horizon and AECI, but I believe ECI extended back to 2019 would look roughly like this).
Credit to @Nikola Jurkovic for the graph
6
I think it's also important to remember that the piecewise-straight line is a part of the map, not a part of the territory. Imagine looking at the graph without those lines drawn over it. The error bars are pretty wide, so it's not obvious that an exponential would necessarily fit poorly.
Going one level further on the map-territory thing, AECI is described as a metric consisting of a mixture of benchmarks. If the benchmarks saturate at different levels, then how sure can we really be that a difference of 10 AECI points at 180 is the same size change as a difference of 10 AECI points at 120?