I do not think it is at ALL clear that LLMs lack some of the important cognitive algorithms that humans possess and have not gotten better at them over the years
Hmm, now that I think about it, I think this might account for most of the model delta between us.
But maybe there's a somewhat different AI architecture that can have those skills. Why should it be so incredibly hard to find?
Pretty sure there is indeed such an architecture, and I'm not at all sure it would be that hard to find; that's what I talk about when evaluating (3). But if such a discovery is required, I think it makes predicting general ASI by straightforward extrapolations of in-paradigm trends much more dubious, smearing out the probability distribution over timelines. Like, I do have pretty short timelines, overall; just not the modern-day instantaneously short timelines.