I disagree with this position, but if I held it, I would be saying somewhat similar things to Zach (even having read the book).

I wouldn't. I roughly agree with Zach's background position (i.e. I'm quite uncertain about the likelihood of extinction conditional on YOLO-ing the current paradigm*) but I still think his conclusions are wild. Quoting Zach:

First, it leaves room for AI's transformative benefits. Tech has doubled life expectancy, slashed extreme poverty, and eliminated diseases over the past two centuries. AI could accelerate these trends dramatically.

The tradeoff isn't between solving scarcity at a high risk of extinction vs. never getting either of those things. It's between solving scarcity now at a high risk of extinction, vs. solving scarcity later at a much lower risk.

Second, focusing exclusively on extinction scenarios blinds us to other serious AI risks: authoritarian power grabs, democratic disruption through misinformation, mass surveillance, economic displacement, new forms of inequity. These deserve attention too.

Slowing down / pausing AI development gives us more time to work on all of those problems. Racing to build ASI means not only are we risking extinction from misalignment, but we're also facing a high risk of outcomes such as, for example, ASI being developed so quickly that governments don't have time to get a handle on what's happening and we end up with Sam Altman as permanent world dictator. (I don't think that particular outcome is that likely, it's just an example.)

*although I think my conditional P(doom) is considerably higher than his