Better, but I still think "myopia" is basically misleading here. I would go back to the drawing board *shrug.
It seems a bit weird to me to call this myopia, since (IIUC) the AI is still planning for future impact (just not on other agents).
I think the contradiction may only be apparent, but I thought it was worth mentioning anyways. My point was just that we might actually want certifications to say things about specific algorithms.
Second, we can match the certification to the types of people and institutions, that is, our certifications talk about the executives, citizens, or corporations (rather than e.g. specific algorithms, that may be replaced in the future). Third, the certification system can build in mechanisms for updating the certification criteria periodically.
* I think effective certification is likely to involve expert analysis (including non-technical domain experts) of specific algorithms used in specific contexts. This appears to contradict the "Second" point above somewhat.* I want people to work on developing the infrastructure for such analyses. This is in keeping with the "Third" point.* This will likely involve a massive increase in investment of AI talent in the process of certification.
As an example, I think "manipulative" algorithms -- that treat humans as part of the state to be optimized over -- should be banned in many applications in the near future, and that we need expert involvement to determine the propensity of different algorithms to actually optimize over humans in various contexts.
You can try to partner with industry, and/or advocate for big government $$$.I am generally more optimistic about toy problems than most people, I think, even for things like Debate.Also, scaling laws can probably help here.
um sorta modulo a type error... risk is risk. It doesn't mean the thing has happened (we need to start using some sort of phrase like "x-event" or something for that, I think).
Yeah we've definitely discussed it! Rereading what I wrote, I did not clearly communicate what I intended to...I wanted to say that "I think the average trend was for people to update in my direction". I will edit it accordingly.I think the strength of the "usual reasons" has a lot to do with personal fit and what kind of research one wants to do. Personally, I basically didn't consider salary as a factor.
When you say academia looks like a clear win within 5-10 years, is that assuming "academia" means "starting a tenure-track job now?" If instead one is considering whether to begin a PhD program, for example, would you say that the clear win range is more like 10-15 years?
Also, how important is being at a top-20 institution? If the tenure track offer was instead from University of Nowhere, would you change your recommendation and say go to industry?
My cut-off was probably somewhere between top-50 and top-100, and I was prepared to go anywhere in the world. If I couldn't make into top 100, I think I would definitely have reconsidered academia. If you're ready to go anywhere, I think it makes it much easier to find somewhere with high EV (but might have to move up the risk/reward curve a lot).
Would you agree that if the industry project you could work on is the one that will eventually build TAI (or be one of the leading builders, if there are multiple) then you have more influence from inside than from outside in academia?
Yes. But ofc it's hard to know if that's the case. I also think TAI is a less important category for me than x-risk inducing AI.
Thanks! Quick question: how do you think these notions compare to factors in an undirected graphical model? (This is the closest thing I know of to how I imagine "notions" being formalized).
Cool! Can you give a more specific link please?