This is a special post for quick takes by Daniel Kokotajlo. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
7 comments, sorted by Click to highlight new comments since:

I found this article helpful and depressing. Kudos to TracingWoodgrains for detailed, thorough investigation.

Technologies I take for granted now but remember thinking were exciting and cool when they came out

  • Smart phones
  • Google Maps / Google Earth
  • Video calls
  • Facebook
  • DeepDream (whoa! This is like drug hallucinations... I wonder if they share a similar underlying mechanism? This is evidence that ANNs are more similar to brains than I thought!)
  • AlphaGo
  • AlphaStar (Whoa! AI can handle hidden information!)
  • OpenAI Five (Whoa! AI can work on a team!)
  • GPT-2 (Whoa! AI can write coherent, stylistically appropriate sentences about novel topics like unicorns in the andes!)
  • GPT-3

I'm sure there are a bunch more I'm missing, please comment and add some!

Some of my own:

  • SSDs
  • laptops
  • CDs
  • digital cameras
  • modems
  • genome sequencing
  • automatic transmissions for cars that perform better than a moderately skilled human using a manual transmission can
  • cheap shipping
  • solar panels with reasonable power generation
  • breathable wrinkle free fabrics that you can put in the washing machine
  • bamboo textiles
  • good virtual keyboards for phones
  • scissor switches
  • USB
  • GPS

Oh yeah, cheap shipping! I grew up in a military family, all around the world, and I remember thinking it was so cool that my parents could go on "ebay" and order things and then they would be shipped to us! And then now look where we are -- groceries delivered in ten minutes! Almost everything I buy, I buy online!

Rereading this classic by Ajeya Cotra:

I feel like this is an example of a piece that is clear, well-argued, important, etc. but which doesn't seem to have been widely read and responded to. I'd appreciate pointers to articles/posts/papers that explicitly (or, failing that, implicitly) respond to Ajeya's training game report. Maybe the 'AI Optimists?' 

Searching for equilibria can be infohazardous. You might not like the one you find first, but you might end up sticking with it (or worse, deviating from it and being punished). This is because which equilbrium gets played by other people depends (causally or, in some cases, acausally) not just on what equilibrium you play but even on which equilibria you think about. For reasons having to do with schelling points. A strategy that sometimes works to avoid these hazards is to impose constraints on which equilibria you think about, or at any rate to perform a search through equilibria-space that is guided in some manner so as to be unlikely to find equilibria you won't like. For example, here is one such strategy: Start with a proposal that is great for you and would make you very happy. Then, think of the ways in which this proposal is unlikely to be accepted by other people, and modify it slightly to make it more acceptable to them while keeping it pretty good for you. Repeat until you get something they'll probably accept.

I think it is useful to distinguish between two dimensions of competitiveness: Resource-competitiveness and date-competitiveness. We can imagine a world in which AI safety is date-competitive with unsafe AI systems but not resource-competitive, i.e. the insights and techniques that allow us to build unsafe AI systems also allow us to build equally powerful safe AI systems, but it costs a lot more. We can imagine a world in which AI safety is resource-competitive but not date-competitive, i.e. for a few months it is possible to make unsafe powerful AI systems but no one knows how to make a safe version, and then finally people figure out how to make a similarly-powerful safe version and moreover it costs about the same.