Context: I created this to share before the start of courses I facilitate to provide a refresher on recent events, but also as a hook to draw students into engaging with some of ideas we think about in AI safety early. I expect the topics I cover in my “Setting the Scene” post to change frequently, so this is less polished than some of my other content. I emphasise to students that these posts are to be considered as bonus content—not core curriculum content—and that as such they are really just there for students who want to go deeper, before or after the course. I also provide a lot of links; this is to make it easy for students to follow up anything that catches their interest.

Any feedback on how to improve these posts would be greatly appreciated.

Cyber Security

I think it’s worth reflecting that it is a particularly pertinent time to be doing a course on AI safety. A lot has happened recently. In the last week, the big thing has been cyber vulnerabilities, most prominently php2shell (see Elias Taylor’s pushback against this bug being easily exploitable and Hacker News commentators pushing back against him), but a lot else has happened as well. The number of CVE’s (Common Vulnerabilities and Exposures) recently has been completely wild:

Drew asks Is this the new normal? and I’d certainly love to know! Elias Taylor argues Anything connected to the internet is going to get hacked to pieces:

Most software engineers do not look at the code they depend on. Do not delude yourself. The code they use is on github, but no one has ever actually audited the code or packages they're using…

Nowadays with LLMs, plenty of engineers aren’t even reading what they’re coding, let alone understanding it. Hilariously, the LLMs are better at security than most software engineers, so it’s slightly less worrying, but nevertheless the code that goes into production at some companies now is almost 100% not-ever-read-or-understood-by-anyone.
Software is not good. It is not secure. Get it out of your head that the people who run this industry are serious. It has never been anything resembling secure. The only reason things haven’t been hacked off the face of the web constantly is that most things aren’t ever being seriously tested.

He doesn't literally mean "anything"—he acknowledges that there are "some large projects like Google Chrome, curl and Linux (that) are generally very wel...

AI Safety

AI Safety