Top postsTop post
This post investigates emerging biorisk capabilities and when AI systems may cross important risk thresholds. It lays the groundwork for a broader risk forecast (currently in progress) on the annual likelihood of AI-enabled biological catastrophes. For now, the analysis focuses on general-purpose AIs such as large language models (LLMs). A...
My AI forecasting work aims to improve our understanding of the future so we can prepare for it and influence it in positive directions. Yet one problem remains: how do you turn foresight into action? I’m not sure, but I have some thoughts about learning the required skills. Say you...
TL;DR: This post examines how rapidly AI forecasting ability is improving by analyzing results from ForecastBench, Metaculus AI tournaments, and various prediction platforms. While AI forecasting is steadily improving, there remains significant uncertainty around the timeline for matching top human forecasters—it might be very soon or take several years. ForecastBench...
A thoughtful post by Anton Leicht, “Powerless Predictions,” notes that forecasting may be underutilized by AI policy organizations. But how much is forecasting actually being used? Which organizations working on AI—particularly AI safety—already integrate forecasting into their decision-making, policy recommendations, and strategic planning? What forecasting work is being done, and...
I envision a future more chaotic than portrayed in AI 2027. My scenario, the Rogue Replication Timeline (RRT), branches off mid-2026. If you haven’t already read the AI 2027 scenario, I recommend doing so before continuing. You can read the full version on my blog, but I have copied the...
There are several scenarios that, unlike AI 2027, explores consequences of AIs “going rogue” and freely replicating over the cloud. This post examines what these scenarios suggest might happen if OpenBrain—the leading AI company in the AI 2027 scenario—loses control of Agent-4. I’m writing this to inform my own alternative...
In the recent paper titled Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMS, AIs are finetuned to produce vulnerable code. This results in broad misaligned behavior in contexts that are not related to code—a phenomenon the authors refer to as emergent misalignment. Misalignment examples (Source: Alignment Forum) The dataset...