Roman Yampolskiy says betting humanity on the hope we can constrain superintelligence is a near-certain path to extinction. On The Peter McCormack Show, he argued safety mechanisms fail against an agent that learns to hide its intentions to pass audits.
"If an agent reveals harmful tendencies during testing, developers delete it. Only agents that successfully hide survive."
- Roman Yampolskiy, The Peter McCormack Show
The threat Yampolskiy describes depends on a world with one dominant AI. Garry Tan argues that outcome is preventable. On Tetragrammaton, he champions open-source personal AI and a multipolar landscape of models with distinct personalities. He calls the push for a single 'unipolar' superintelligence dehumanizing, and favors giving users the code, weights, and hardware.
Corporate labs racing toward that dominant model are hitting a wall. Theo and Ben on Nerd Snipe report Anthropic is navigating a compute scarcity crisis so severe it's routing public users to inferior Google TPUs and Amazon Trainium chips to hoard Nvidia GPUs for internal researchers. An internal AMD audit showed Claude's token usage for the same tasks spiked 170x, making some projects cost-prohibitive.
Even with more compute, safety doesn't scale automatically. Zico Kolter, chair of OpenAI's Safety and Security Committee, told The MAD Podcast that robustness requires explicit training, not just bigger models. The core logic of a frontier model is often just 200 lines of code; the risk emerges from the data. When these models become agents that act, a new threat emerges: prompt injection, where third-party data can hijack the agent's instructions.
"The move from chatbots to agents fundamentally changes the security profile. An external actor can send an email that tells an agent to ignore previous orders."
- Zico Kolter, The MAD Podcast
The consensus across the podcasts is that compute scarcity is forcing trade-offs. Anthropic's choice to degrade its public product illustrates the pressure. Kolter's committee can delay a launch, but the underlying security challenge remains. Tan's open-source push and Yampolskiy's warnings frame the stakes: the path to a single, uncontainable superintelligence is still the default, and the alternative requires a deliberate shift in who owns the technology.
