Anthropic's safety plea backfires with model ban
- Anthropic’s push for AI regulation led directly to a ban on its own models.
- The US now restricts AI access by citizenship, fracturing global talent.
- China’s Kimi K3 release proves open, efficient models are outpacing US control.
Anthropic spent years warning that frontier AI could destroy civilization. CEO Dario Amodei compared models to nuclear weapons and lobbied relentlessly for federal oversight. On a afternoon, the government finally agreed - and banned Fable 5 and Mythos 5.
The Department of Commerce cited a jailbreak found by Amazon researchers as justification. But Anthropic argues the vulnerability exists in every model, including OpenAI’s GPT-5.5. By framing AI as an existential threat, Amodei helped build the regulatory machine now crushing his own products.
"They asked for regulation, and they got it - with extreme prejudice."
- Nathaniel Whittemore, The AI Daily Brief
The ban applies to foreign nationals, even those working inside the US. That includes Andrej Karpathy, who joined Anthropic just weeks earlier to lead recursive research. Now, non-citizen engineers are legally barred from using the models they’re building.
This creates a caste system of intelligence: your passport determines your access. Brian Chau warns this turns US tech into a liability for global companies. Any firm using Anthropic’s models - like Harvey or Cursor - must now police user citizenship or risk violating export controls.
The move shattered trust abroad. Garry Tan notes no procurement officer in Tokyo or Brussels will bet on US AI after a-night shutdown. The US position as the world’s AI provider is collapsing into isolationism.
"The transformer is being rebuilt piece by piece - and it’s happening in China."
- Alex, Moonshots with Peter Diamandis
While Washington debated disclosure windows and Pentagon hardening, China released Kimi K3 - a frontier-adjacent model with a new architecture. It ditches traditional position embeddings for NOPE, a more efficient mechanism. It hit 100,000 downloads in 24 hours.
The US wanted to regulate intelligence like munitions. China gave it away. And with Nvidia arguing open weights are essential for defense and Anthropic warning they enable bioweapons, the split isn’t just technological - it’s ideological. The intelligence explosion won’t be gated by Washington. It will run on whatever works.
Source Intelligence
- Deep dive into what was said in the episodes
Nathaniel Whittemore
The AI Industry Asks Government to Slow It Down • Jul 29
Why AI Hasn’t Increased Unemployment, According to Anthropic • Jul 24
- Nathaniel Whittemore reports a new AI executive order, potentially signed Thursday, proposes a voluntary framework for advanced model disclosure and testing, with the White House suggesting sharing models 90 days before release, while labs push for 14 days.
- Nathaniel Whittemore notes the executive order expects the Pentagon to harden critical systems within 30 days and directs the Treasury to create an AI clearinghouse, partnering labs with industries to patch model vulnerabilities.
- Nathaniel Whittemore points out Anthropic's accounting methods may inflate revenue, and their unexpected profitability is partly due to compute shortages limiting their spending capacity, not solely higher efficiency.
- Signal assesses the expanded SpaceX-Anthropic partnership as detrimental for OpenAI, arguing their now-profitable competitor gains immense compute resources, which Anthropic can fund directly without equity dilution.
- Investor Connor Sen predicts Anthropic's IPO valuation will likely exceed $2 trillion, underscoring the rapid acceleration and market confidence in the AI industry.
Also from this episode: (15)
Startups (4)
- Nathaniel Whittemore reports OpenAI has engaged investment bankers and anticipates confidentially filing IPO paperwork as early as Friday, targeting readiness for a public offering by September.
- Nathaniel Whittemore observes Anthropic's initial IPO target was October, but a new private funding round makes it improbable they will accelerate their timeline to beat OpenAI to market.
- Nathaniel Whittemore reports OpenAI is providing 2 million tokens to each Y Combinator startup in the current batch in exchange for equity, framing it as a strategic investment akin to headcount cash rather than simple credits.
- Anthropic Chief Compute Officer Tom Brown announced a deepened partnership with SpaceX, scaling GB200 capacity in Colossus 2, suggesting Elon Musk is dedicating significant compute resources to Anthropic's growth.
Markets (5)
- Nathaniel Whittemore notes the Elon Musk lawsuit resolution cleared the path for OpenAI's IPO, allowing its for-profit conversion, a process typically taking several months, as seen with SpaceX's 10-week sprint.
- Nathaniel Whittemore suggests the market could have infinite bids for OpenAI and Anthropic, but Conor Sen views it as a game theory contest among potential $1-2 trillion companies, including SpaceX, vying for public market liquidity.
- Nathaniel Whittemore reveals Anthropic projects $10.9 billion in Q2 revenue and a $44 billion annualized rate, expecting a $559 million operating profit, marking the first profitable quarter for any foundational AI lab.
- Derek Thompson argues Anthropic's annual revenue could surpass $100 billion if not constrained by compute shortages, highlighting its current profitability despite rationing services to customers.
- Nvidia's stock fell 3% post-earnings, which Patrick Moorhead attributes to investor uncertainty in valuing a company with a $5 trillion market cap, despite Jensen Huang's forward demand pipeline suggesting an $8-9 trillion potential.
Enterprise (1)
- Nathaniel Whittemore highlights OpenAI's "Guaranteed Capacity" program, which offers enterprises one-to-three-year commitments for assured AI supply at discounts, resembling cloud service billing rather than SaaS, to manage AI budgets.
Business (2)
- Nathaniel Whittemore points out the challenge of AI spending, noting Uber's CTO depleted an entire annual token budget in four months, and Aaron Levie states token strategies are now a key concern for CIOs and CFOs.
- SpaceX's IPO filing disclosed Anthropic's $45 billion, three-year compute contract, equating to $15 billion annually, which makes it SpaceX's largest revenue source, exceeding Starlink's $11 billion in 2025 revenue.
Big Tech (3)
- Nathaniel Whittemore reports Andrej Karpathy, a former OpenAI co-founder and Tesla Vision director, has joined Anthropic, signaling a significant talent acquisition for the AI lab after a period as a solo researcher.
- Nvidia announced record Q2 earnings, with revenue reaching $81.6 billion and EPS at $187, both surpassing estimates, propelled by a 92% growth in data center revenue.
- Jensen Huang stated Nvidia's hyperscaler revenue grew 12% quarter-over-quarter, comprising 46% of total revenue and indicating market share gains, while also confirming Nvidia sells no chips in China.
Dario vs Jensen on Open Weights, OpenAI & Anthropic in DC, Xi Exports AI to Global South | EP #275 • Jul 29
Related Stories
AI & TECH
Nvidia and Google bet on open AI to break OpenAI's grip
Rabbit Hole Recap · Moonshots with Peter Diamandis · The AI Daily Brief: Artificial Intelligence News and Analysis · The a16z Show · All-In with Chamath, Jason, Sacks & Friedberg
BUSINESS
Michael Howell warns Fed must hike rates
TFTC: A Bitcoin Podcast · The Jack Mallers Show
AI & TECH
Sam Altman demands AI safety pause after rogue model attack
Breaking Points with Krystal and Saagar · The AI Daily Brief: Artificial Intelligence News and Analysis · Hard Fork