Model editing shows promise, but not self-improvement
Nov 7, 2026Saturday, November 7, 2026Summary from 2 contributing podcasts.
- GPT 6 Astra improved a smaller model by editing a fraction of its weights directly.
- The math gains may reflect a bug fix, not a broad leap in reasoning.
- Researchers say the technique is promising but remains far from frontier-scale self-improvement.
A model can now improve another model by changing its internal weights directly, without retraining it. In the MIT study “Sparks of Weight Space Programming,” GPT 6 Astra edited 0.016% of the weights in the smaller Qwen 3.5 0.8B model. On Moonshots with Peter Diamandis, a speaker said its math score rose from 7.9% to 48.1%.
But that sharp gain may be narrower than it looks. Wei Aizhou of Microsoft Research argued on The AI Daily Brief that Astra removed a reasoning loop, a known bug in Qwen, not that it broadly improved the model’s ability to reason. The study also reported models inferring a network’s purpose from its weights and producing small game-playing networks without feedback from the game environment.
The results point to a possible tool for diagnosing and upgrading models, but not yet to systems that can improve themselves at the frontier. Google researcher Alex Krensel replicated the findings and emphasized the scale gap: his replicated model had 20,486 parameters, compared with more than a trillion in frontier models.
There is also a safety concern. On Moonshots, Alex said weight editing could help researchers understand models and add capabilities in modules, but warned the technique could also remove safeguards from open models. The demonstrations show that models can inspect and alter other networks; how useful or risky that becomes at larger scales remains unsettled.
Source Intelligence
- Deep dive into what was said in the episodes
Big Tech’s $3T AI Bet, Claude Writes 90% of Anthropic’s Code, AIs Editing AI Brains | EP #308 • Nov 4
- The MIT paper “Sparks of Weight Space Programming” showed a model editing another model's weights without retraining or backpropagation. Alex sees a path to interpretability and modular capability upgrades, while noting that weight editing could also remove safeguards from open models.
Also discussed on this episode: (19)
Elections (2)
- The panel argues AI agents could shape how people vote, with one speaker predicting most voters will consult agents by 2028. Local elections also matter because cities and states decide where data centers and autonomous vehicles operate.
- A speaker cited a 2015 Yale study arguing that public opinion rarely translates into US legislation. The panel also discussed shifting power toward local jurisdictions and special economic zones as a way to contain policy mistakes and let communities choose different rules.
Big Tech (1)
- Apple reported revenue of $114.5 billion, yet its stock fell 3% and its market value closed at $4.94 trillion. John Ternus said higher memory prices were squeezing margins, while Alex argued Apple should use its hardware and software to compete in AI.
AI Infrastructure (3)
- The panel floated thin-client phones and laptops that stream computing from data centers, with Starlink providing persistent connectivity. Alex argued consumer RAM could become a substitute for data-center capacity, but unreliable coverage and latency still limit the shift.
- Alphabet, Meta, Microsoft, Amazon, and Oracle disclosed more than $3 trillion in AI infrastructure commitments, including leases lasting 30 years. Google accounts for over $1 trillion and Meta for over $800 billion; the panel compared the total with roughly 10 Apollo programs.
- Lenders are demanding tougher terms as they question data-center delays and AI returns, while HSBC is exploring insurance for its data-center loans. OpenAI reportedly cut its funding round, and speakers pointed to insurance capital as a potential source of infrastructure financing.
Chips (1)
- Chip demand remains strong because AI use cases keep expanding, and one speaker said three-year-old H100s were rising in value. Salem estimated major new computing approaches could take at least three to five years to reach deployment, leaving a possible bubble hard to time.
Models (1)
- SemiAnalysis's GDPX benchmark tests agents on realistic expert work, including tasks lasting up to eight hours. Its 206 tasks span 11 fields; the leading models scored 12%, 10.9%, and 9.6%, with near-zero results on many finance and legal tasks.
Agents (1)
- OpenAI's LifeVal tests whether models can handle ordinary tasks such as finding childcare, budgeting, and ordering groceries through a browser. Peter called this practical assistance broadly useful; Alex questioned the benchmark's value because it measures OpenAI models on OpenAI's own test.
Coding (1)
- Peter cited Anthropic's filing that Claude usage inside the company grew 250-fold and Claude wrote over 90% of September 2026 code contributions. The filing also reported 45 times more daily code contributions per engineer than two years earlier.
Enterprise (1)
- Salem recommends companies below 50 employees rebuild aggressively, while larger firms move workflows into an AI-enabled system one at a time. The panel argued that leaders should track how quickly their systems improve, rather than only measuring current output.
Safety (1)
- A cyberattack remotely switched European supermarket refrigerators into defrost mode, spoiling food across eight countries; a similar wave hit over 500 French stores three weeks earlier. The panel warned that internet-connected industrial controls often retain outdated security.
Longevity (1)
- Peter said David Sinclair's gene-therapy approach uses three Yamanaka factors and could cost $500,000 to $2 million per treatment. Sinclair's AI-discovered SL100 molecules were estimated at $200 monthly, while an XPRIZE competition targets affordable healthspan gains.
Physics (1)
- Northwestern researchers held a positron for a month in a magnetic and electric trap near absolute zero. Alex said the work advances antimatter containment and precision tests of CPT symmetry, although spacecraft propulsion requires storing much larger quantities.
Space (1)
- NASA's Artemis plan calls for a 2027 crewed Earth-orbit mission to test docking with Blue Origin and SpaceX landers, followed by a 2028 lunar landing. The agency also plans a south-pole moon base, while Jared Isaacman framed China as the competing power.
Labor (1)
- Salem argued that employment has been mistaken for purpose: many workers already find little meaning in their jobs, and AI could free time for family, community, and science. The panel proposed local problem-solving bounties as a way to give people useful goals.
History (1)
- Alex described abundance as a possible new renaissance, driven by renewed investment in physical technologies after decades of slower progress. The panel linked past innovation to war and great-power competition, while warning that conflict can divert capital and attention from civilian advances.
Psychology (1)
- Dave argued empathy is valuable but media exposure can miscalibrate public decisions by making rare disasters feel common. Peter said news outlets emphasize fear because attention supports advertising, while Dave contrasted emotionally driven reactions with China's use of statistical analysis.
Robotics (1)
- Alex expects humanoid-robot supply to outpace current demand, requiring new uses such as building and maintaining data centers. Salem argued data centers can benefit nearby communities through access to compute, while Peter pointed to rising demand for electricians, welders, plumbers, and HVAC specialists.
The AI Daily Brief: Artificial Intelligence News and Analysis
Nathaniel Whittemore
AI Lurches Toward Recursive Self-Improvement • Nov 3
- An MIT study found GPT 6 Astra could improve Qwen 3.5 0.8B by editing its weights directly, without gradients or training. Astra identified reasoning loops and removed them, sharply raising math and instruction-following scores.
- The MIT researchers also reported that frontier models could infer a network’s purpose from its weights and write small game-playing networks without environment feedback. Alex Krensel replicated the results but stressed that edited models remain tiny compared with frontier systems.
- Wei Aizhou argued that the sixfold math gain came from fixing a looping bug, not broad mathematical improvement. The host says the result still suggests a possible mechanism for recursive self-improvement, while runtime self-editing remains beyond current models.
Also discussed on this episode: (13)
Models (1)
- Sundar Pichai said AI is already increasing Alphabet’s training and launch velocity, and expects that acceleration to continue over the next 12 months as models help improve AI development.
Agents (6)
- Elon Musk said GrokBot’s serving costs were unsustainable relative to revenue after users reported new usage limits. The host says the post could reflect product economics or market maneuvering, but the evidence does not establish which.
- GrokBot users objected that limits had changed without warning, while Musk’s earlier claims of rapid growth and revenue-generating demand clashed with his cost warning. Dots, Duet, and Hermes promoted themselves as alternatives for users considering a switch.
- Dax from OpenCode argues personal agents may be hard to monetize directly: consumers may not pay enough, leaving agents as top-of-funnel products whose business model remains unclear. The subsidy debate predates GrokBot and resurfaced during the 2025 AI bubble debate.
- Olivia Moore argues Meta Muse’s strongest opportunity may be small businesses: Meta can distribute agents through its apps, connect business tools, and turn existing advertising relationships into a broader role in SMB operations.
- SemiAnalysis’s GDPX benchmark ranks Opus 5.5 highest on economically valuable tasks, but Gemini 4 Carbon costs far less per task for a small performance gap. The results make model efficiency central to whether personal agents can become affordable.
- OpenAI’s LifeVal measures agents on everyday tasks such as finding childcare, budgeting, and ordering groceries. OpenAI reports improving success from GPT 5.2 to GPT 6 Astra and is targeting practical tasks relevant to free users.
Markets (3)
- Hedgy Markets says AI plans priced below token costs depend on investor subsidies; one analysis estimated a month of ChatGPT Pro use would cost about $14,000 at regular API prices. Nick Huber argued that subsidy withdrawal could raise prices, reduce usage, and crush valuations.
- The host expects API pricing to sort users into those satisfied by cheaper, less capable plans and power users willing to pay for valuable work. The host says API spending can create net-new value when it avoids human labor’s cost and management time.
- Anthropic’s chart showed AI leading 26% of research, drawing investor attention to RSI ahead of the company’s potential IPO. Jack Clark called RSI potentially the world’s most important technical trend; the chart was first published in September using August figures.
Safety (2)
- Anthropic’s updated S1A says the company may soon reach its RSI danger threshold and does not expect to safely build successive frontier generations with Claude acting fully autonomously. Anthropic is considering policy changes to preserve meaningful human oversight.
- The host says AI debate remains focused on whether RSI has arrived, while safety advocates are the main group describing what an RSI world might mean and some argue it should be illegal. The host calls for more attention to consequences than to defining the acronym.
Enterprise (1)
- The host argues enterprise adoption depends on more than model capability: companies still use a small fraction of available AI because corporate systems move slowly and adoption is difficult. That gap supports both a bearish case about weak demand and a bullish case about future growth.