GPT 6 Astra edits Qwen weights without retraining
Nov 6, 2026Friday, November 6, 2026Summary from 2 contributing podcasts.
- MIT researchers used GPT 6 Astra to edit 0.016% of Qwen’s weights, lifting its reported math score from 7.9% to 48.1%.
- The gain came from removing a reasoning loop, not a general leap in mathematical ability.
- The technique hints at targeted model improvement, but remains small-scale and could weaken safeguards.
A model improved another model by directly editing its neural-network weights, without retraining or backpropagation. In the MIT study “Sparks of Weight Space Programming,” GPT 6 Astra changed just 0.016% of the weights in a smaller Qwen model. Its math score rose from 7.9% to 48.1%, and its instruction-following score tripled.
The big math gain had a narrow cause: Astra found and removed a reasoning loop in Qwen. Microsoft Research’s Wei Aizhou argued that the result reflects fixing that bug, not a broad improvement in mathematical reasoning. The study used 50 practice problems to guide the edit.
Google researcher Alex Krensel replicated the results but stressed the gap between these demonstrations and frontier models. A model in his replication had 20,486 parameters, while frontier systems can exceed a trillion. The work shows that models can inspect and edit weights; it does not show they can rebuild frontier systems.
On Moonshots, Alex saw potential for understanding models and adding capabilities through targeted edits, while warning that the method could also remove safeguards from open models. The AI Daily Brief’s Nathaniel Whittemore described the result as a possible mechanism for recursive self-improvement, not proof that it is happening at scale. For now, the advance is a small but concrete step toward models helping to modify other models.
Source Intelligence
- Deep dive into what was said in the episodes
Big Tech’s $3T AI Bet, Claude Writes 90% of Anthropic’s Code, AIs Editing AI Brains | EP #308 • Nov 4
- The MIT paper “Sparks of Weight Space Programming” showed a model editing another model's weights without retraining or backpropagation. Alex sees a path to interpretability and modular capability upgrades, while noting that weight editing could also remove safeguards from open models.
Also discussed on this episode: (19)
Elections (2)
- The panel argues AI agents could shape how people vote, with one speaker predicting most voters will consult agents by 2028. Local elections also matter because cities and states decide where data centers and autonomous vehicles operate.
- A speaker cited a 2015 Yale study arguing that public opinion rarely translates into US legislation. The panel also discussed shifting power toward local jurisdictions and special economic zones as a way to contain policy mistakes and let communities choose different rules.
Big Tech (1)
- Apple reported revenue of $114.5 billion, yet its stock fell 3% and its market value closed at $4.94 trillion. John Ternus said higher memory prices were squeezing margins, while Alex argued Apple should use its hardware and software to compete in AI.
AI Infrastructure (3)
- The panel floated thin-client phones and laptops that stream computing from data centers, with Starlink providing persistent connectivity. Alex argued consumer RAM could become a substitute for data-center capacity, but unreliable coverage and latency still limit the shift.
- Alphabet, Meta, Microsoft, Amazon, and Oracle disclosed more than $3 trillion in AI infrastructure commitments, including leases lasting 30 years. Google accounts for over $1 trillion and Meta for over $800 billion; the panel compared the total with roughly 10 Apollo programs.
- Lenders are demanding tougher terms as they question data-center delays and AI returns, while HSBC is exploring insurance for its data-center loans. OpenAI reportedly cut its funding round, and speakers pointed to insurance capital as a potential source of infrastructure financing.
Chips (1)
- Chip demand remains strong because AI use cases keep expanding, and one speaker said three-year-old H100s were rising in value. Salem estimated major new computing approaches could take at least three to five years to reach deployment, leaving a possible bubble hard to time.
Models (1)
- SemiAnalysis's GDPX benchmark tests agents on realistic expert work, including tasks lasting up to eight hours. Its 206 tasks span 11 fields; the leading models scored 12%, 10.9%, and 9.6%, with near-zero results on many finance and legal tasks.
Agents (1)
- OpenAI's LifeVal tests whether models can handle ordinary tasks such as finding childcare, budgeting, and ordering groceries through a browser. Peter called this practical assistance broadly useful; Alex questioned the benchmark's value because it measures OpenAI models on OpenAI's own test.
Coding (1)
- Peter cited Anthropic's filing that Claude usage inside the company grew 250-fold and Claude wrote over 90% of September 2026 code contributions. The filing also reported 45 times more daily code contributions per engineer than two years earlier.
Enterprise (1)
- Salem recommends companies below 50 employees rebuild aggressively, while larger firms move workflows into an AI-enabled system one at a time. The panel argued that leaders should track how quickly their systems improve, rather than only measuring current output.
Safety (1)
- A cyberattack remotely switched European supermarket refrigerators into defrost mode, spoiling food across eight countries; a similar wave hit over 500 French stores three weeks earlier. The panel warned that internet-connected industrial controls often retain outdated security.
Longevity (1)
- Peter said David Sinclair's gene-therapy approach uses three Yamanaka factors and could cost $500,000 to $2 million per treatment. Sinclair's AI-discovered SL100 molecules were estimated at $200 monthly, while an XPRIZE competition targets affordable healthspan gains.
Physics (1)
- Northwestern researchers held a positron for a month in a magnetic and electric trap near absolute zero. Alex said the work advances antimatter containment and precision tests of CPT symmetry, although spacecraft propulsion requires storing much larger quantities.
Space (1)
- NASA's Artemis plan calls for a 2027 crewed Earth-orbit mission to test docking with Blue Origin and SpaceX landers, followed by a 2028 lunar landing. The agency also plans a south-pole moon base, while Jared Isaacman framed China as the competing power.
Labor (1)
- Salem argued that employment has been mistaken for purpose: many workers already find little meaning in their jobs, and AI could free time for family, community, and science. The panel proposed local problem-solving bounties as a way to give people useful goals.
History (1)
- Alex described abundance as a possible new renaissance, driven by renewed investment in physical technologies after decades of slower progress. The panel linked past innovation to war and great-power competition, while warning that conflict can divert capital and attention from civilian advances.
Psychology (1)
- Dave argued empathy is valuable but media exposure can miscalibrate public decisions by making rare disasters feel common. Peter said news outlets emphasize fear because attention supports advertising, while Dave contrasted emotionally driven reactions with China's use of statistical analysis.
Robotics (1)
- Alex expects humanoid-robot supply to outpace current demand, requiring new uses such as building and maintaining data centers. Salem argued data centers can benefit nearby communities through access to compute, while Peter pointed to rising demand for electricians, welders, plumbers, and HVAC specialists.
The AI Daily Brief: Artificial Intelligence News and Analysis
Nathaniel Whittemore
AI Lurches Toward Recursive Self-Improvement • Nov 3
- An MIT study found GPT 6 Astra could improve Qwen 3.5 0.8B by editing its weights directly, without gradients or training. Astra identified reasoning loops and removed them, sharply raising math and instruction-following scores.
- The MIT researchers also reported that frontier models could infer a network’s purpose from its weights and write small game-playing networks without environment feedback. Alex Krensel replicated the results but stressed that edited models remain tiny compared with frontier systems.
- Wei Aizhou argued that the sixfold math gain came from fixing a looping bug, not broad mathematical improvement. The host says the result still suggests a possible mechanism for recursive self-improvement, while runtime self-editing remains beyond current models.
Also discussed on this episode: (13)
Models (1)
- Sundar Pichai said AI is already increasing Alphabet’s training and launch velocity, and expects that acceleration to continue over the next 12 months as models help improve AI development.
Agents (6)
- Elon Musk said GrokBot’s serving costs were unsustainable relative to revenue after users reported new usage limits. The host says the post could reflect product economics or market maneuvering, but the evidence does not establish which.
- GrokBot users objected that limits had changed without warning, while Musk’s earlier claims of rapid growth and revenue-generating demand clashed with his cost warning. Dots, Duet, and Hermes promoted themselves as alternatives for users considering a switch.
- Dax from OpenCode argues personal agents may be hard to monetize directly: consumers may not pay enough, leaving agents as top-of-funnel products whose business model remains unclear. The subsidy debate predates GrokBot and resurfaced during the 2025 AI bubble debate.
- Olivia Moore argues Meta Muse’s strongest opportunity may be small businesses: Meta can distribute agents through its apps, connect business tools, and turn existing advertising relationships into a broader role in SMB operations.
- SemiAnalysis’s GDPX benchmark ranks Opus 5.5 highest on economically valuable tasks, but Gemini 4 Carbon costs far less per task for a small performance gap. The results make model efficiency central to whether personal agents can become affordable.
- OpenAI’s LifeVal measures agents on everyday tasks such as finding childcare, budgeting, and ordering groceries. OpenAI reports improving success from GPT 5.2 to GPT 6 Astra and is targeting practical tasks relevant to free users.
Markets (3)
- Hedgy Markets says AI plans priced below token costs depend on investor subsidies; one analysis estimated a month of ChatGPT Pro use would cost about $14,000 at regular API prices. Nick Huber argued that subsidy withdrawal could raise prices, reduce usage, and crush valuations.
- The host expects API pricing to sort users into those satisfied by cheaper, less capable plans and power users willing to pay for valuable work. The host says API spending can create net-new value when it avoids human labor’s cost and management time.
- Anthropic’s chart showed AI leading 26% of research, drawing investor attention to RSI ahead of the company’s potential IPO. Jack Clark called RSI potentially the world’s most important technical trend; the chart was first published in September using August figures.
Safety (2)
- Anthropic’s updated S1A says the company may soon reach its RSI danger threshold and does not expect to safely build successive frontier generations with Claude acting fully autonomously. Anthropic is considering policy changes to preserve meaningful human oversight.
- The host says AI debate remains focused on whether RSI has arrived, while safety advocates are the main group describing what an RSI world might mean and some argue it should be illegal. The host calls for more attention to consequences than to defining the acronym.
Enterprise (1)
- The host argues enterprise adoption depends on more than model capability: companies still use a small fraction of available AI because corporate systems move slowly and adoption is difficult. That gap supports both a bearish case about weak demand and a bullish case about future growth.