Your signal. Your price.
Brown notes the human genome is only about 3 GB, a small fraction of which codes for the brain. This compactness is plausible if evolution mainly writes 'Python code' for specific reward functions and bootstrapping rules, not the entire learned model.
Brown says current LLM training uses a 'really dumb' form of reinforcement learning without value functions, which is surprising it works so well. In contrast, parts of the basal ganglia may implement simple model-free RL, while the cortex builds a model-based system.
Brown states that creating a competent, misaligned agent like a 'paperclip maximizer' likely requires only minimal innate drives for curiosity and exploration, not the full suite of human social instincts. This is an alignment concern.
Brown advocates for massively scaling up neuroscience to get a 'ground truth,' specifically by driving down the cost of connectomics. The Welcome Trust estimated the first mouse brain connectome would cost billions; E11 Bio aims to reduce it to tens of millions.
Brown describes a moonshot idea of 'behavior cloning' or brain-regularized AI, where models are trained not just on labels but also to predict internal brain activity patterns. This could shape representations and improve generalization, but requires scalable brain scanning tech.
On automated theorem proving, Brown says RL from formal verification, as in Lean, will automate the mechanical parts of math. The harder challenge is automating the conceptual creativity of conjecturing interesting new theorems, which might require a loss function for explanatory power.
Brown says current models are not at the point where infinite inference budget yields superintelligence across all tasks. Factual retrieval plateaus quickly, while Sudoku improves indefinitely, showing a spectrum.
Brown uses poker bot creation as a personal evaluation. GPT-4.2 required steering but improved his code 10x; GPT-4.5 can now build a full solver with minimal guidance.
Brown says models lack research taste and cannot replace researchers yet. They accelerate some tasks 100x but bottleneck others, leading to gradual transformation rather than overnight replacement.
Large-scale test-time compute makes fast takeoff unlikely, Brown argues. Time becomes a bottleneck because models need long runtimes to unlock full capability, preventing an instantaneous intelligence explosion.
Brown states frontier labs understand the stakes and risks of their models. Competitive pressure exists, but researchers share a goal of steering toward positive outcomes.
Brown says he trusts GPT-4.5 outputs for high-stakes decisions like tax advice and condo paperwork more than expert human advice, indicating a shift in practical reliability.
Multi-agent systems are scratching the surface of capability, Brown believes. Frontier models are needed to unlock their potential, analogous to human civilization's accumulated knowledge scaffolding.
Benchmark gaming is easy by scaffolding multiple model runs or using judges, Brown warns. This inflates scores without controlling for test-time compute, making comparisons misleading.
Brown argues the research community agrees benchmarks should include a cost axis but is stuck in a bad equilibrium. Everyone publishes the grid because everyone expects it, despite knowing it's wrong.
Connor Brown compared this to the 1990s cryptography wars, predicting a fight over KYC and anti-compete laundering laws for frontier models with much higher stakes.