The Frontier

Adam Brown argues the most important unanswered question in science is how the human brain achieves high sample efficiency and general capabilities with far less data than modern LLMs. His meta-level take is that neuroscience needs a technological power-up to answer it.

Brown's personal hunch is that AI has neglected complex, developmentally staged loss functions. Evolution encodes a specific learning curriculum through many different loss functions, which could be the key to the brain's efficiency.

Brown suggests the cortex might be an omnidirectional inference engine, predicting any subset of variables from any other subset, unlike LLMs which are natively optimized only for next-token prediction.

Brown outlines Steve Byrnes' theory that the brain's learning subsystem learns to predict the innate responses of a separate steering subsystem, wiring abstract concepts like 'spider' to primitive reflexes like flinching and enabling generalization.

Single-cell atlas data shows many more diverse and bespoke cell types in subcortical steering regions like the hypothalamus than in the cortex. Brown interprets this as evidence that evolution's genomic complexity is spent wiring innate reward functions, not the general learning algorithm.

Brown notes the human genome is only about 3 GB, a small fraction of which codes for the brain. This compactness is plausible if evolution mainly writes 'Python code' for specific reward functions and bootstrapping rules, not the entire learned model.

Brown says current LLM training uses a 'really dumb' form of reinforcement learning without value functions, which is surprising it works so well. In contrast, parts of the basal ganglia may implement simple model-free RL, while the cortex builds a model-based system.

A key disadvantage of biological brains is they cannot be copied or externally read, unlike digital models. Advantages include energy efficiency, collocation of memory and compute, and hardware co-designed for potential stochastic, sampling-based inference.

Brown states that creating a competent, misaligned agent like a 'paperclip maximizer' likely requires only minimal innate drives for curiosity and exploration, not the full suite of human social instincts. This is an alignment concern.

Brown advocates for massively scaling up neuroscience to get a 'ground truth,' specifically by driving down the cost of connectomics. The Welcome Trust estimated the first mouse brain connectome would cost billions; E11 Bio aims to reduce it to tens of millions.

Brown describes a moonshot idea of 'behavior cloning' or brain-regularized AI, where models are trained not just on labels but also to predict internal brain activity patterns. This could shape representations and improve generalization, but requires scalable brain scanning tech.

On automated theorem proving, Brown says RL from formal verification, as in Lean, will automate the mechanical parts of math. The harder challenge is automating the conceptual creativity of conjecturing interesting new theorems, which might require a loss function for explanatory power.

Grant Sanderson predicted three years ago that AI passing the International Math Olympiad (IMO) would be just another benchmark, not an AGI "aha moment," a prediction that proved true.

AI excels at geometry problems in the IMO, solving them rapidly via brute force, but struggles with combinatorics problems, which require creative, puzzle-like approaches.

Grant Sanderson outlines three ways AI could solve the Riemann Hypothesis: by connecting disparate fields, building new theories, or through extensive, brute-force computation.

The Montgomery-Dyson anecdote illustrates connecting fields, where a number theorist's observations on Riemann zeta function zeros aligned with a physicist's random matrix theory.

Fermat's Last Theorem exemplifies theory building; its simple statement required centuries of complex mathematical machinery like elliptic curves and modular forms for a solution.

Dwarkesh suggests future AI benchmarks involve generating interesting problems and creating new definitions that unify fields, reflecting the hierarchy from theorem-provers to definition-creators.

Evaluating AI's ability to generate conjectures or definitions will be subjective, measured by a "tone shift" among mathematicians, not a clear quantitative score, Grant Sanderson states.

Galois's group theory, which proved the insolvability of the quintic polynomial by radicals, took around 100 years for its profound utility to be fully recognized.

Early mathematical societies rejected Galois's incoherent papers. Liouville and Jordan decades later formalized his abstract ideas into modern group theory.

Grant Sanderson notes that a key human goal in math is understanding, not just proving, exemplified by Timothy Chow's concept of "unsolved expository problems" like the continuum hypothesis.

Grant Sanderson now believes AI will excel at explanation and distillation, potentially surpassing human capabilities in clearly communicating complex ideas, not just proving theorems.

Grant Sanderson anticipates mathematicians' future role will shift towards "curation," guiding humans through a vast landscape of AI-generated ideas, akin to art curators.

Dwarkesh attributes AI's rapid progress in math and coding to "grindability" - the ability to run parallel, deterministic training simulations that quickly resolve credit assignment problems.

Lean formalization, while not directly driving current AI math breakthroughs, offers the unique potential for AIs to endlessly extend formalized math libraries (Mathlib) without human supervision.

Grant Sanderson raises concerns that auto-regressive AI generation, being "a slave to its context," struggles to make unlikely, cross-field connections necessary for significant breakthroughs.

Dwarkesh suggests AI can counter "entropy collapse" by systematically exploring negations and introducing deliberate biases in different agents to foster diverse research paths.

AI struggles with good writing because unlike modular code or math, writing itself *is* the end product, demanding meticulous, non-sloppy quality in every word.

AI's difficulty in building accurate mental models of people, evidenced by its poor performance on space-repetition prompts, suggests a struggle with anticipating human cognitive states.