Your signal. Your price.
Dwarkesh states frontier models are trained on tens to hundreds of trillions of tokens. Humans see about 200 million tokens from birth to adulthood.
Dwarkesh notes AI robotics requires millions of hours of demonstrations but still fails at complex open-ended tasks, while humans learn robotic operation within hours.
Dwarkesh compares human learning to driving with 20 hours of practice. Tesla and Whimo use three to four orders of magnitude more data.
Dwarkesh counters Karpathi's evolution pre-training argument. He states the human genome is 3 GB with only 1-2% protein coding.
Dwarkesh says scaling laws cannot solve AI's sample inefficiency. Increasing parameters infinitely only reduces required data by a factor of 10.
Dwarkesh argues AI labs can automate white-collar work despite inefficiency. AI can absorb gigawatts of training and amortize skills across billions of sessions.
Dwarkesh predicts demand for human software engineers will increase in 2027 due to AI's complementary role, despite automation expectations.
Dwarkesh outlines AI labs' plan to use automated AI researchers to solve the remaining sample efficiency problem.