The Frontier

Andrej Karpathy defines software 1.0 as explicit rules, software 2.0 as learned weights, and software 3.0 as programming via prompting and the LLM context window as a lever over an interpreter.

Karpathy states that OpenClaw's installation exemplifies software 3.0. Instead of a complex bash script, you copy-paste instructions for an agent, which uses its intelligence to adapt to the environment and debug issues.

Karpathy says his MenuGen app, which uses OCR and an image generator to illustrate menus, is rendered obsolete by software 3.0. The raw approach is to give a menu photo to Gemini with NanoBanan and get a directly annotated image.

Karpathy argues LLMs enable new applications, like automated knowledge base creation from documents, which couldn't exist before because there was no code to reframe unstructured data.

Karpathy posits that future computing could invert the current architecture. Neural networks would become the host process, with classical CPUs serving as co-processors for deterministic tasks.

Karpathy's verifiability framework holds that LLMs excel in domains where outputs can be verified, like code and math, because frontier labs use reinforcement learning with verification rewards during training.

Karpathy cites the 'car wash' problem as current jaggedness: state-of-the-art models can refactor a 100k-line codebase but incorrectly advise walking 50 meters to a car wash.

Karpathy notes that GPT-4's chess capability improved significantly from GPT-3.5 not just from scaling, but because a large amount of chess data was added to its pre-training set.

Karpathy distinguishes vibe coding, which raises the floor for all programmers, from agentic engineering, which preserves professional software quality standards while using agents to accelerate development.

Karpathy suggests hiring for agentic engineering should involve a large, practical project like building a secure Twitter clone and then stress-testing it with adversarial agents, not puzzle-solving.

Karpathy argues that as agents handle more implementation, human skills like aesthetic judgment, taste, system design, and oversight become more valuable, not less.

Karpathy describes current infrastructure as built for humans, not agents. His pet peeve is documentation that tells a human what to do instead of providing text to copy-paste directly to an agent.

Karpathy endorses a tweet stating 'you can outsource your thinking but you can't outsource your understanding.' He sees LLM knowledge bases as tools to enhance, not replace, human understanding.