Julien Launay, co-founder and CEO of Adaptive ML and formerly extreme-scale team lead at Hugging Face and an industrial PhD researcher at LightOn, argues that reinforcement learning in post-training is becoming the main stage of LLM development. In his view it is the cheapest route to frontier-level performance on a given task from a smaller open model, and the main way past the limits of human-written data. He ends on a forecast: superintelligent systems within about five years (10 for bears), arriving unevenly. He expects the transition to be messy and what follows to be, probably, the best time ever.
Pre-training and post-training. Pre-training predicts the next token over "nearly every text in existence," plus images and video, on tens or hundreds of thousands of GPUs. The raw result is unwieldy: asked a question, it may produce 10 similar questions. Post-training sharpens it for use, and its big success has been RL, learning from a good or bad signal rather than from demonstrations. The phases are blending. Pre-training shifts from bulk data toward "higher quality" code, maths and chat, and RL is creeping into pre-training. Launay says some recent models, Grok 4 among them, spent nearly as much on post-training as on pre-training, and he expects post-training to dominate.
Kinds of signal. RLHF from human annotators doesn't scale well. Verifiable or execution feedback, from maths answers, code tests mined from GitHub or game scores as in AlphaGo and Atari, scales readily. AI feedback from judge models covers tasks that can't be verified. To skeptics who fear model-generated data collapses, Launay answers that it can be done badly but usually works. He says agreement between human annotators is far lower than assumed and model-human agreement matches it, and that verification is easier than generation. The host reports finding GPT-4 indistinguishable from his own team's expert raters a year earlier.
Why RL generalizes. Launay's analogy is that supervised fine-tuning is like memorizing a worked problem about spinning black holes and failing once the hole carries a charge, while RL lets the student try and be corrected. RL is mostly online: samples come from the current weights, so they stay in the model's distribution and reshape it gradually. SFT drops in a new distribution and hopes it connects to old knowledge. He concedes pre-training generalizes by sheer diversity, and that post-training is now run at larger scale too. PPO and GRPO share these properties and differ mainly in credit assignment: PPO trains a value model to spread a sparse reward across tokens, GRPO averages over several rollouts.
What Adaptive does. Launay says RL's obstacle is engineering. It mixes inference and training, and needs judges (Adaptive runs five to 10 per pipeline) and environments: SQL databases for text-to-SQL, compilers, tools, VMs for computer use. Adaptive's "RLOps" tooling turns a data scientist's Python recipe into a distributed run, starting from open models such as Llama, Qwen or Kimi. It never trains from scratch, because a pre-trained model makes RL "easy mode." He claims RL always yields the best trade-off between cost and performance. The firm, a bit over 18 months old, sells to large enterprises through its sales team, but synthetic data's unexpected strength makes production data optional, so it plans wider access. He claims an 8B model can reach frontier performance on a task mostly with synthetic data. In one client project, about 70 human critiques and rewrites seeded about 80,000 synthetic conversations of roughly 10 turns each, and with exploration nearly 10 million data points. AT&T, a named client, now runs it independently. He stresses that RL work is about defining and measuring success: "if you can measure it, you can optimize it."
Data and compute. Launay sees "a bit of truth" in both Ilya Sutskever's claim that web data is exhausted and the CEOs who see no end to scaling. Data can be cleaned and reordered, but there are only so many human words. In Richard Sutton's terms, the way forward is learning from experience, in billions of virtual environments and eventually real-world experiments. Compute is the bottleneck now, but, having worked on LightOn's photonic chip, he is bearish on new paradigms: an Nvidia chip embodies a decade-long supply chain no rival can quickly reproduce, and GPUs are already specializing for transformers.
Superintelligence. He expects capability to be "spiky." Maths can be solved in a closed box with formal verification, while drug and vaccine development stay bound by experiments. Scaling AI-controlled wet labs is the next bottleneck he names. Some fields may advance more in a year than in all of history while others barely move. That is the nuance he adds to fast-takeoff stories.