Kyunghyun Cho, co-director of NYU's Global AI Frontier Lab and co-author of the 2014 neural machine translation paper that introduced attention, ends up arguing that the next step change in AI will not come from more data or compute. It will come from algorithms that actively choose what data to collect, built on models already trained broadly enough to know what is worth looking for. He is candid that he cannot say where this leads, and expects the change to feel less dramatic in hindsight than it looks in prospect.
The lab and its people. Cho co-directs the Global AI Frontier Lab with Yann LeCun. It is funded by the Korean government with NYU and hosts researchers from seven Korean institutions for visits of 3 to 12 months, with about 11 NYU principal investigators involved. He contrasts research cultures: isolated grinding in Korea, isolated work-life balance in Finland, and collective, international work in North America, which he calls the US's "superpower". He recalls telling his PhD student William Falcon that a startup and a PhD are each "200% jobs" and to take leave instead. The host puts Lightning AI's ARR at over $500 million.
How attention came about. Cho joined Yoshua Bengio's lab in August 2013, when it had about 35 to 40 people against Mila's roughly 1,500 today, and chose machine translation from Bengio's options despite having no NLP training. Recurrent networks underfitted and would not scale, and a Google team led by Ilya Sutskever executed a similar idea far better on eight GPUs. Cho names three levers for scaling: data, which he leaves to companies; compute; and algorithms that change the slope of the scaling law, the only one he thinks suits universities and one that needs an "aha moment". That moment came from intern Dzmitry Bahdanau, whose attention prototype Cho sensed would work halfway through hearing it, and which ran within three to four days. With a 50,000-word vocabulary and unknown tokens, the model translated "[unknown] Korea" as South Korea beside "friend of the United States" and North Korea beside "enemy". From this Cho concluded the models held knowledge drawn from WMT news and parliamentary data, not just translation skill.
Intelligence as information processing. Revisiting his 2018 remark that brains are biologically implemented computers, Cho says progress is "moving very fast". Problems that look like they need intelligence become tractable when framed as information flow, as with genes and proteins in biology. Current AI has partly found the right abstraction for capturing correlations, but "correlation is always just one side of the coin".
Sample efficiency reframed. The host raises dogs that learn a trick from one example. Cho answers that asking for the same performance from less passive data only makes sense for bad algorithms. Modern transformers already capture most correlations in a given dataset, and low perplexity shows it. The question becomes active: with the same budget, which data should be collected so that rare, important events turn up more often? Doing that requires a world model, which is why RL tuning works only after massive pre-training, and the purposeful data then feeds back into the general pool. He treats protein-binder screening, iterative web search and robotics as one problem, with physical work harder mainly because the physical world is fragile.
World models. Video models such as Sora and Genie bet that high-fidelity imagination yields understanding. Cho's 2025 ICML-workshop best paper with LeCun on latent dynamics, and LeCun's JEPA, bet on high-level abstraction, like planning a Paris trip without imagining the walk to the gate. He calls the choice unclear, since step-by-step prediction suits digital hardware, but says the work must move into latent space.
Teaching and retrieval. In his undergraduate course about 80% of 200 computer-science students had never installed a coding agent, so Cho now builds one app a week with agents and has students follow. He traces RAG-style re-ranking to student Rodrigo Nogueira's 2016 goal of an autonomous researcher, a goal that led to fine-tuned BERT re-ranking. He values models that see across fields no single human can master.