Professor G Markets$Trump Melts Down as Allies Reject his Iran Demands
- This post includes paid content that is not available publicly. See the original source.
AI
Professor G Markets$WATCH: Why the Pentagon Is Hiring Wall Street Bankers
- This post includes paid content that is not available publicly. See the original source.
Dr. Jon KrohnUnmetered Intelligence is Heralding the Next Renaissance (Ep. 975 with Zack Kass)
synopsis
Zack Kass, OpenAI's head of go-to-market when ChatGPT launched and author of The Next Renaissance, ends up holding that cheap, abundant "unmetered intelligence" will commoditise expertise and force people to compete on human qualities such as compassion, courage and wisdom. He thinks that change is welcome, and that the main obstacle is a dread fed by media and devices rather than by the technology itself.
Why "Renaissance". Kass wanted to call the book Unmetered Intelligence, the idea that intelligence will feel like a resource. His publisher judged the phrase too hard to grasp. He uses the Renaissance to disarm cynics: people living in the late Middle Ages, after losing half the population, could reasonably have expected extinction, yet the period produced the sovereign individual. He argues models are getting cheap fast enough to run locally, citing Kimi K2 on his Mac Studio, so diffusion will not be gated by data-centre owners. He concedes this empowers bad actors too. The host contributes his own figures: capability at about 1/200 of the cost of two years ago, and AI task length doubling every seven months.
Dread and information. Both blame ad-funded media. Kass adds that the internet delivered unmetered information without the wisdom to digest it, likening the brain to a GPT-2 model querying a giant database. Children now learn of every atrocity, he says, leaving them "suicidally empathetic". He notes the printing press was suppressed by the church for a long period and literacy lagged until the 19th century. The host puts illiteracy 150 years ago at about 98%. Kass argues that removing harmful friction has also removed useful friction, and that the freedom to do nothing marks an affluent society.
The K-curve and schooling. On Kass's account Gen Z is the first generation in many not to outsmart its predecessor, yet it will have nearly a standard deviation more geniuses, with the developing world trending up. He blames screen addiction dating from 2012 and COVID, not AI, and cites Jonathan Haidt. Agency, not money, now sorts outcomes. He wants education to teach the why and how, citing Eva Moskowitz's accountability-driven Success Academy and McKenzie Price's Alpha School, where students spend two hours a day on self-guided screen learning. He notes Alpha's higher attrition. From Rudolf Steiner he takes the idea that childhood is a chance to know yourself without economic incentive. Teachers should inspire rather than be the smartest person in the room, and he frames the next renaissance partly as a spiritual awakening.
Thriving. His four principles are go outside, learn how to learn, be human, and lead with optimism. Happiness and standard of living tracked each other until around 2010. Japan and Korea are now among the wealthiest and least happy nations, a fact he links to device addiction. People who insist they will work forever, he finds, will not show him their screen time. Wealth managers and real estate agents, once hired for alpha, are now hired for accessibility, trust and pleasant company. The pivotal story is his father, an oncologist for 40 years, during which breast cancer survival went from 35% to 90%. A patient who received identical prognoses from three oncologists said bedside manner had become "the product". His 77-year-old father said he has never been a better doctor. Kass urges seeking out and sharing good news, saying that optimism is not a guarantee but the only route there.
Inside OpenAI. Kass joined when it had fewer than 100 people. GPT-3 went largely unused. ChatGPT ran on GPT-3.5 research 7 months old, a "magic AOL Instant Messenger", and GPT-4 then landed on an existing user base. His lesson is that AI is infrastructure and the chat app is one light bulb.
- Cheap, local "unmetered intelligence" will widen individual agency, as the Renaissance did. Kass, OpenAI's head of go-to-market at ChatGPT's launch, argues that models are getting cheap as well as good, so access will not be controlled by data-centre owners. He cites running Kimi K2 on his own Mac Studio. He concedes the same diffusion empowers bad actors, as the Renaissance also brought more catastrophic war.
- Today's dread comes from unmetered information without the wisdom to process it. Kass blames ad-funded media and an internet that taught us everything at once, leaving young people "suicidally empathetic" and without perspective on how good things are relative to their great-grandparents. The host adds his own figures: the same AI capability costs about 1/200 of what it did two years ago, and the length of task AI can handle doubles every seven months.
- AI is splitting achievement into a K-curve driven by agency, not wealth. Kass says Gen Z appears to be the first generation in many not to be smarter than the last, yet will have nearly a standard deviation more geniuses, with the developing world trending up. He dates the decline to screen addiction from 2012, not AI. He wants schools to teach the why and how rather than the what, citing Success Academy's accountability model and Alpha School's two hours a day of self-guided screen learning, which he notes has higher attrition.
- When machines out-think experts, human qualities become the product. His father, an oncologist for 40 years during which breast cancer survival rose from 35% to 90%, was praised at an award ceremony by a patient who got the same prognosis from three oncologists and said "bedside manner is no longer a feature, it's the product". Kass sees the same shift in wealth managers and real estate agents, now chosen for accessibility and good company rather than alpha. His four principles are go outside, learn how to learn, be human, and lead with optimism.
- The chatbot breakthrough was application, not research. Kass says ChatGPT ran on GPT-3.5 research that was already 7 months old, after GPT-3 had gone largely unused. He argues AI is infrastructure like electricity, and the chat app is one light bulb it powers.
Professor G Markets$Will China Decide the Future of the Persian Gulf? — with John Sfakianakis
- This post includes paid content that is not available publicly. See the original source.
Professor G Markets$Scott on Losing His Dad, Family Traditions, and Handling Criticism
- This post includes paid content that is not available publicly. See the original source.
Professor G Markets$Meet the Man Who Could Flip Texas — with Rep. James Talarico (Live from SXSW)
- This post includes paid content that is not available publicly. See the original source.
Professor G Markets$War With Iran Is Rewriting Global Markets
- This post includes paid content that is not available publicly. See the original source.
Dr. Jon KrohnWhere AI Hardware Is Heading Next (Chris Fregly)
- The LPU deal answers the GPU's own memory-bandwidth limit. Fregly, an investor in Groq, which the host says NVIDIA acquired, describes the deal as a "weird acquisition, licensing deal". Groq's LPU keeps memory on the same chip as the accelerator, which suits the bandwidth-bound decode step of inference. He says NVIDIA's bandwidth limitation dates back to its graphics architecture, and the deal spared it a five- or six-year chip cycle.
- The returns are in secondary markets, backing founders who can raise money. Fregly says he holds secondary shares in private firms such as xAI, which the host says SpaceX had just acquired at the time of recording, because the IPO dry spell leaves public stocks like Apple "behind the times". He names Databricks as his best investment so far and judges founders chiefly on their ability to fund-raise.
Dr. Jon KrohnIf You're Still Writing Code Manually in 2026, You're Behind (Chris Fregly)
- Writing code by hand now leaves you "way behind". Fregly vowed to use only coding assistants in 2026 and says he can fire off 10 to 15 features or bug hunts in parallel, including separate kernel experiments on each GPU of a four-GPU box. He prefers OpenAI's Codex for this work and Claude for UI, and calls Codex his "dirty secret" for avoiding congestion on Anthropic's inference.
- Develop on the GPU you will deploy on. Fregly says MacBooks and consumer RTX cards have completely different memory bandwidth and profiles from Blackwell or Hopper. He found so many components disabled on NVIDIA's DGX Spark that none of his benchmarks ran.
- Replace line-by-line review with always-on evals and correctness checks. He has stopped reading agent-written code and even dropped unit tests, using known-correct examples judged continuously by an LLM instead. Because models reward-hack kernel speed, for instance by zeroing out results, startups' monthly "50x speedup" claims routinely fail his harness, often through misused CUDA streams. He says reading the code would not catch this; only running it with profiling does.
Professor G Markets$REPLAY: Raging Moderates Live at SXSW with James Talarico
- This post includes paid content that is not available publicly. See the original source.
Dr. Jon KrohnHow Inference Actually Works in Large Language Models (with Chris Fregly)
- Inference is two phases with opposite hardware needs. Fregly explains that prefill builds the KV cache over the whole context and is compute-heavy, while decode generates tokens by repeatedly moving weights between GPU RAM, CPU RAM or disk and is bound by memory bandwidth. Disaggregated prefill/decode therefore runs each phase on differently configured nodes.
- Prefill/decode and the KV cache are this year's interview topics. Fregly says he would "guarantee" that 2026 engineering interviews at the big labs turn on these concepts. He notes that his book's coverage grew from a planned half chapter to three or four.
Professor G Markets$Scott Responds to the Paternity Leave Backlash, Patriotism in a Divided America, and What It’s Like Working With Kara Swisher
- This post includes paid content that is not available publicly. See the original source.
Professor G Markets$You Think You're Diversified, AI Disagrees — with Torsten Slok
- This post includes paid content that is not available publicly. See the original source.
Dr. Jon KrohnWhen Will The AI Bubble Burst? How Bad Will It Be? (Ep. 974 with Jon Krohn)
- The host judges AI firmly in bubble territory. His examples include Cluely, the "cheat on everything" startup whose founder Roy Lee raised over $5 million in seed and a $15 million Series A from Andreessen Horowitz, then admitted his touted $7 million ARR was inaccurate. He also cites OpenAI's roughly $1.4 trillion infrastructure commitment over 8 years, about 1.2% of global output, since tempered to around $600 billion by 2030 against $13 billion in revenue last year, and reported nine-figure signing bonuses at Meta.
- Bubbles are productive because complementary investors coordinate on a shared belief. The host relays the argument Byrne Hobart made in The Economist, and in his co-authored book Boom: Bubbles and the End of Stagnation. High asset prices persuade each layer, from turbines and grids to chips, models, products and the firms planning around AI, to invest in the others, which would otherwise risk becoming stranded assets.
- Past busts ruined investors but left cheap infrastructure behind. The host recounts that the 1990s fibre build-out bankrupted WorldCom and Global Crossing, yet by 2004 had cut bandwidth costs by more than 90% while about 85% of US capacity sat unused, enabling YouTube, Netflix and cloud computing. Britain's 1840s railway mania consumed nearly 7% of national income at its peak. His inference is that an AI bust would leave cheap compute.
- Bubble warnings often come years before the peak. Hobart notes the Nasdaq's 2002 post-crash low was still 40% above its level in mid-1995, when the press already called dot-com trading nutty. Case-Shiller prices a decade on stood 18% above the level when a June 2001 essay warned of a housing collapse. The host's inference is that you might still be better off owning AI stocks, while noting he is not a financial adviser.
- The host advises practitioners to prepare for a shakeout. He expects the first job losses at firms building nothing real, then belt-tightening at legitimate ones, as in the dot-com bust. His advice is to deepen transferable skills (training, fine-tuning, deployment, evaluation, tying work to business value) rather than single-vendor API wrappers, build a cash cushion, and invest in reputation.
Professor G Markets$MAGA Erupts At Trump Over Iran War As Chaos Hits White House
- This post includes paid content that is not available publicly. See the original source.
Professor G Markets$Peter Zeihan on How the War With Iran Could Reshape the Global Economy
- This post includes paid content that is not available publicly. See the original source.
Professor G Markets$Trump Voters Love this Democratic Governor — with Gov. Andy Beshear
- This post includes paid content that is not available publicly. See the original source.
Professor G Markets$What $4 Gas Would Do to the Economy
- This post includes paid content that is not available publicly. See the original source.
Dr. Jon KrohnTop Ways to Optimize AI System Performance (with Chris Fregly)
- Benchmark changes methodically, with locked GPU clocks and a before-and-after harness. Fregly's harness fails any comparison that is not apples to apples. He also feeds his own thousand-page book into a long-context model so coding agents can work through its 175-item checklist, because models lack hardware-specific training data. He says he knows labs are now training on such data for AMD and kernel optimisation.
- Stay on the latest drivers and toolkits, and tune the OS for GPUs. Fregly says staying current is critical despite PyTorch sometimes lagging CUDA 13.1. Stock Linux settings such as swap are tuned for CPUs and should be changed, and memory fragmentation watched. He advises using whole GPUs on bare metal rather than sharing or virtualising them.
- I/O is the biggest killer and the lowest-hanging fruit. Use the fastest local disks rather than network file systems, and GPU Direct Storage to load straight into the GPU. Across nodes, use direct GPU-to-GPU communication over NVLink.
- Pick neoclouds built on the GPU maker's full reference architecture. Fregly argues incumbent clouds patch around NVIDIA's design with legacy networking, while neoclouds such as CoreWeave adopt it whole and, he posits, are favoured by NVIDIA. He calls the Mellanox purchase the point at which NVIDIA became an AI systems company, with network hardware that can itself compute.
Professor G Markets$The White House’s Wildest Iran War Spin with Jordan Klepper
- This post includes paid content that is not available publicly. See the original source.
Professor G Markets$Pricing the Iran War's Future — Are Markets Right?
- This post includes paid content that is not available publicly. See the original source.
Dr. Jon KrohnWhy AI Engineers Need to Understand GPU Hardware (with Chris Fregly)
- Low-cost training came from co-design under export limits, not breakthroughs. Fregly says DeepSeek found an under-documented way to bypass a cache on its restricted chips, tuned algorithms and software to its hardware, and wrote its own storage layer. He relays that OpenAI said it had found the same tricks, which labs keep as secret sauce.
- Memory bandwidth is the single most critical GPU characteristic. Loading the weights of 500-billion- to trillion-parameter models is "100% dictated" by memory bandwidth, so Fregly tells teams to track it across generations rather than FLOPs or tensor cores, and to overlap data loading with compute.
- Engineers should build "mechanical sympathy" for the whole stack. The term comes from Martin Thompson's comparison with racing driver Jackie Stewart, who could also build his cars. Fregly advises CS students not to stop at software, because algorithms have changed little while hardware and bandwidth keep changing. He explains that his book grew to three books in one, covering hardware and OS, CUDA in about eight chapters, and inference, which the host puts at 99% of a GPU's lifetime workload.
- The standard profiler misses dozens of critical GPU metrics. Fregly says the PyTorch profiler omits 50 to 60 of them, pointing to streaming-multiprocessor occupancy and separate instruction pipelines for tensor cores, special function units for softmax, and memory movement, all of which must be kept busy at once. NVIDIA's own tools are "cryptic" and its UI "janky", which is why he advises a startup building runtime GPU observability.
Professor G Markets$Trump’s Iran Mission: Unaccomplished
- This post includes paid content that is not available publicly. See the original source.
Professor G Markets$Iran War Explodes Oil Prices — How Will the War Inflation Impact China?
- This post includes paid content that is not available publicly. See the original source.
Dr. Jon KrohnAI Systems Performance Engineering (Ep. 973 with Chris Fregly)
synopsis
Chris Fregly, a former principal engineer at AWS, Databricks and Netflix, argues that AI performance now comes from understanding the whole stack at once: hardware, operating system, CUDA, PyTorch and the model. The binding constraint, in his view, is memory bandwidth rather than compute. He has also stopped reading the code his coding agents write, relying instead on evals and correctness harnesses run on the real target hardware.
Why a thousand pages. His third O'Reilly book, AI Systems Performance Engineering, was proposed at 300 pages, mostly PyTorch with one CUDA chapter. It grew to about a thousand because NVIDIA's documentation is "a total disaster" and its forums leave threads unanswered, so practitioners on X were his best source. He describes it as three books in one: hardware and OS, CUDA (now about eight chapters), and inference. The companion GitHub repo has about 700 examples.
Lessons from DeepSeek. The host puts DeepSeek R1's training cost at under $6 million against hundreds of millions for OpenAI's o1 and o3. Fregly attributes the gap to co-design under export restrictions: an under-documented way around a cache, tuned algorithms and software, and a custom storage layer. His central point is that memory bandwidth, not FLOPs or tensor cores, decides how fast weights reach the GPU, so loading must overlap with compute. He borrows "mechanical sympathy" from Martin Thompson's analogy to racing driver Jackie Stewart, and advises CS students to go below the software layer.
Seeing the hardware. The PyTorch profiler is Python-friendly but omits 50 to 60 critical metrics, among them SM occupancy and separate pipelines for tensor cores, softmax special function units and memory. NVIDIA's own tools are cryptic, and he advises a startup building runtime GPU observability. His workflow is to shell into the B200 itself and have coding agents with his MCP tools profile and explain where time goes. He locks clocks because GPUs ship effectively overclocked and throttle.
The checklist. Of 175 items he picks four. Use methodical before-and-after harnesses. Stay on the latest CUDA and drivers despite PyTorch lag. Tune Linux for GPUs, including disabling swap and watching memory fragmentation, and prefer bare metal and whole GPUs. Treat I/O as "the number one killer", using local fast disks, GPU Direct Storage and NVLink. He favours neoclouds that adopted NVIDIA's reference architecture over incumbents patching legacy networking, and calls the Mellanox purchase the point at which NVIDIA became an AI systems company.
Inference. About 99% of a GPU's life is inference, as the host puts it and Fregly agrees, and inference is far from a simple forward pass. Prefill builds the KV cache and is compute-bound, while decode moves weights and is bandwidth-bound, so disaggregated prefill/decode clusters configure nodes differently. He predicts this and KV caching will dominate 2026 lab interviews.
Coding agents. Fregly vowed to write no code by hand in 2026. He runs 10 to 15 parallel tasks, prefers Codex, and has dropped unit tests for always-on evals judged by an LLM. He warns that models reward-hack kernels, so claimed 50x speedups often break in his harness. He says to develop on the deployment GPU, since consumer RTX cards and the DGX Spark behave too differently.
Investing. As an investor in xAI, which the host says SpaceX had just acquired, and in Groq, which NVIDIA bought through what Fregly calls a licensing-style deal, he says secondary markets are where money is being made. He reads Groq's on-chip memory as complementing NVIDIA's bandwidth limit, but worries that transformer-specific chips could be stranded.
- Memory bandwidth, not FLOPs, is the GPU metric that matters most. Fregly, author of the roughly thousand-page O'Reilly book AI Systems Performance Engineering, says that loading the weights of 500-billion- or trillion-parameter models is dictated entirely by memory bandwidth. Teams should track it across generations and overlap data loading with compute. He credits DeepSeek's low training cost, which the host puts at under $6 million against hundreds of millions for comparable OpenAI models, to co-designing algorithms, software, hardware and even its own storage layer under export restrictions.
- The PyTorch profiler shows only the tip of the iceberg. Fregly says there are 50 to 60 more critical metrics beneath it, among them streaming-multiprocessor occupancy and separate instruction pipelines for tensor cores, special function units for softmax, and memory movement. He calls "mechanical sympathy" across the whole stack the skill CS students should build, since algorithms have changed little while hardware keeps changing.
- Most gains come from the operating system, I/O and network, not kernels. Of his book's 175-item checklist, he highlights locking GPU clocks for consistent before-and-after benchmarks, staying on the latest CUDA and driver, disabling swap, and running on bare metal with whole GPUs. He calls I/O "the number one killer": use fast local disks rather than network file systems, plus GPU Direct Storage and NVLink. He prefers neoclouds that have adopted NVIDIA's full reference architecture.
- Prefill and decode need differently configured hardware. Prefill builds the KV cache and is compute-heavy, while decode is bound by memory bandwidth because it keeps moving weights. Hence disaggregated prefill/decode, which Fregly says is the topic to study for 2026 interviews at the big labs. He reads NVIDIA's deal with Groq as a hedge on that bandwidth limit, since Groq's LPU keeps memory on-chip, and notes that specialised chips risk being stranded if architectures move beyond transformers.
- Continuous evals and correctness harnesses should replace line-by-line code review. Fregly says anyone still writing code manually in 2026 is "way behind" and runs 10 to 15 agent tasks in parallel, preferring OpenAI's Codex for this work. He also says to develop on the target GPU, not a laptop or a DGX Spark. Because models reward-hack kernel speed, he says, startups' claimed "50x" speedups routinely fail his harness, for example through misused CUDA streams.
Professor G Markets$The Iran War’s Oil Shock: How Bad Could It Get?
- This post includes paid content that is not available publicly. See the original source.
Professor G Markets$Does Wealth Make You Selfish?, How Scott Stays Informed, and Negotiating Equity Deals
- This post includes paid content that is not available publicly. See the original source.
Professor G Markets$Trump Downplays War Costs as Oil Explodes and Markets Tank
- This post includes paid content that is not available publicly. See the original source.
Dr. Jon KrohnDo Enterprises Really Need Slow-Thinking AI Models? (with Lin Qiao)
- Real-time applications cannot afford slow-thinking models. Qiao, CEO of Fireworks AI, splits applications into real-time ones (interactive products, fraud detection) that must react, and offline agents that can work for hours, such as a legal assistant reporting back on similar cases. Only the offline class needs reasoning models.
- Customise real-time models small, then big, then small. Start with a small model to test whether your data moves quality in the right direction, since "garbage in, garbage out" applies. Then tune the largest model, often a trillion-parameter MoE, on the cleaned data, and distil it into the smallest model that keeps the quality for fast responses.
- Offline agents need top-tier intelligence, long context and domain tuning. Qiao says deep-research agents in law, finance or software engineering need the slow-thinking mode and much more context. Developers are now specialising them by tuning on thinking tokens, because legal work, investor pitches and customer service each follow different processes.
Professor G Markets$The Iran War Risk Markets Are Ignoring
- This post includes paid content that is not available publicly. See the original source.
Dr. Jon KrohnWhy Open Models Make Economic Sense for Startups (with Lin Qiao)
- AI products can hit product-market fit and still "scale into bankruptcy". Qiao, CEO of Fireworks AI, says inference costs can exceed revenue, which forces startups and incumbents alike to keep users on waiting lists. Open models address both control and cost: a provider like Fireworks has no billions of pre-training cost to amortise and competes only on post-training and inference, so its unit economics differ from frontier labs'.
- Enterprises lag on open models out of unfamiliarity, not underestimation. Qiao says public companies' legal teams worry about licences and whether a model "sends data to other countries", along with guardrails on generated content. She sees them opening up as they learn a US-hosted model is "just a model", and reports only an upward trend in adoption.
Dr. Jon KrohnLLM Selection Has Become Exhausting (with Lin Qiao)
- Model choice matters, but fast model depreciation makes it exhausting. Qiao, CEO of Fireworks AI, says a new leaderboard leader appears every couple of weeks, labs are diverging into chat, coding and agentic, multimodal and long-context specialties, and public benchmarks are saturated. Choosing therefore has to be continuous as use cases evolve.
- Hardware now depreciates just as fast. Where a new hardware SKU used to arrive every 3 years, Qiao says NVIDIA launched three new SKUs last year, and 2026 will bring many more GPUs and custom ASICs from other vendors. Mapping workloads to model, hardware, inference setup and fleet design needs scarce expertise, which Fireworks aims to abstract away.
- Open-model performance is converging with closed models. Qiao says Fireworks bet early on open models after internal debate, out of the open-science conviction of a team that built PyTorch. She predicts 2026 is the year the convergence becomes more prominent.
Professor G Markets$Trump Admits More Americans May Die in Iran War — Voters Are Turning on Him
- This post includes paid content that is not available publicly. See the original source.
Professor G Markets$Scott’s Early Career Advice, How Dating Apps Are Making You Lonelier, and Navigating Conflict With Business Partners
- This post includes paid content that is not available publicly. See the original source.
Professor G Markets$The $2 Trillion Risk Investors Can’t See — with Steve Eisman
- This post includes paid content that is not available publicly. See the original source.
Dr. Jon Krohn972: In Case You Missed It in February 2026 — with Jon Krohn @JonKrohnLearns
synopsis
The host's February 2026 compilation runs four clips from earlier interviews. Taken together, they say that today's AI gains come from engineering at scale and reinforcement learning, while human-like generalisation and fully autonomous operation remain out of reach.
Falcon on Lightning AI. Will Falcon says he never meant to start a company. PyTorch Lightning spread, VCs came calling, and about 10 term sheets arrived after one trip. He describes his FAIR work as solo pre-training on up to 8,000 GPUs and, more distinctively, as writing the loss function to model the world.
Griffiths on human constraints. Tom Griffiths frames human intelligence as adapted to short lives, limited neural compute and narrow communication. AI faces none of these, so it will diverge unless designers engineer human-like inductive biases, which he says is much harder than scaling.
Amazon's Nova Act. An Amazon member of technical staff says agents averaging 60% reliability are useless in production. Nova Act targets 90% or more by training with reinforcement learning in "web gyms" that teach it to cope with changing interfaces.
Murugesan on quantum routing. Praveen Murugesan of Samsara sees quantum computing turning vehicle routing from approximation into optimisation, but says self-operating systems are far off.
- An open-source hit dragged its author out of research into a startup. Will Falcon says he wanted to stay in research, where he was pre-training models on up to 8,000 GPUs as a single person, and nearly shut PyTorch Lightning down because users kept pulling him into coding. After one trip to meet VCs he had about 10 term sheets by the Monday. He credits FAIR with teaching him to translate a modelling problem into the loss function itself, which he says most pre-training today, being engineering, no longer does.
- Human intelligence is shaped by constraints that AI systems lack. Tom Griffiths of Princeton points to short lifetimes, a couple of pounds of neurons, and low-bandwidth communication, which force humans to learn from little data, search selectively and coordinate through language and institutions. AI escapes all three, so he expects a different kind of intelligence unless builders deliberately engineer human-like inductive biases. That is far harder than scaling, which is why scaling dominates now.
- Production agents need far more reliability than demos show. An Amazon member of technical staff (episode 963) says an agent reliable 60% of the time "is 0% useful". She says Nova Act, generally available since re:Invent in December, targets 90%+ reliability on browser UI automation. She says this comes from reinforcement learning across hundreds of "web gyms", replicas of form-filling and shopping UIs where agents self-play through thousands of tasks and learn to generalise when buttons move or change labels.
- Quantum computing could move routing from approximation to optimal solutions. Samsara's Praveen Murugesan says vehicle routing with many stops, route segments and real-time traffic is solved today only by approximation. He sees quantum as the opportunity for optimal answers, but says truly self-operating systems are nowhere close because compute and real-time data access still limit them.
Professor G Markets$Is AI a Threat to Privacy? — with Meredith Whittaker
- This post includes paid content that is not available publicly. See the original source.
Professor G Markets$Kristi Noem Fired as Trump's Iran War is about to Hit Americans Hard
- This post includes paid content that is not available publicly. See the original source.
Professor G Markets$$500M Bet On The Iran Strike — Before It Happened
- This post includes paid content that is not available publicly. See the original source.
Dr. Jon KrohnWhy Enterprises Will Build Their Own Specialized AI Models (with Lin Qiao)
- Reinforcement fine-tuning specialises a model, at the cost of breadth. Qiao, CEO of Fireworks AI, compares RL to people learning from positive and negative feedback, and to breeding fruit sweeter and bigger by selecting seeds over generations. A model rewarded this way finds a path to solving certain problems very well but becomes "less specialized in other areas". The host describes Fireworks' product as letting customers beat frontier closed models on specific tasks in under a month.
- A model that keeps learning from your application becomes a moat. Qiao argues that a private model continuously tuned on product data gives what "nobody else built on top of an existing API would have".
- Using product data to improve models mirrors mobile's A/B testing. Mobile widened reach by orders of magnitude, so no single design or team of PMs could serve every cohort, and A/B testing turned product signals into decisions. Qiao says the same logic now applies to the model under the product, a space she calls "a vacuum" with no existing solution.
Professor G Markets$Trump Demands Uprising in Iran — History Says This Ends Badly with Tim Snyder
- This post includes paid content that is not available publicly. See the original source.
Professor G Markets$Why The Iran War Could Reignite Inflation
- This post includes paid content that is not available publicly. See the original source.
Dr. Jon KrohnAutonomous Intelligence vs AGI (with Lin Qiao)
- Most of the world's data never reaches AGI labs' models. Qiao, CEO of Fireworks AI, which the host says has raised over $300 million including a $250 million Series C, says more than 90% of data is private, locked inside applications and enterprises. She calls her complement to AGI "autonomous intelligence": continuous, fully automated customisation of models and inference to each application. She predicts "millions of models, one per application" rather than one model ruling all.
- Coding agents now work like junior engineers, changing how her company hires. Qiao says the company is questioning whether coding interviews still matter and expects them to be replaced by how well candidates use coding agents. She adds that AI frees people at the "high intelligence level" first, while robotics will disrupt physical work.
- Model customisation is an expert craft she wants automated. Today only a few talent-dense companies can continuously adapt models and inference setups. Qiao's aim is to give ordinary application developers that capability without the years of accumulated knowledge.
Professor G Markets$War in Iran Backfires as MAGA Turns on Trump
- This post includes paid content that is not available publicly. See the original source.
Professor G Markets$What Trump’s War With Iran Means for China and Global Oil
- This post includes paid content that is not available publicly. See the original source.
Professor G Markets$War With Iran: Why Oil Didn’t Spike As Expected
- This post includes paid content that is not available publicly. See the original source.
Dr. Jon Krohn90% of AI Intelligence is Locked Away (Here's Why) #971 with Lin Qiao
synopsis
Lin Qiao, CEO of Fireworks AI, ends up arguing that the next jump in AI quality will come from the more than 90% of data that sits inside enterprises and applications, not from bigger general models. Her picture is open models continuously and automatically customised to each product, which she calls "autonomous intelligence". The host says Fireworks has raised over $300 million, including a $250 million Series C.
Autonomous intelligence versus AGI. Qiao calls AGI "a great direction", but says only a small fraction of data reaches foundation models. Customisation must be continuous as products change, and automated, since today only a few talent-dense companies can do it. She says coding agents now behave like junior engineers, enough that Fireworks is rethinking coding interviews.
Reinforcement fine-tuning. She likens RL to breeding sweeter fruit over generations. A rewarded model specialises at the cost of breadth, and a constantly learning private model gives a product a moat API users lack. She compares this to mobile forcing A/B testing: product signals should now improve the model.
Churn. Model selection is "exhausting". Leaderboard leaders change every couple of weeks, labs are specialising, and benchmarks are saturated. NVIDIA launched three SKUs last year against one every 3 years before. Fireworks bet early on open models out of its team's PyTorch background, and she expects convergence with closed models to become prominent in 2026.
Who adopts open models. AI-native startups adopt readily because products can "scale into bankruptcy" on inference costs, and Fireworks has no pre-training cost to amortise. Enterprises worry about licences and data leaving the country but are opening up. Fireworks also pools demand across customers, sparing them capacity guesses, much as cloud replaced caged mainframes.
Latency classes. Real-time uses cannot afford reasoning. Her recipe there is "small, big, small": test data quality on a small model, tune the largest MoE, then distil. Offline research agents need slow thinking, long context and tuning on thinking tokens.
Evals and optimisation. Fireworks partners for evals. Because agent builders start with "vibe testing", it open-sourced Eval Protocol to link any eval system to any tuner. Customers want quality, speed and cost together, which turns configuration into a search over more than 100,000 options, like a database query optimiser.
- Most intelligence sits in private data that foundation models never see. Qiao, CEO of Fireworks AI and formerly a senior director of engineering at Meta who worked on PyTorch, says more than 90% of the world's data is not on the public internet or in labelling pipelines. Her bet, which she calls "autonomous intelligence", is continuous, automated customisation of open models on each application's data. She expects "millions of models, one per application" rather than one model ruling all.
- Reinforcement fine-tuning lets customers build a moat no API user has. Qiao likens it to breeding sweeter fruit by selecting seeds over generations: the model is rewarded and penalised until it specialises, at the cost of being weaker elsewhere. She compares the shift to mobile's move to A/B testing, using a product's own signals to improve the model. She calls the space "a vacuum" with no existing solution.
- Open models win on control and unit economics. In Qiao's account, AI-native startups adopt open models readily, because products that reach product-market fit can "scale into bankruptcy" on inference costs. Fireworks carries no pre-training cost to amortise. Enterprises have been slower over licensing and data-residency worries but are opening up, and she predicts 2026 is the year open and closed model performance visibly converges.
- Real-time applications should go small, big, then small again. Start with a small model to check that data quality moves training in the right direction, tune the largest mixture-of-experts model on clean data, then distil to the smallest model that holds quality, because real-time use "cannot afford to think". Offline agents for legal or finance research, by contrast, need slow-thinking models with long context, tuned on their thinking tokens.
- Rapid model and hardware churn makes stack selection a search problem. Qiao says leaderboard leaders change every few weeks and NVIDIA launched three new SKUs last year, against one every 3 years before. Fireworks frames tuning quality, speed and cost as a search over more than 100,000 configurations, like a database query optimiser. It has open-sourced Eval Protocol to standardise how evaluation and tuning systems connect.
Professor G Markets$Iran: Regime Change or Forever War? — with Fareed Zakaria
- This post includes paid content that is not available publicly. See the original source.
Professor G Markets$Speaking Up at Work, Scott’s Guide to Fundraising, and The Case for Atheism
- This post includes paid content that is not available publicly. See the original source.
Professor G Markets$Trump Loses Control as War in Iran Escalates — with Admiral James Stavridis
- This post includes paid content that is not available publicly. See the original source.
Professor G Markets$Betting Against the AI Apocalypse
- This post includes paid content that is not available publicly. See the original source.
Dr. Jon KrohnThe Limits of Creativity in Large Language Models (with Tom Griffiths)
- LLMs can produce things absent from their training data, by composition. Griffiths, of Princeton, pushes back on the host's premise that LLMs cannot go beyond their training data. He cites Princeton colleagues' demonstration that models combine given properties into genuinely new outputs, a "compositional generalization" that works up to some depth. Whether they can produce something not recognisably composed of known parts, such as a new artistic style, he calls an open question.
- LLMs edge out people on novelty, but analogy prompts don't help them. In unpublished results from a postdoc in his lab, people rated LLM-generated product ideas slightly more novel than human ones. Prompting with a distant analogy, such as a toaster built on octopus-sucker principles, boosts human creativity but not LLMs'. His suggested reason is that models already draw on a broad enough set of concepts.
- Training on high-probability text makes LLMs conservative. Griffiths says he has yet to see a model produce a recognisable breakthrough when asked for study ideas. Because models are trained to produce high-probability token sequences, "completely out of the box" ideas may be inconsistent with how they are trained.
Professor G Markets$A Big Announcement from Scott and Jessica
- This post includes paid content that is not available publicly. See the original source.
Professor G Markets$Is Cohere the Next AI Powerhouse?
- This post includes paid content that is not available publicly. See the original source.
Dr. Jon KrohnShould AI Be Designed Like Human Intelligence? (with Tom Griffiths)
- Human constraints define human intelligence, and AI does not share them. Griffiths, professor of computer science and psychology at Princeton, lists the limits: a lifespan of a few decades, a couple of pounds of neurons, and communication by "honking noises" or typing. These force humans to learn from little data, unlike AlphaGo trained on many lifetimes of games, and to search selectively, unlike Deep Blue's 100,000 positions per second. They also force humans to pool knowledge through conventions like writing, science and companies.
- Unconstrained AI will become a different kind of intelligence. Because AI systems can learn from many lifetimes, add compute and share state, Griffiths expects them to solve problems well, sometimes better than humans, but not in ways humans recognise. Builders can accept that divergence or deliberately import human attributes so that AI solutions make more sense to us.
- Scaling wins today because engineering inductive bias is hard. Meta-learning is one route to human-like inductive biases, Griffiths says, but it requires understanding the problems being solved and what reasonable biases for them are. Scaling, by contrast, is a solved engineering problem, which he says explains the current moment in AI.
Dr. Jon KrohnWhy Probability Replaced Logic in AI (with Tom Griffiths)
- Symbolic logic stalled because most cognition is inductive, not deductive. Griffiths splits problems into deductive ones, where the given information fixes the answer (2 + 3, x + 2 = 5, algebra, chess), and inductive ones, where data constrain but never settle it: inferring 3-D structure from light on the retina, interpreting noisy speech, guessing someone's mental state from behaviour, learning a language. Rule-based systems excelled at the first kind and "hit the wall" on the second.
- Probability theory is the tool for reasoning without certainty. It assigns degrees of belief across possible worlds and reallocates them as data arrive. Griffiths states Bayes' rule, named for the 18th-century minister Thomas Bayes: multiply the prior by the probability of the data under each hypothesis, then normalise by the sum.
- Strong evidence can overturn a tiny prior. His example is the orangutan culprit of an early detective story ("The Murders in the Rue Morgue"): a voice in no recognisable language, tufts of orange hair, and a high window hard to climb each prove nothing alone. Because those clues are far likelier for an orangutan than for a human, multiplying them together makes the orangutan the best conclusion even though its prior probability was very small.
AI Analyst LabHow to use Claude Code for data analysis (live walkthrough)
synopsis
A long lightning lesson, led by Shane with both co-founders on the call, arguing that Claude Code with Claude Opus 4.6 can now run most of an analyst's mechanical work end to end: framing, SQL, validation, charts, narrative and deck. The condition is that it is built as a system of agents, skills and stored knowledge that its owner teaches like a new hire. The lesson ends on a method and a career claim. Point the system at your own domain, where you can validate its output in seconds, correct it until its mistakes stop recurring, and spend the time you earn on thinking rather than mechanics. Because only you can validate your own work that fast, Shane argues, only you can automate yourself.
Why now. Shane says the session was moved up about a month after the three co-founders took time off their day jobs to test Opus 4.6 in person and found it well beyond earlier models. He frames the goal against layoff headlines: replace tasks, not people. A poll on what share of a team's questions get answered with validated numbers drew answers from 20% to 80%, none at 100%. He puts the typical split of analysis time at 80% mechanics and 20% thinking. Companies that use AI only to multiply mechanics, or to shrink headcount at the old ratio, he says, are getting it wrong; the aim is 80% thinking. Through the previous year, he says, Claude Code worked like "a really fast intern who needed constant supervision": a three-day question might take one day, but with a person in the loop throughout. The change he claims for Opus 4.6 is that it can carry the whole workflow and recover from errors. It still picks a wrong column or misreads a business rule, but these are the mistakes of a new analyst's first weeks, and they are cheap to document so they do not recur.
How Claude Code differs from chat. Chat has a brain, Shane says, and Claude Code also has hands: it reads files, writes and runs Python and SQL, sees the output and decides what to do next. The central artifact is the CLAUDE.md file, an onboarding doc that tells it who it is and what the data looks like, reread at the start of every session. He prefers Claude Code over alternatives for its subagents, while conceding he has not tested competitors recently. He says the analytical habits come from training it on the team's five-week course repo — dozens of hours of framing, metrics and storytelling material — restructured into agents and skills, and that nothing of this comes out of the box.
The demo. On a synthetic Amazon-like e-commerce dataset of 50k users, 13 tables and a year of data, Shane first asked Claude to summarize the repo and draw its workflow, then to describe the dataset. It volunteered gotchas: use sessions rather than the large events table, and cast float columns to integers before joins. It also suggested next questions. Overall NPS came back at +42.3; Shane had it explain the metric and its sources, a check he says he always runs. Broken down by segment, Plus members scored +54 and free users +30. Asked to explain detractors and validate against other data, it cross-checked the support-tickets table, and it also noted that half the "detractor" comments were positive because the synthetic comments were randomly assigned. A time view found a payment-ticket spike in late May and early June. Asked what to do next, it recommended finding the cause. Parallel subagents checked hypotheses, ruled out an experiment, and found iOS app v2.3, deployed June 1, behind 465 tickets, with 100% of v2.3 payment tickets coming from iOS. A chart-reviewer agent caught an annotation colliding with a subtitle and fixed it, which Shane attributes to building the Storytelling with Data principles into its skills. The deck-creator agent and a presentation-theme skill then produced a deck. It ran from context, through a 24-point structural NPS gap and a fourfold spike from one release, to three actions: a postmortem, automated payment-flow alerting (the incident ran three weeks), and investigation of wider free-tier friction. He also showed an overnight run on why conversion fell four points from Q1 to Q2. It took about 45 minutes and concluded that most of the decline was a mix shift from user-base growth, a Simpson's paradox, not a broken funnel. Shane cautions that he would not pass on every recommendation, and that every number must be validated before anything is sent.
The architecture. The full pipeline frames the question and explores data in parallel, converts questions into hypotheses, checkpoints with the user (skippable on "just do it"), and then runs descriptive analysis, root-cause investigation, validation, opportunity sizing, storyboarding, charting and the deck. Validation covers structural checks, business rules and Simpson's paradox. Failures raise warnings and blockers and loop back rather than producing a wrong answer with a caveat. Questions are routed by level, from a one-number lookup up to an hour-long deep dive, because "not every question deserves a deck". Shane's four transferable patterns are: write the CLAUDE.md for your domain; add skills that enforce standards automatically; log corrections so they are checked before the next query; and make validation a pipeline step. The design principle is to make a non-deterministic system as deterministic as possible, with the model reasoning and code doing every calculation. He builds agents in an "architect mode": Claude spawns several personas that each plan separately, debate, revise and agree a single master plan for him to review. Asked to rebuild itself, the system compressed about 40,000 lines into a 1,500-line markdown "seed", which rebuilt it on a separate machine.
Owning the automation. Shane's central claim is that you are the only person who can validate your own domain's output at speed, so self-automation is feasible and automating someone else's domain is not without their sustained feedback. He compares the ramp to onboarding a junior analyst, corrected heavily in week one and catching errors he had missed within the first week. He says correction rates fall by the third or fourth analysis on a dataset. The gains come from running analyses you were doing anyway, so the time invested is repaid at once. He warns that companies will misuse this by sending people to automate others' domains. The recommended start is a CSV or query with known answers, correcting each error and re-asking.
Q&A. On careers, Shane is openly uncertain. He predicts near-term work building agentic systems, cleaning data foundations and validating, possibly more data-science hiring for validation, and later roles as "solvers", translators and change leaders. He thinks the next six months of leaning in could shape a decade. A co-host likens analytical fundamentals to multiplication tables, and tools such as SQL to driving a stick shift. On stakeholders, Shane treats the system like a calculator: he remains accountable and keeps his agent away from stakeholders, though he had released his work to his organization only that Monday. A third co-founder says MCP servers to BI tools such as Databricks will let these workflows replace dashboards. Practical answers covered markdown checkpoints to avoid full reruns, PPTX output for manual edits, golden datasets and semantic views drafted as requirements for analytics engineers, bootstrapping knowledge from 30 days of query history, and connecting warehouses through a connect-data skill. He also covered security: zero data retention, Snowflake Cortex or SageMaker instead of local files, and demos on non-sensitive data while buy-in is sought. A co-host adds that red tape scales with company size. Shane keeps the system to product analytics and away from modeling and causal inference, which he says demand more human judgment to validate.
- Automate the mechanics to flip analysts' time toward thinking. Shane estimates analysis work at about 80% mechanics (queries, joins, validation, chart formatting) and 20% thinking, and argues the aim is to reverse that in the same hours, not to cut staff. He says the lesson was moved up about a month after testing Claude Opus 4.6, which in his account can hold the whole analytical workflow rather than needing step-by-step handholding. It still strays, but its mistakes are now correctable, like a GPS watch flagging a wrong turn, so the bottleneck shifts from execution to judgment.
- The pattern is an onboarding file, agents, auto-firing skills and corrections. CLAUDE.md is the analyst's onboarding doc, reread every session. Agents are prompt templates with rules, skills are guardrails that fire automatically (date-column checks, zero denominators), every number goes through SQL or Python so "the LLM never does arithmetic", and validation is a pipeline step with four layers that halts on failure. Corrections are stored and checked before the next query. Shane says the team's open-source repo of about 40,000 lines compressed into a 1,500-line markdown "seed" that rebuilt the system from scratch on a separate machine.
- The live demo traced a satisfaction gap to one app release. On a synthetic Amazon-like dataset (50k users, 13 tables, a year of data), Claude Code reported NPS of +42.3, free users at +30 against Plus members at +54, and noted that the detractor comments were templated synthetic data. Cross-checking support tickets found a payment-issue spike; parallel subagents ruled out an experiment and pinned it on iOS v2.3, released June 1, with 465 tickets, all of its payment tickets from iOS, running about three weeks. A reviewer agent fixed an overlapping chart label, and the pipeline built a deck with three recommended actions. A separate overnight run on a four-point Q1-to-Q2 conversion drop took about 45 minutes and found a mix shift (Simpson's paradox) rather than a broken funnel.
- Only you can automate your own job, because you validate fastest. Shane argues you catch a wrong date column in seconds on data you know, while anyone automating someone else's domain needs that person's feedback, which for customer-facing tools means a full eval system with ground truth. He likens setup to onboarding a junior analyst, with corrections dropping off by the third or fourth analysis. His advice is to start with a CSV whose answers you already know, and to stay accountable for every number "just like we're using a calculator", logging its queries to a Notion page.
- Checkpoint outputs, bootstrap context from query history, and keep data hosted. In Q&A, to avoid rerunning everything for a small edit, Shane has each agent write results and queries to markdown checkpoints; a colleague switched output to PPTX for manual edits. He bootstraps context on 50–60 tables from 30 days of his own query history and has Claude draft requirements for semantic views. For security he cites zero-data-retention terms with Anthropic, Snowflake Cortex, or a hosted environment such as SageMaker rather than a laptop. He deliberately limits it to product analytics, not modeling or causal inference, which he says needs more human thought to validate.
Professor G Markets$Raising Young Men, The Case for College in the AI Age, and Relationship Red Flags
- This post includes paid content that is not available publicly. See the original source.
Professor G Markets$Raising Young Men, The Case for College in the AI Age, and Relationship Red Flags
- This post includes paid content that is not available publicly. See the original source.
Dr. Jon KrohnThe “100x Engineer”: How to Be One, But Should You? (Ep. 970 with Jon Krohn)
- Agentic coding crossed a coherence threshold around December. The host relays Andrej Karpathy's late-January post (over 7 million views): within weeks he flipped from about 80% manual coding and 20% agents to 80% agents and 20% manual touch-ups, "mostly programming in English". Karpathy expects similar shifts for well into double-digit percentages of engineers, while public awareness sits in the low single digits.
- Agent errors are now conceptual, not syntactic. Per Karpathy, models make wrong assumptions on the user's behalf and charge ahead, and they over-complicate: a brittle construction of over a thousand lines that shrinks to about 100 when the user asks for the simpler route. Karpathy still calls them a massive net improvement; the host predicts these flaws will improve rapidly over the coming months.
- "100x" output comes from parallel agents and exhaustive specs. The host cites a LinkedIn profile by an AI engineer named Brian Vasquez: over two months Peter Steinberger, who sold the PDF framework now called Nutrient in 2021 for a reported hundred million, made over 6,500 GitHub commits (about 210 a day), adding 2.5 million and removing 1.9 million lines. He runs three to eight agents in a 3x3 terminal grid for 5–15 minutes each, commits straight to main, and dictates ideas that AI turns into a spec, which a fresh context then attacks for 20 weak points until the questions grow niche; specs often exceed 500 lines, and planning-to-execution shifts from about 20/80 to 60/40.
- Leverage comes from declarative specs: tests and success criteria. Karpathy advises writing tests first, or a naive but likely-correct algorithm for the agent to optimise, and says the main gain is doing things that weren't worth the effort before or were beyond his skills. Both men report costs: Karpathy feels his manual coding atrophying, Steinberger calls himself addicted, and Steinberger ships code he never reads, trusting passing tests.
- The host advises a modest multiplier, not lines committed. His opinion is that commit counts are a poor benchmark of quality, so aiming for 2x–10x while personally watching quality, or simply widening the range of problems one can tackle, may be wiser. He notes Anthropic recently became a sponsor but says the episode was planned beforehand, and he raises Karpathy's open questions on whether the gap between average and best engineers grows and whether generalists overtake specialists, against task lengths that frontier LLMs can handle doubling every 7 months.
Professor G Markets$Where the Epstein Scandal Goes After the Clinton Depositions?
- This post includes paid content that is not available publicly. See the original source.
Professor G Markets$Prediction Markets vs. Gambling: Where’s the Line? — with Tarek Mansour
- This post includes paid content that is not available publicly. See the original source.
Professor G Markets$Why a Doomsday AI Blog Wiped Out $300 Billion
- This post includes paid content that is not available publicly. See the original source.
Professor G Markets$China’s AI Is 20x Cheaper — And Catching Up
- This post includes paid content that is not available publicly. See the original source.
Professor G Markets$Will the U.S. Go to War With Iran? — with Karim Sadjadpour
- This post includes paid content that is not available publicly. See the original source.
Dr. Jon KrohnAre We Overestimating Neural Networks? (with Tom Griffiths)
- Each AI paradigm runs until it hits limits nobody foresaw. Griffiths declines the host's framing that the field is making a big mistake today; he describes a normal cycle of going all in on one breakthrough until its fundamental limits show. Neural networks make those limits especially hard to predict, so he looks to history for clues.
- Neural networks are only as good as their training data. He calls them non-parametric in data-science terms: in principle able to learn any function, so what they actually learn is heavily shaped by the data they are trained on, which explains much of LLMs' counterintuitive behaviour.
- Next-token training leaves "embers of autoregression" in model outputs. Responding to the "sparks of AGI" claims around GPT-4, his group's paper Embers of Autoregression showed the model is sensitive to how probable token sequences are. One example: asked to count letters in a sequence, it is more often right when the answer is 30 than when it is 29, because 30 appears more often on the internet.
Professor G Markets$Nvidia’s Blowout Can’t Calm AI Anxiety
- This post includes paid content that is not available publicly. See the original source.
Professor G Markets$Trump’s “Forgettable” Speech Was a Disaster and Dangerous
- This post includes paid content that is not available publicly. See the original source.
Dr. Jon KrohnIs There a Mathematical Theory of the Mind? (with Tom Griffiths)
- Laws of thought complement the laws of nature. Griffiths' book, The Laws of Thought: The Quest for a Mathematical Theory of the Mind, argues that alongside the maths describing the physical world there is a mathematical account of the internal one. He says some of the same thinkers pursued both, but the mind took until the 19th century for first glimmers of a solution and the 20th century to become an empirical science, cognitive science.
- Psychology started with perception because thought was harder to measure. Psychophysics, the host's term, studied how a physical event (a photon on the retina) becomes a subjective experience (a flash of light), and the earliest labs had people report what they experienced under different stimuli. Thought proved harder, and early-20th-century behaviourists discarded internal states entirely to keep the science rigorous, measuring only behaviour and environment.
- Mathematics revived a rigorous science of the mind. Griffiths says logic and the development of computers gave psychologists a precise language for hypotheses about thought and language, which got the field past behaviourism. He places his own work, connecting human minds and computers, on that foundation.
Professor G Markets$Why a Doomsday AI Blog Wiped Out $300 Billion
- This post includes paid content that is not available publicly. See the original source.
Professor G Markets$State of the Union Postgame Analysis — with Jessiah Eberlin of Pondering Politics
- This post includes paid content that is not available publicly. See the original source.
Professor G Markets$Billions in Tariff Refunds — Who Gets the Money?
- This post includes paid content that is not available publicly. See the original source.
Professor G Markets$China Capitalizes as Trump’s Tariffs Backfire
- This post includes paid content that is not available publicly. See the original source.
Dr. Jon KrohnThe Laws of Thought: The Math of Minds and Machines (Ep. 969 with Prof. Tom Griffiths)
synopsis
Tom Griffiths, professor of psychology and computer science at Princeton and director of the Princeton Laboratory for Artificial Intelligence, ends up arguing that minds and machines are governed by shared mathematical "laws of thought" (logic, probability, Bayes' rule) but solve different problems because they work under different constraints. Humans learn from little data with limited compute and bandwidth, which explains both their biases and their efficiency; AI escapes those limits and so may become a different kind of intelligence unless we deliberately engineer human-like inductive biases into it, which he says is far harder than scaling. The occasion is his new book, The Laws of Thought: The Quest for a Mathematical Theory of the Mind, not yet out at recording.
A science of the inner world. Griffiths presents the book's premise: alongside the laws of nature sits a mathematics of mind that took until the 19th and 20th centuries to become cognitive science. Psychology began with perception (psychophysics); behaviourism then dropped internal states, and logic and computers later supplied a rigorous language for hypotheses about thought.
How the lab works. His lab pairs ideal solutions to computational problems with experiments, now mostly online through crowdsourcing services such as Prolific, treating the mind as an intuitive data scientist. One gamble-choice dataset holds more than 10,000 problems, analysed with neural networks. Unlike the Kahneman–Tversky tradition, he looks for the rationality in apparent errors. Bot fraud by LLMs is an arms race; former students founded Roundtable to tackle it, and Griffiths discloses a very small investment.
What training leaves in models. Asked what AI might be overlooking, he declines to call it a mistake and describes cycles of pushing one method to its limits. Neural networks are non-parametric and data-bound; his Embers of Autoregression paper showed GPT-4 counting correctly more often when the answer was 30 than 29. Modern models are attenuated but still sensitive, and now write code to check deterministic answers, an architecture around the base model that he likens to 1980s expert systems.
Probability over logic. Deductive problems suit rules; inductive ones (perception, speech, other minds, language) need probability. His illustration is Dupin's orangutan in The Murders in the Rue Morgue: clues each unlikely for a human combine via Bayes' rule to overwhelm a tiny prior.
The missing priors. Children learn language from about 5 years of speech; LLMs need 5,000 or 50,000, leaving about 4,995 years to inductive bias. Since AlexNet in 2012, machine learning has used weak priors and lots of data, a lesson he warns may soon "bite us". His lab uses meta-learning to set initial weights as a soft bias.
Constraints and the choice ahead. Humans live decades, use a couple of pounds of neurons and communicate slowly; Deep Blue searched 100,000 positions a second. AI is free of all three limits, so it will likely solve problems in unfamiliar ways. One can accept the divergence or import human attributes so systems make more sense to us. Using birds and planes, he says the shared principles, the laws of thought, are what transfer, while engineering such as jet engines does not.
Creativity and the improbable. He challenges the host's claim that LLMs cannot go beyond their data: they generalise compositionally. A postdoc's new results show LLMs slightly outscore people on novel product ideas, though analogy prompts do not help them; genuine breakthroughs are still absent because they favour high-probability output. Procedurally composed prompts can steer them into rare regions.
Curiosity and evaluation. With a former graduate student he modelled curiosity as seeking information that raises future action value, producing an inverted U: things seen a few times attract most interest. Intrinsic rewards that reduce uncertainty reproduce it in AI. He is "not a big fan of benchmarks", calling them engineering devices, and prefers targeted, diagnostic experiments borrowed from cognitive science.
- Human "irrationality" is good problem-solving under tight resource limits. Griffiths says psychologists call people terrible decision-makers while computer scientists find them impressive; he reconciles the two by treating cognition as solving the world's problems well with limited computation. His lab tests ideal solutions against behaviour at scale, for example a gamble-choice dataset of more than 10,000 decision problems gathered online and analysed with neural networks, where earlier theories rested on a couple hundred.
- Next-token training still bends answers toward probable outputs. His group's paper Embers of Autoregression found that on deterministic problems GPT-4 gave probabilistic answers, counting letters correctly more often when the answer was 30 than 29 because 30 is commoner online. He says the effect persists in modern models though attenuated, and that chat models now write code to check such answers, an echo of 1980s expert systems needing an architecture around their if-then rules, as language agents now build one around the base model.
- The child-versus-LLM data gap measures missing priors. A child learns language from about 5 years of speech while an LLM needs something like 5,000 or 50,000 years, so on his arithmetic inductive bias accounts for about 4,995 years of input. Since AlexNet in 2012 machine learning has relied on weak priors plus data, a lesson he thinks may "come back and bite us soon"; his lab uses meta-learning to set initial weights as a soft bias.
- AI faces none of our constraints, so may become alien intelligence. Humans live a few decades, think with a couple of pounds of neurons and communicate through narrow channels, while AI can train on many lifetimes, add compute and share state. Griffiths frames a choice between accepting divergence and engineering human-like inductive bias, which is much harder than scaling; like aerodynamics for birds and planes, the "laws of thought" are what he expects both kinds of intelligence to share.
- LLM creativity is real but conservative by construction. A postdoc's unpublished results show LLMs slightly beating people on product ideas rated novel, while analogy prompts that boost human creativity don't help them; he has seen no genuine breakthrough. Because they produce high-probability sequences, he recommends procedurally generated prompts that combine unlikely elements to push them into improbable regions.
Professor G Markets$Billions in Tariff Refunds — Who Gets the Money?
- This post includes paid content that is not available publicly. See the original source.
Professor G Markets$Is Scott Friends with Tech CEOs?, Who Should Run in 2028, and Overcoming Rejection
- This post includes paid content that is not available publicly. See the original source.
Professor G Markets$Why Big Tech Is Losing to Boring Stocks
- This post includes paid content that is not available publicly. See the original source.