AI

2025-05-08 · open the original ↗

• Polars rescued a pandas pipeline that could not scale. Janssens says Alliander, the Dutch grid operator which runs electricity and gas infrastructure for a third to half of the Netherlands, needed to cut cost and process far more data, but its Python and pandas code was inefficient and ran on a single AWS instance with over 700 GB of RAM. He believes it was one of the first companies with Polars in production, over a year before Polars 1.0.
• Win over a sceptical team with one small benchmark. Janssens says the team was understandably wary of a library promoted by three consultants from Xomnia, where Polars was being developed. The consultants rewrote a piece of low-hanging-fruit pandas code in Polars and showed the speed and memory numbers. The full migration then went piece by piece, idiomatically rather than line by line, until R and pandas could be dropped as dependencies.
• Memory per job fell more than tenfold. Nieuwdorp puts it at about 500 GB down to 40 GB, and attributes the drop partly to Polars and partly to restructuring code along the way.
• Production use and book-writing exposed Polars' gaps. Janssens says real work surfaced missing features, such as weighted random sampling that pandas has, while filling in the book's reference tables revealed naming inconsistencies and missing operators. The authors fed these back to the project as issues and some pull requests.

2025-05-07 · open the original ↗

• Quiet biases are more worrying than obvious ones, being harder to find. Will Landecker, founder of the consultancy Accountable Algorithm, separates clear, measurable harms, such as loans approved at different rates, from "whispers", such as slightly different interest rates that only a large pile of data and a strong hypothesis can reveal. A tiny gap multiplied by 8 billion users is an enormous effect, and finding these gaps takes proactive searching.
• "Good" usually means accurate on the majority, which entrenches gaps. If 90% of applicants are men, an accurate model can still give women worse rates, and Landecker says executives everywhere he has worked could ask "what does the data say?" without anyone scrutinizing it. A portrait generator that worked for him hallucinated the features of a mostly Black nonprofit's members, which he calls an evaluation-set failure as well as a training-data one. Evaluating only on current users locks a product out of new markets, and a host adds that A/B tests should be read by segment.
• Machine-to-machine feedback loops strip out the human checkpoints. Landecker's main agentic concern: when agents interact directly, independent errors may cancel out, but correlated errors push the system in their direction, faster than human-speed review or user reports can catch it. He notes that long-running projects that collect human-written text now say they can no longer guarantee incoming text is human-written.
• Using one LLM to test another for bias is fraught. Subtle bias, such as different responses when pronouns or racially coded names are swapped, needs massive varied datasets, and generating them with an LLM leaves no clear party to blame when a disparity appears. Landecker recalls Word2Vec at Twitter around 2013 completing "man is to woman as doctor is to" with "nurse". He calls the current shotgun of approaches (LLM judges, human red teams, interpretability) passable but not enough.
• Every AI organization should ask how its success metric lies to it. This is Landecker's one recommended habit: enumerate the compromises behind a measurement, such as a cheap, high-coverage data source or a quick evaluation function with no human in it, and ask how they mislead, rather than whether the number went up. He also advises finding an executive sponsor and treating internal policy as the main guardrail, since he is not hopeful about government action. He recommends the FAccT conference proceedings.

2025-05-07 · open the original ↗

• Polars code reads as declarative paragraphs, not nested brackets. The authors say pandas code is full of brackets and often hard to reason about, whereas Polars expressions (in their book's metaphor, recipes whose operations are steps and whose functions are cooks) compose into queries that read like a paragraph. Users declare the end result and leave processing and optimisation to the engine.
• Polars was shaped by pandas' failures and strengths alike. Relaying conversations with creator Ritchie Vink, the authors say a formative frustration was pipelines crashing 20 minutes in on errors that could not be seen up front. Polars kept what Vink liked in pandas and borrowed from Spark, whose syntax it resembles, and from Rust. They stress they are not bashing pandas: without Wes McKinney's library there would be no Polars.

2025-05-06 · open the original ↗

• Polars' declarative style makes pipelines easier to read than pandas. Janssens and Nieuwdorp, authors of O'Reilly's Python Polars: The Definitive Guide (released on the recording date, 1 April 2025), say pandas code is bracket-heavy and hard to reason about, while Polars expressions compose into a query that reads like a paragraph, leaving optimisation to the engine. They credit pandas as Polars' foundation, and relay that creator Ritchie Vink was shaped by pipelines crashing 20 minutes in, borrowing from Spark and Rust.
• A production migration cut memory needs by an order of magnitude. At Alliander, memory per job fell from about 500 GB to 40 GB. The Dutch grid operator's pandas pipeline ran on a single AWS instance with over 700 GB of RAM. The authors won over a sceptical team by benchmarking one small rewrite, then migrated idiomatically until R and pandas were dropped. Nieuwdorp says the same time frame that once fit 25 simulation samples later fit the 50 stakeholders wanted, with code restructuring contributing alongside Polars.
• The GPU engine pays off on modest data sizes. With five Nvidia workstation cards in a Dell lab, both companies being podcast sponsors, the authors ran their own benchmarks for the book's appendix rather than reuse promotional figures. Nieuwdorp says Nvidia's RAPIDS team built an engine that takes Polars' optimised plan instead of mapping the API onto cuDF. Speed-ups appeared from around 1 GB, sooner than promotional material implied, and even on smaller cards.
• uv is fast enough to change how tools get used. Nieuwdorp says uv replaced Poetry and a planned Docker setup for the book's repo, since readers need only uv sync. Its speed allowed throwaway environments per command, which he used to benchmark Polars versions, find a roughly 10% regression around version 1.2-something, and git-bisect it to a commit.
• Format a table's presentation, never its data. Janssens says Great Tables, by his colleagues Rich Iannone and Michael Chow and exposed in Polars via df.style, is the only Python table-styling package he knows. Turning numbers into strings to add dollar signs breaks later calculations. He also recounts rewriting most of the visualisation chapter (16) when Marco Gorelli switched Polars' plotting backend from hvPlot to Altair, a feature then marked unstable.

2025-05-05 · open the original ↗

• AI development will reach many more non-developers. Dell's Logan Lawler says that, not being a developer when hired about a year earlier, he taught himself to pull SDKs from GitHub and train his own LoRAs for an animation studio. He expects that kind of access to become far easier and more common.
• AI will simplify small everyday tasks. Lawler's example is a RAG system over his 70-year-old mother's thousands of unsorted recipe cards that would return her pumpkin pie recipe on request. He notes someone would still have to set it up.
• Agents will coordinate with one another, then move into the physical world. Nvidia's Sama Bali, echoing Jensen Huang's GTC keynote (about five weeks before release, the host notes), describes factory agents passing word of a raw-material shortfall to the floor agent and then the transport agent so fewer trucks are booked. She points to Bay Area driverless cars visibly improving week by week as physical AI, and adds her personal hope for guardrails and regulation.

2025-05-04 · open the original ↗

• NIM microservices exist because models change every few months. Bali says Nvidia packages open, proprietary and its own models as containers with TensorRT-LLM-style inference optimisation, so developers can swap Llama 3 for 3.1 or 3.2 without rebuilding the pipeline; NIM stands for Nvidia inference microservice. Lawler adds that a NIM loads with one line of code and arrives already tuned to the GPU it runs on, so the only check is that the model fits in GPU memory, 96 GB on the Dell Pro Max machine discussed.
• Nvidia AI Enterprise layers training, guardrails and recipes on top. Bali lists NeMo for building, fine-tuning and guardrailing models, and AI Blueprints as reference workflows to which companies add their own data, which she calls their differentiation. She says NIMs can be prototyped free at build.nvidia.com and downloaded free through Nvidia's developer program.
• CUDA libraries accelerate data science without code changes. Bali says CUDA's parallelism can cut training from weeks to days. By her account, RAPIDS cuDF mimics pandas and Polars APIs to speed preprocessing by 100x on RTX 6000 GPUs with one added line, and cuML, announced at GTC, speeds scikit-learn tasks by up to 50x.

2025-05-03 · open the original ↗

• Data-science acceleration comes down to the GPU. Dell's Lawler says what matters is how much fits on the GPU and how fast it runs, with the rest of the Pro Max system in support. He says the T2 tower, launched on the recording day, grew in capacity for extra drives and cards without much bigger footprint, and runs new Intel processors at a sustained 250 W, which he calls unique in the industry.
• Scarce cloud and data-centre GPUs push experimentation onto the desk. Nvidia's Bali says fine-tuning and running models is now mainstream, even for product marketers like her and Lawler, while data-centre resources are scarce. She cites a data scientist who had to queue in a Slack channel for an instance, and argues developers need a local sandbox to keep up with daily new models and techniques.
• Doubled GPU memory is the new generation's biggest feature. For the RTX Pro Blackwell GPUs, Bali calls the jump from 48 GB to 96 GB per GPU, with up to four in one workstation, the biggest feature, and "a game changer" for running, fine-tuning and serving models locally.

2025-05-02 · open the original ↗

• MCP addresses the integration problem holding back agentic AI. The host says LLMs have always struggled to reach data beyond their training, and custom per-source connectors were brittle and could not scale. Anthropic introduced MCP in November 2024, but only in the past couple of months, he says, has it taken off.
• Adoption has been fast because MCP is open and model-agnostic. He cites early adopters Block, Apollo, Replit and Sourcegraph, and over a thousand community-built MCP servers by February. Any model, Claude, GPT-4 or open source, can use it and anyone can build an integration without permission, positioning it as a "USB or HTTP of AI integration".
• Dynamic discovery lets agents find tools without hard-coding. The host explains that agents detect available MCP servers and their capabilities automatically, so a new server for, say, a CRM is usable immediately. Anthropic provides pre-built servers for Google Drive, Slack and databases. He says MCP complements rather than replaces custom APIs, plug-in systems, LangChain-style frameworks and RAG.
• MCP is the action layer, not a silver bullet. He places it as a standard integration layer for the action component of agents. Its open problems are managing many tool servers, getting models to use tools well, an evolving standard, security and monitoring, and overkill for simple apps. He relays Anthropic's plans for remote servers with OAuth, an official registry, standardised discovery endpoints, streaming and proactive server behaviour.

2025-05-01 · open the original ↗

• Dell collapsed its many PC brands into three tiers. Lawler says the rebrand to Dell, Dell Pro and Dell Pro Max replaces many disparate lines (Precision, OptiPlex, XPS and others) to match how customers shop. Dell Pro Max targets heavy GPU workloads from independent software vendors, such as engineering and Adobe workflows, and was announced at CES and launched at GTC.
• Purpose-built systems bring server power to the desk side. Lawler says the traditional Blackwell-accelerated towers and mobiles remain, but the new part is two system-on-a-chip Grace Blackwell machines purpose-built for data scientists and developers, which he presents as Dell recognising where the market is going.
• Nvidia's software layer makes the hardware a full solution. Bali, who leads Nvidia's AI solutions go-to-market, says its AI software gives a consistent, pre-optimised experience from workstation to server to data centre and back. Developers then need not tune models to hardware themselves; Nvidia and Dell sell the combination as the Dell AI Factory with Nvidia.

2025-04-30 · open the original ↗

• Most people are taught math in a language they don't speak. Luis Serrano of Serrano Academy argues that abstraction is the native tongue of a small academic minority, and that concrete framings unlock what people already know: few can draw a written equation, but everyone draws a parabola for a tennis ball falling off a building. His own case is that he was failing grade-8 math in Colombia, then placed first in the country on a puzzle-style olympiad exam.
• Learn machine learning by playing first, theory later. His analogy is that you hand a child the soccer ball before the rulebook. He advises taking a dataset from a hobby (he used tennis), plotting it with only the line of code needed, and learning linear regression when a line shows up. He demonstrates the method by explaining gradient descent as walking down a foggy mountain, with the loss as your height.
• Make AI do your whole job to find what's still yours. His test is to try to get the computer to do 100% of the work and see the 1% where it falls short. Serrano's pyramid runs from physical to rational to emotional and intuitive intelligence. Machines already beat humans at chess and go, and he predicts math and physics come next, so he expects the unreplaceable part to sit higher up. He makes his videos with language models, and the part that is still his is the questions he asks; asking a question, he says, is harder than answering one.
• Quantum computing will be a small quantum step inside classical machines. His analogy is an airplane that is a bus most of the time and only uses its wings for the step that would otherwise take a million years, such as factoring, which would break RSA. He believes, hedging that he may be wrong, that quantum machines suit generative AI and classical ones suit supervised learning. In his own small experiments, quantum generators produced output further from the training data. He then speculates that he cannot rule out quantum randomness carrying empathy or intuition.
• Malleability matters more than knowledge for data professionals. This is his answer to the show's most-underrated-skill question. He says knowledge is overrated because it no longer lasts 50 years, and when hiring he looks at how many new things a candidate has learned. His analogy is that someone who runs marathons will manage an Ironman even before learning to swim.

2025-04-30 · open the original ↗

• GTC has become a festival-scale event spilling across San Jose. Bali, of Nvidia, describes partners from cloud providers to startups filling the town, many conversations with robots over five days, and people lining up from 6:00 a.m. to get into the keynote, which Jensen Huang preceded by serving food from a Denny's food truck.
• Attendees should arrive with a plan or be overwhelmed. Lawler, at his first GTC, says the volume of learning tracks and booths was overwhelming in a good way, and advises picking a learning track suited to your role beforehand rather than wandering the show floor.

2025-04-29 · open the original ↗

• Scarce cloud compute is pushing AI development back to the desk. Bali, who leads AI solutions go-to-market at Nvidia, recalls a data scientist who had to queue in a Slack channel for an instance, and argues experimentation needs a local sandbox. Nvidia's RTX Pro Blackwell GPUs double memory from 48 to 96 GB per GPU, with up to four in one workstation.
• Nvidia's software layer is meant to remove hand-tuning models to hardware. Bali describes NVIDIA AI Enterprise: NIM microservices package models in containers already optimised for the GPU they land on, so a model can be swapped in with one line of code as Llama versions turn over every couple of months. She cites build.nvidia.com for free prototyping, NeMo for fine-tuning and guardrails, and CUDA libraries such as RAPIDS cuDF (up to 100x on pandas-style preprocessing, no code change) and cuML, announced at GTC (up to 50x on scikit-learn tasks).
• Data-science-only users should choose Dell's Linux GB10 and GB300. Lawler says these Grace Blackwell Dell Pro Max systems ship Linux-only with the Nvidia stack preloaded, while anyone also needing tools like Adobe Creative Cloud, which doesn't run on Linux, should take a traditional Windows Pro Max. Bali adds that the palm-sized GB10 can sit beside a Windows laptop, and two can be daisy-chained over ConnectX.
• The GB300 offers server-class power without server infrastructure. Lawler gives it 784 GB of unified memory (288 GB GPU, 496 GB CPU) and 20,000 TOPS at FP4, against roughly 4,000 for a single 96 GB RTX Pro 6000, and pitches it at mid-market firms with no racks or cooling. Pricing was unreleased at recording; he expects the GB10 near Nvidia's DGX Spark ("3 to 4,000") and the GB300 to cost more but less than a server.
• Both guests forecast AI spreading to non-developers and the physical world. Lawler, not a developer, says he trained his own LoRAs for an animation studio and imagines a RAG system over his mother's recipe cards. Bali, echoing Jensen Huang's GTC keynote, predicts a shift from generative to agentic AI, with agents handing off across a factory, and then physical AI such as driverless cars, adding her personal hope for guardrails and regulation.

2025-04-28 · open the original ↗

• Large language models are here to stay, but every layer around them is in flux. Webber says integration, fine-tuning, agentic systems, pre-training, data-set selection and evaluation will all keep changing, while the core problem of training a model as well as possible and serving it efficiently has not changed and, she expects, never will.
• Knowledge belongs inside the model wherever possible. Drawing on her SageMaker years, Webber argues that pushing knowledge into the network simplifies development, data management and application management, and calls RAG and agentic layers "fluff" next to what the weights hold. She predicts problems first solved at the agentic level will be absorbed into RAG, then pre-training, then the data set itself.
• AWS's accelerator roadmap is only beginning, by her account. Webber says Trainium 3, pre-announced at re:Invent, is on the way, and that customers are "very much just getting started" with what Trainium and Inferentia will deliver.

2025-04-27 · open the original ↗

• Trainium and Inferentia share a core and differ in topology. Webber says both use the same NeuronCore and software stack, so workloads can move between them. Trn1 connects cards in a 4D torus suited to gathering results across cards for the backward pass and optimizer update, while Inferentia lays cards out more simply for sharding a large tensor through a forward pass.
• Inferentia offers more sizes because inference needs vary more. Webber says Inferentia instances come in many combinations of accelerator count and HBM capacity, suited to hosting a 7 billion or 11 billion parameter model, while Trainium comes essentially small and very large: develop on a single Trn1, then scale to as many large instances as you can get.

2025-04-26 · open the original ↗

• AWS builds its own accelerators to give customers hardware choice. Webber frames AWS's position as choice over data sets, models and accelerated hardware, and says customers pick Trainium and Inferentia above all for price-performance and energy efficiency. She claims Trn2 is the most powerful EC2 instance on AWS for AI/ML.
• Annapurna Labs' track record underpins the chips, in her telling. Webber says Amazon acquired the startup in 2015 to rebuild the hypervisor, and its Nitro system physically separates customer data from instance control, which she credits for both AWS's scaling and its security; every modern EC2 instance runs on it.
• Graviton CPUs now take most new AWS compute. Webber cites re:Invent: more than half of new compute coming onto AWS is Annapurna's Arm-based Graviton, flagged by a trailing "g" in instance names such as C6g. Trainium and Inferentia are its third product line, focused on AI/ML.

2025-04-25 · open the original ↗

• ASML alone can build the machines behind leading-edge AI chips. The host says the Dutch firm's latest lithography system weighs 150 tons, is roughly the size of two shipping containers and costs $350 million. No other company can reliably print features below 7 nanometres, and ASML holds over 90% of the market even for mature technologies, supplying TSMC, Samsung and Intel.
• EUV light comes from tin plasma about 40 times hotter than the sun. The host describes the machine firing 50,000 droplets of molten tin into a vacuum, each hit twice by lasers, producing plasma near 220,000°C. The light is bounced off mirrors with imperfections measured in trillionths of a metre onto a chip blueprint. The resulting processors can hold over 100 billion transistors, more than 70 layers and over 100 km of wiring on silicon about 1.5 times a US postage stamp.
• Lithography leadership is measured in decades, not months. The host argues that ASML spent two decades perfecting EUV and money cannot compress that. Its high-NA EUV raises aperture from 0.33 to 0.55 to print 8 nm features, with mirrors doubling in size and growing 10 times heavier. Hyper-NA above 0.75 is next, and wavelengths around 6 nm are seen as a plan B.
• China is squeezing older tools while building its own. Barred by the US from ASML's most advanced equipment, Chinese makers use multi-patterning on older machines to print features two or four times smaller, at a cost in complexity and speed. A state-owned firm is reportedly progressing on a 28 nm-capable tool. The host says matching EUV would mean replicating ASML's supply chain of more than 5,000 specialised suppliers.
• Canon's nanoimprint bet is cheaper but not yet chip-grade. Per the host, Canon's press-like nanoimprint lithography could in theory reach nanometre accuracy at about 40% lower cost. Defects, alignment and speed have so far confined it to uses like smartphone displays and memory chips, where higher defect rates are tolerable.

2025-04-24 · open the original ↗

• NxD hides most of the Neuron SDK's complexity from developers. Webber says the Neuron SDK spans low-level runtime, driver and compiler up to NeuronX Distributed (NxD), the main modelling library. NxD compiles a model and shards a checkpoint such as Llama or PixArt across an instance's accelerators, handling communication, optimizer updates and the forward pass. Prebuilt NxD model packages let users run alignment, supervised fine-tuning or hosting experiments with little exposed complexity.

2025-04-24 · open the original ↗

• Leave big tech for a startup with explicit push and pull reasons. Jesse Lee, former head of data at Descript, says the pull was learning the whole data lifecycle on the way to leading a data function, plus a window of low financial pressure while her child was one. The push was Meta, where zero-to-one products carried heavy sign-off processes and six-month performance cycles that demanded incremental value before product-market fit. She contrasts this with Figma's six years in stealth.
• Score job offers against dimensions you listed before interviewing. Lee advises collecting signal on each dimension from the first interview onward and scoring offers by weighted or simple average. For people who just need a job, she passes on advice to think two jobs ahead and to treat the current employer as paying you to build the skills you want.
• Imported best practices fail unless you meet the company where it is. Lee says her biggest shock was that there was no data culture, no vision and no roadmap, and that bringing in big-tech practice wholesale met resistance. She organizes the work in three buckets: data foundation, data insights (strategic and operational), and data team and culture. Her emphasis shifted from foundation, where she hired a founding analytics engineer about three months in, toward insights and culture. In hindsight she would have hired a data or analytics engineer first.
• Split capacity by fixed shares and triage requests by decision. Each quarter her team aims for about 50% on "big rock" projects, 30% on operational asks, and 10–20% on side projects, a "data university" and brown bags. Day to day, she asks stakeholders what decision the data would change. She estimates about 40% of requests can be pushed back, and many of the rest end once a scoped-down version is delivered.
• Startup fit means self-starting and calibrating your own expertise. Lee lists autonomy, resourcefulness, comfort with ambiguity and a growth mindset. She says work-life balance as a top priority is a poor fit, though she probes rather than rejecting anyone who asks about it. As the "ceiling" of her own function, she found the hardest part was judging which past practices still applied. She says she was overly humble, became disoriented, and learned by deliberately reading experimentation vendors' documentation and taking dbt and data-engineering courses.

2025-04-23 · open the original ↗

• A kernel is where the user overrides the compiler on the chip. Webber explains that PyTorch code for Trainium and Inferentia passes through PyTorch XLA into an HLO graph, which AWS's compiler turns into the chip's instruction set. A kernel replaces that generated code with operations the user defines through NKI, the Neuron Kernel Interface.
• Kernel work is the study of compute optimisation for a workload. Kernels range from a hello-world tensor add to a full forward or backward pass or just an MLP projection, Webber says, and writing one means studying data movement, memory and compute utilisation to improve performance on large language models.

2025-04-22 · open the original ↗

• Customers pick AWS's own accelerators mainly for price-performance. Webber, who moved from SageMaker to the Trainium and Inferentia team because she saw accelerator supply and health become "make or break", frames the chips as part of AWS offering hardware choice. She traces them to Annapurna Labs, acquired in 2015, whose Nitro system and Arm-based Graviton CPUs (over half of new AWS compute, per re:Invent) preceded Trainium, and claims Trn2 is AWS's most powerful AI/ML instance.
• The Neuron software path lets PyTorch code reach the chip, with kernels as an override. Webber explains that PyTorch XLA turns a model into an HLO graph that AWS's compiler lowers to the chip's instruction set, while a kernel written in NKI, the Neuron Kernel Interface, replaces compiler output with user-defined operations tuned for data movement and utilisation. NeuronX Distributed (NxD) handles compiling and sharding checkpoints across accelerators, and Trainium also runs under SageMaker Studio and JumpStart.
• Trn2 quadruples Trn1's compute with the same NeuronCore. Webber gives Trn2 eight NeuronCores and four HBM banks per card versus two and two, 96 GB of HBM per card, 16 cards and 1.5 TB per instance. An UltraServer joins four Trn2 instances, 64 cards over two racks, via NeuronLink with at most two hops card to card. Because the core is unchanged, kernels and NxD code move up easily; she recommends Trn1 for models around 70 billion parameters and Trn2 for much larger ones.
• Tensor-parallel degree is a tuning knob with real costs. Webber corrects the host, who took TP for numeric precision: it is the number of NeuronCores sharing one copy of a tensor. Changing it, from TP8 on Trn1 to TP32–128 on Trn2, shifts time spent in collectives such as all-reduce, batch size and step time, and can help or hurt. Trn2's logical NeuronCore setting regroups 128 visible cores into 64 to give each more HBM bandwidth.
• Anthropic is the flagship customer, and academics get $110 million in credits. Webber cites Project Rainier, a very large Trainium cluster built with Anthropic, plus small-language-model startups and Databricks; the host adds Poolside. Build on Trainium offers $110 million in credits to academic researchers, scaling projects on a cluster of up to 40,000 Trn1 cards, and recently paid $25,000 for the fastest Llama implementation in NKI.

2025-04-21 · open the original ↗

• A model without web access may silently invent what a link says. The host recounts giving OpenAI's o1 pro a URL in deep research, at a time it lacked internet access; its reasoning trace said it would assume what the page contained, and he says the output alone would have misled him had he not checked the trace or the site.
• LLMs graded each other erratically but showed no self-preference. Michaelson describes generating hundreds of SAT essay questions, having four LLMs answer them, then having the same four rank the answers. Results were "wildly unstable" between reruns, but he saw no sign of a model favouring its own answers.
• LLM committees are his answer to evaluation bias. Michaelson still finds LLMs effective at judging LLM output, and suggests playing models off one another, for instance letting a cheap model screen answers and calling an expensive one only when an answer looks questionable. He mentions teacher-student distillation from a recent AWS talk in Ireland but says he hasn't looked into its specifics.

2025-04-20 · open the original ↗

• This upload holds a different clip than its title names. The video posted as "The LLM Revolution Threatens SaaS Businesses" is a copy of the 2025-04-19 clip on deploying models: both run 5:56 and their captions are identical. Greg Michaelson's SaaS argument is summarized under the 2025-04-17 clip and episode 879, dated 2025-04-15.

2025-04-19 · open the original ↗

• Handing models to engineers is a risky dependency, not just a bottleneck. Michaelson argues that the developers who recode data scientists' prototypes usually aren't data scientists and can easily introduce mistakes, while data scientists often lack the skills or infrastructure privileges to serialise, dockerise and deploy models themselves.
• Zerve packages environments and models so data scientists can deploy. Michaelson says every Zerve canvas runs in its own reusable, shareable Docker container, ending dependency mismatches for new hires, and every executed block serialises its variables. An API route can point directly at a trained random forest, and deployments run on Lambda-style serverless infrastructure with no long-running services. Where organisations still forbid data scientists from deploying, a handoff system passes deployable software rather than prototypes to other teams.

2025-04-18 · open the original ↗

• DeepSeek set off an AI mania in Chinese markets and enterprises. The host reports that the Manus agent's registration site crashed within hours of its March 6 launch, with codes reportedly resold on the black market. The Hang Seng Tech Index rose more than 40% from mid-January, and hundreds of large firms, Tencent, city governments, hospitals and universities announced plans to integrate DeepSeek's models.
• Chinese investors are funding startups while calling it a bubble. One Hangzhou-based investor, quoted by the host, said there was "no choice" given the weak economy, with a strategy of entering at an A round and exiting at an A+ round a few months later. On March 6 the central government announced a venture fund of a trillion yuan, about $140 billion, for tech investment.
• Big tech is betting cloud AI will offset weak core businesses. Per the host, Alibaba declared human-like AGI its main objective, released a reasoning model it claims matches DeepSeek, and committed around $53 billion over three years to data centres, more than in the past decade, from a 36% share of China's cloud market. Baidu's cloud revenue has risen.
• The boom is driving a server and hardware spending surge. Demand for AI servers has soared since early February, including all-in-one on-premises units favoured by state-owned firms. A server maker founded by former Huawei staff is up about 140% in 2025, and Jefferies analysts estimate server makers may spend over 1.4 trillion yuan, about $200 billion, over two years.
• Chip access is the boom's chief vulnerability. Morningstar's Kai Wang argues DeepSeek won't fundamentally change most rally beneficiaries, and the host notes an earlier rally faded. Chinese firms still buy Nvidia's weaker H20, local designers such as Cambricon and Huawei lag, and SMIC is capacity-constrained. The host warns that reported US plans to restrict even the H20 could push costs back up and end the euphoria, while welcoming the open-sourcing DeepSeek leads.

2025-04-17 · open the original ↗

• Zerve pitches itself as a full-stack environment for the data science life cycle. Michaelson says that apart from hardware and data warehousing, which it connects to, Zerve covers data connection, exploration, visualisation, reports, GPU model training, self-built or SageMaker deployment and scheduled jobs. SQL, R and Python users work side by side, syncing to GitHub or Bitbucket so engineers can stay in VS Code.
• LLMs have made building so cheap that bespoke SaaS faces an existential crisis. Michaelson argues the old build-versus-buy trade-off has shifted sharply toward building, threatening SaaS vendors that sell custom use-case solutions "and then charging a mint for them", with knock-on effects for venture capital. He doesn't predict what the space will look like in a few years.

2025-04-16 · open the original ↗

• AI mainly speeds up learning, and learners now expect it through assistants. Jess Haberman, director of product content at Anaconda, reports that Anaconda Learning users get through its Python instruction faster and are asking for intermediate and advanced content the two-year-old platform has not built yet. She puts the share of Anaconda AI-assistant users who use it for learning and exploration at roughly 85%, saying not to quote her exactly. Her inference is that content must be "assistant friendly" as well as SEO-friendly, and built on original expert material.
• Weeks-long courses are giving way to modular content in many formats. Haberman is skeptical bootcamps will survive, and says she does not care about certifications when hiring, while noting that she does not hire highly technical roles. She expects academic curricula to change slowly and the gap with messy industry data to persist. Engaging, trusted human instructors, she argues, will still outperform AI instructors.
• AI-generated slop makes a trusted brand the differentiator. She cites AI-made counterfeit copies of bestselling books on Amazon that confuse buyers. Her advice is that trainers and authors build a strong personal brand or align with one such as O'Reilly, and she calls the present a very hard time to self-publish.
• Find good sources by stating your goal and demanding provenance. Her method is to tell a chatbot your level, goal and target field, ask for its top five recommendations with links, and skip anything over two or three years old. She also asks it for the field's top experts, then follows them, their podcasts and their guests. She says to build give-and-take relationships rather than cold-asking influencers for course lists, and notes that posting something wrong reliably draws corrections.
• Communication and cross-functional range are the durable skills. Haberman says AI removes the excuse for poor writing but must be read before it is sent. She adds that experts paid well should show value rather than offload trivia, and that cross-functional people survive a shrinking market. A host adds that strong soft skills now come with a lower barrier to technical skills. Her own upskilling hack is networking, including a managers' group that compiled learning resources for their teams.

2025-04-16 · open the original ↗

• Serverless blocks make parallel execution free of extra code or cost. Michaelson says Zerve's new "fleet" feature, launching the week of recording, fans code out across serverless compute, so a thousand slow LLM calls can run at once instead of in series. Because each block in Zerve's DAG already spins up its own serverless compute, parallelising needs no pool management and, he says, costs no more since total compute is the same.
• A shared DAG canvas does for notebooks what Google Docs did for documents. Code lives in the graph's nodes and data frames flow along its edges, with forks and merges, Michaelson explains, so several people can run Python, R and SQL blocks in one canvas simultaneously. He contrasts Jupyter, where metadata changes cause merge conflicts and "final final" file names; the host supplies the Google Docs analogy.
• Low-code tools hit a ceiling, so value comes from people who code. Michaelson, who spent about seven years at DataRobot, says the emperor of 2010s low-code/no-code "has no clothes", since complex problems exceed such tools, which is why DataRobot and Databricks added notebooks. Zerve, he says, remains code-first, though LLMs have turned coding into describing what you want and then tweaking the result. The host relays Zerve's claim of cutting cycle times by up to nine times; the clip has no evidence for the figure.

2025-04-15 · open the original ↗

• Cheap in-house building with LLMs threatens many SaaS vendors. Michaelson argues the build-versus-buy calculation has shifted so far that firms selling custom use-case software face an existential crisis, though nobody will rebuild something like Salesforce, and he separates "dime a dozen" generative AI bolt-ons from deeply integrated agents. The host sets this against Klarna's and Satya Nadella's similar views and Zoho's CEO's counter-argument that AI will spawn new vertical SaaS, and notes a friend who cut a $20-a-month ChatGPT Plus cost to $2 by calling the API through his own interface.
• Zerve's serverless blocks make parallelism and collaboration nearly free. Michaelson explains that code lives in the nodes of a DAG canvas, with data frames passing along its edges, and each block spins up its own serverless compute. That lets several people run Python, R and SQL in one canvas at once, and lets the new "fleet" feature fan out, say, a thousand slow LLM calls with no extra code and, he says, no extra cost.
• Zerve lets companies choose where prompts go to protect their IP. Michaelson says OpenAI's models are the best in his experience, though he distrusts LLM benchmarks, and sending prompts there is unavoidable with them. Zerve also offers AWS Bedrock and self-hosted Hugging Face models, which can be fine-tuned, and Zerve itself is self-hosted. Beyond prompt-driven GenAI blocks, its AI assistant is an agent that plans, creates and connects blocks from a natural-language request.
• Data scientists should ship software, not prototypes for engineers to recode. Michaelson calls the handoff to developers a risky dependency, since they may misunderstand the model. Zerve gives each canvas a Docker container and serialises every block's variables, so an API route can point straight at a trained random forest and deploy on Lambda-style serverless compute, with a handoff path where data scientists aren't allowed to deploy.
• LLM committees are his answer to bias and error in AI judgment. In Michaelson's test, four LLMs answered and then ranked responses to about a hundred SAT essay questions; rankings were "wildly unstable" across reruns, but no model favoured its own answers. He suggests cheap models screening answers and escalating doubtful ones to expensive models, and warns that a model that gets a problem badly wrong first time is rarely correctable.

2025-04-14 · open the original ↗

• Dell estimates its toolkit cuts NPU app delivery from six months to six weeks. Gupta says that by Dell's estimate a typical, not very complex AI PC app such as a RAG chatbot takes about six months from discovery to fleet deployment, covering model and device selection, runtime compatibility, on-device model operations and manual deployment. Dell Pro AI Studio, in early access at recording and due fully mid-2025, would bring that under six weeks, about a 75% reduction, leaving developers to spend time on picking the use case and validating output accuracy.

2025-04-13 · open the original ↗

• Today's NPUs comfortably run quantised models of about 8 billion parameters. Gupta puts acceptable on-device output at around 15 to 20 tokens per second, which he says rules out going much beyond an 8 or 9 billion parameter model with quantisation. NPUs passing 40 TOPS and smaller LLMs growing more capable are, in his view, the inflection that makes local inference practical.
• Dell-validated models end trial-and-error pairing of models and machines. Gupta describes developers otherwise iterating through 14 and 30 billion parameter models and memory upgrades only to land at two tokens per second, a deterrent to building for the NPU. Dell Pro AI Studio, a developer and IT toolkit rather than a preinstalled app, offers open, permissively licensed language, speech and vision models tested against Dell systems across silicon vendors, published on the Dell Enterprise Hub on Hugging Face.
• Enterprise readiness and on-device middleware are what local-chat demos lack. Gupta says enterprises need attestable models, control of data in and out, and lifecycle management of permissions, updates, deprecations and telemetry, which a Dell management portal provides. On-device middleware detects the silicon, pulls the intended model from a registry, loads, unloads and verifies it, replacing hundreds of lines of code developers would otherwise write and rebuild with each model change.

2025-04-12 · open the original ↗

• Build anywhere, but integrate and test AI PC apps on Windows. Gupta argues that because NPU inference apps will mostly be consumed by Windows users, developers can do much of their work in an IDE of choice on Unix-based machines, but must integrate the model into the app, set up the local host and verify it runs on a Windows device, since that is where the app will run.

2025-04-11 · open the original ↗

• AGI needs planning over an unlimited horizon, which LLMs lack. Burkov, in an episode 867 clip, contrasts crows and monkeys that plan a day or two ahead with humans who plan for generations, and says LLMs are reactive: an "agent" has only the agency its system prompt gives it. He doesn't hazard a timeline, likening AGI to teleportation. The host suggests it may need brain-like modules such as a prefrontal cortex rather than a scaled-up transformer.
• Models can't tell what they don't know, and fine-tuning hacks don't fix it. Burkov says o3-mini's visible reasoning admitted it knew nothing of his new book, released after its cutoff, yet its answer praised the book as if fact. He says Qwen models fine-tuned to say "I don't know this person" denied knowing him, then named him as author of his first book.
• Doubling down on embodied experience is the human edge over AI. In an episode 873 clip, a digital-twin entrepreneur named Natalie argues AI is part of our cognitive evolution as language is, but not all of us. Embodied, serendipitous experience, like the host's childhood browsing of dictionaries, builds insight AI can't replicate, so people should collaborate with AI and use it to free themselves to be more human.
• The AI stack looks different to executives and developers. Richmond, a MongoDB software engineer, says in an episode 871 clip that executives see three layers: application (such as Cursor), tooling (such as MongoDB) and compute (such as Nvidia). Developers need more, starting with programming language, since many libraries exist only in Python or JavaScript, then model provider and database.
• Google's tuning playbook tries to make neural-network training systematic. Varun, a former Google engineer and first author, says in an episode 869 clip that the playbook moves training from "alchemy" toward chemistry. Architecture-agnostic, it helps practitioners choose the next experiment given a compute budget. It was released as a Markdown file on GitHub so others could fork it, and had 28,000 stars at recording, per the host.

2025-04-10 · open the original ↗

• Local accelerators give low latency for persistent real-time work. Gupta's own mnemonic starts with "accelerated": an on-device NPU suits latency-sensitive, always-on tasks such as translation, transcription and captioning.
• On-device AI can learn an individual user's style. For "individualized", Gupta cites a healthcare customer auto-generating ER physicians' reports from their diagnoses; a physician reported that as the model saw his edits it began to sound like him, freeing time for patients.
• Data never has to leave the device. For "private", Gupta argues local models avoid sending confidential or PII-laden data even to a private cloud tenant.
• Offloading inference to PCs cuts data-centre and per-token costs. For "cost", Gupta says a financial-services customer found 15% of its data-centre compute going to developers' code generation and completion, work he believes NPUs can take over without H100s, and cloud inference bills per token. He concedes cloud inference costs are falling, but argues for "the right engine for the right workload". Edge vision cases include real-time factory defect detection and an insurer classifying damage photos into rough cost sizes to get adjusters "maybe 70% of the way there".
• Dell works with chip partners so models run across NPU silicon. Asked about chips from Intel, Qualcomm and AMD, Gupta says the NPU architecture is so new that Dell is preparing models with its silicon partners. This lets developers target different chips without starting over, one reason Dell built Dell Pro AI Studio.

2025-04-09 · open the original ↗

• Marketing science exploits predictable irrationality using experiments and models. The guest, an Amazon marketing-science leader formerly at Walmart's Sam's Club and Nielsen, opens with Dan Ariely's Economist offer: $59 online, $125 print, $125 print plus online. With all three options, 84% of students picked the combo and 16% online-only; without the decoy print option, the split became 68% online-only and 32% combo. He lays out five pillars: experimentation, attribution, machine learning, brand measurement, and optimization and automation.
• Target persuadables, not the people propensity models favor. He sorts customers into lost causes, sure things, do-not-disturb and persuadables, and says propensity models waste discounts on people who would have bought anyway. Uplift modeling, fed by experiments, is what he credits for over 100% lift at Sam's Club and a 2020 award. He cites an eBay experiment in which pausing branded search ads shifted clicks to organic results with no lost sales, though he calls some branded spend a defensive play. He also reports direct mail as one of the most incremental channels in one past role.
• Use media mix models offline and multi-touch attribution for digital. He describes media mix models as aggregate, market-level regressions on about two years of data that control for seasonality and promotions, and says they have existed for 40-plus years. Multi-touch attribution compares customer paths to credit each touchpoint, and cross-channel effects require a DAG, for example TV driving search but not the reverse. He says neither is truly causal until its recommendations are tested against business as usual in a randomized trial.
• Win stakeholders with small staged tests against their own KPIs. He offers a test in four markets of the team's choosing, then scales from 5% to 25% to 50% of the network. In one case, a model that called for cutting spend about 90% showed about 65% in its first test. He says roughly 80% of experiments fail, and he names leadership acting on anecdotes as the most common mistake.
• Cost-center marketing tolerates cuts that top-line marketing never will. In Amazon's tier-one hiring, where recruiting-marketing decisions were decentralized in 2022, his team centralized them behind a media-mix-driven product. He reports a 90% cut in marketing spend over two years with no impact on hiring, stressing that this covers only his team's scope and is his personal view. He says he has tried and failed to get revenue-driving teams to make such cuts. His forecast is that 2025–26 looks like 2023–24, and in about five years generative AI hits creative work first and leaves smaller, specialized teams.

2025-04-09 · open the original ↗

• NPUs are power-efficient matrix-math engines new to ordinary PCs. Gupta says NPUs reached the market about a year before recording, first in Intel's Meteor Lake chipset, joining the CPUs and integrated or discrete GPUs PCs already had. Being purpose-built, almost hard-coded, for the matrix multiplication and addition underlying AI, they use far less power than running models on a CPU or GPU, which would drain a laptop battery.
• For now NPUs are for inference, while GPUs still build the models. Gupta expects data scientists and developers to keep using GPU workstations for training and fine-tuning. NPUs will serve the knowledge workers who consume the results, from assistants answering questions over a knowledge base to AI embedded in workflows and, he foresees, multi-agent systems, without tanking battery life.

2025-04-08 · open the original ↗

• NPUs are inference engines for end users, not training hardware. Gupta, of Dell, says NPUs arrived about a year before recording with Intel's Meteor Lake and are near-hard-coded for matrix math, so they run AI far more power-efficiently than a PC's CPU or GPU. GPUs remain for training and fine-tuning. He puts acceptable output at 15 to 20 tokens per second, which limits today's NPUs to quantised models of about 8 or 9 billion parameters, now capable enough to make local inference practical.
• His "AIPC" mnemonic makes the case for on-device AI. Gupta's letters stand for accelerated (low latency for translation or captioning), individualized (an ER report generator that learned a physician's style), private (data never leaves the device) and cost. On cost, a financial-services customer found 15% of its data-centre compute going to developers' code completion, which he thinks NPUs could absorb. He concedes cloud inference is getting cheaper and that not every use case belongs on the device.
• Dell says its toolkit cuts NPU app delivery from six months to six weeks. Gupta describes Dell Pro AI Studio, in early access at recording and due mid-2025, as three parts. Validated models on the Dell Enterprise Hub on Hugging Face end trial-and-error pairing of models and machines; a management portal adds attestation, permissions, updates and telemetry; on-device middleware detects the silicon and loads the right model. By Dell's estimate, a typical app such as a RAG chatbot goes from discovery to fleet deployment about 75% faster.
• Build on workstations, scale on servers, deploy to AI PCs. Gupta advises embedding data scientists with the people whose problem needs solving, then moving production inference to AI PCs if H100-class hardware isn't needed. His example is a healthcare customer that trained a custom vision transformer on its own radiology images and reports physician-level diagnostic accuracy. It now runs in a HIPAA-compliant data centre, and the customer wants to bring it to AI PCs for doctors worldwide.
• On-device agents await small reasoning models; hybrid compute comes sooner. Gupta forecasts agents acting on users' behalf within set parameters, such as buying concert tickets on approval, but says reasoning models must first shrink enough to run locally, which "might take some time". He also expects workloads to shift seamlessly between device and cloud as compute needs demand.

2025-04-07 · open the original ↗

• Merck's display dominance plateaued, so it moved into semiconductors. Beckmann, who heads Merck KGaA's 8,000-employee electronics business, says its late-1960s and 1970s liquid-crystal research took decades to commercialise. At peak, roughly 2005 to 2010, it held 80% of the liquid-crystal materials market. Recognising that every innovation ramps then plateaus, it entered semiconductors around 2014 and built a leading portfolio through acquisitions including AZ, Sigma-Aldrich's high-tech business, Versum and Intermolecular.
• Incumbents' worst innovation mistake is assuming yesterday's success lasts. Beckmann argues that innovating inside an established company is far harder than in a startup because "you are in your own way", and calls believing that what worked yesterday will work tomorrow probably the most fatal mistake in business. He traces Merck from its 1668 founding after the Thirty Years' War to what he says was industry's first quality promise to customers around 1850.

2025-04-06 · open the original ↗

• High-bandwidth memory depends on hundreds of specialised materials. Beckmann says HBM needs the best possible DRAM performance, and DRAM capacitors rely on high-k materials where he says Merck leads. Spin-on dielectrics are among many promising materials, but with hundreds or thousands of precursors drawn from 80% of the periodic table, he declines to single out one as most important.
• Light-based data transfer promises speed and energy gains. Beckmann says photons move data faster than electrons and without the same heat, and that Merck reorganised its electronics sector the year before recording to create an optronics unit built on its display-era expertise in generating and manipulating light.

2025-04-05 · open the original ↗

• Heterogeneous integration scales performance beyond shrinking transistors. Beckmann describes it as the step between front-end transistor making and back-end packaging: combining several dies into one system, stacking dies into memory stacks or placing memory beside a GPU, as in CoWoS-style structures, to shorten data paths. It requires far smaller structures, better heat removal and power optimisation, and front-end-like precision, which opens room for materials and metrology innovation.
• Merck uses AI to skip unnecessary materials experiments. Beckmann says its "materials intelligence" approach applies AI across millions of candidate options to predict how a material will behave in a customer's setup, rather than improving properties sequentially by experiment.

2025-04-04 · open the original ↗

• smolagents keeps agent logic small enough to read and tweak. The host describes Hugging Face's new Python library as giving any LLM the ability to use tools, call APIs and reason over multiple steps. Its core logic fits in a few thousand lines with minimal abstraction over raw Python, and it succeeds the transformers.agents module, which it will eventually replace.
• Its agents act by writing Python code, run in a sandbox. Rather than expressing actions in JSON or text, smolagents' code agents generate Python that is then executed, which the host says gives them Python's full flexibility, with sandboxed execution as the safeguard. A conventional tool-calling agent is also offered.
• Tools are plain decorated functions, shareable on the Hub, with any model. Any Python function becomes a tool with an @tool decorator, type hints and a docstring, and tools or whole agents can be shared and loaded from the Hugging Face Hub, "like an app store". Models can be open ones via Transformers or the inference API, proprietary ones from OpenAI or Anthropic via LiteLLM, or local ones.
• It lowers the bar for agents in business, data science and research. The host lists research question-answering, as in Hugging Face's demo of a leopard running across a Paris landmark, one-line CLI trip planning, browser automation via its web agent, and on-the-fly pandas analysis. He argues firms can prototype agents in a few lines and keep data in-house with local models, and researchers can benchmark small open models against large proprietary ones. He states the episode is not sponsored by Hugging Face.
• Heavier frameworks beat it for scale, data and multi-agent work. By the host's comparison, LangChain surpasses smolagents in scalability, memory management and integrations, and LlamaIndex in large-data retrieval. Pydantic AI offers type-safe validated outputs, while Microsoft AutoGen and CrewAI are built for coordinated multi-agent systems, where smolagents focuses mainly on single agents.

2025-04-03 · open the original ↗

• A chip takes about 1,400 repeated steps from blank wafer. Beckmann explains that leading chipmakers build transistors and power and data lines on 300 mm silicon wafers, the size of a vinyl record, cycling through lithography, patterning, deposition, cleaning and planarisation until the wafer is cut into centimetre-scale dies.
• Chipmaking draws on about 80% of the usable periodic table. Beyond photoresists for lithography, Beckmann says the specialty materials for etching, deposition and planarisation span about 80% of the non-radioactive periodic table. His examples mostly concern logic chips such as CPUs and GPUs, which he calls the main driver of compute, as distinct from memory.
• His route to leading Merck's electronics arm ran through IT and HR. Beckmann studied computer science with a semiconductor focus and worked on chip design in the late 1980s, then spent years in Merck's corporate IT, ran a country sales organisation, and served as board member for HR during a period of transformation before being asked to head electronic materials.

2025-04-02 · open the original ↗

• Scanned PDFs, not text, are document automation's core obstacle. The guest, a senior machine learning engineer working on generative AI in Adobe Acrobat, says tilted, inverted and cropped phone scans blocked enterprise finance, legal and contract automation for years. Layout was handled with OCR and object-detection models trained on open-source and internally annotated data. Vision-language models can now read pages, but he says enterprises cannot trust them for OCR because their output is non-deterministic.
• Don't spend LLM calls where a small model will do. He says LLM costs have fallen roughly 100–200x over two to three years, so tasks such as entity extraction are shifting from trained spaCy or NLTK models to a single prompt returning JSON. At millions of requests, though, he favors distilled or fine-tuned BERT-style models and small language models. He also routes questions to models by complexity, keeping large models for generation and for RAG-grounded answers.
• Production RAG breaks first at naive chunking and ingestion. Previewing his O'Reilly AI Superstream talk on March 4, he says student-style projects chunk every document the same way with default settings. His alternative is to choose and tune chunking per document type, and to parallelize ingestion asynchronously so that indexing billions of documents does not add user latency.
• Evaluate generation with metric suites, judge panels and experts. He combines retrieval metrics such as contextual relevancy and ranking with embedding-based groundedness and faithfulness, then uses LLM-as-judge panels drawn from several providers. Judges score on a defined 1–5 scale, because he says models handle 1–100 scales badly. Where a subject-matter expert scores lower than the judges, he treats the gap as the signal for revising prompts. An automatic flywheel computes metrics on live traffic and stores the metrics rather than user data.
• Newcomers should build projects alongside fundamentals, aimed at business impact. He estimates a few months to a year to master machine learning, deep learning and LLMs from scratch, and says the field moves faster than that. His advice is to build current generative-AI projects while studying in-depth courses, not "master RAG in five hours" videos. He names understanding business impact as the most underrated skill. His forecast is a year of agents, risk for repetitive jobs, and more AI engineers rather than fewer, which he attributes to the Jevons paradox.

2025-04-02 · open the original ↗

• Neuromorphic chips may pay off first in inference, within about five years. Beckmann argues neuromorphic architectures could matter even more for cost-efficient, scalable inference on everyday devices than for training, given that data transfer is a key optimisation target. He calls their use in general compute "a five years plus question rather than a 10 years question", but says it probably won't reach human-brain efficiency; the light-bulb-powered brain comparison is the host's.
• Data-centre AI already scales on several fronts at once. Beckmann says chips such as Nvidia's Hopper and Blackwell combine transistor shrinkage via EUV lithography and gate-all-around nanosheet designs with CoWoS-style integration that places memory beside compute. The same latest technologies are reaching smartphones, laptops, cars and production systems.

2025-04-01 · open the original ↗

• Chip performance now scales along three dimensions, not just shrinkage. Beckmann, CEO of Merck KGaA's electronics business, distinguishes "more Moore" (smaller transistors via EUV and gate-all-around nanosheets), "more than Moore" (heterogeneous integration such as CoWoS, placing memory stacks beside GPUs as in Hopper and Blackwell) and post-von Neumann architectures. He expects neuromorphic computing in general compute in "five years plus" rather than 10, though not at human-brain efficiency. Superconducting quantum approaches need near-zero-kelvin cooling and remain a tiny market; Merck's Intermolecular facility in San Jose is exploring qubit materials with partners.
• Co-optimising materials, delivery and inspection is meant to compress years-long cycles. Beckmann says chipmakers' yield problems are solved fastest by integrating chemistry, physics, electrical testing and chemical delivery systems. That is the logic for acquiring UnitySC, whose high-resolution visual inspection checks 2.5D and 3D structures such as through-silicon vias. Merck calls its use of AI, including on metrology data, to cut unnecessary materials experiments "AI for AI".
• AI drives leading-edge chips while the rest of the industry stays in a cyclical slump. Beckmann says volume still depends on smartphone and PC replacement, which remains low as people keep COVID-era devices into a fourth or fifth year, and AI data-centre demand "doesn't bail everything else out". Because fabs run either at full capacity or idle, the industry is inherently cyclical; after semiconductor-materials growth above 20%, he says Merck stays humble, knowing shrinking years follow.
• Good AI regulation protects data contributors without slowing permits. Beckmann argues frameworks should ensure those who make data available get returns when it feeds LLMs, since that stability helps investment. But he says that needing about a year to get a permit to make a new material in Darmstadt hurts innovation, and that Europe is not leading on approval speed.
• Every innovation plateaus, so incumbents must switch gears deliberately. Beckmann recounts that Merck's liquid-crystal research from the late 1960s led to an 80% share of liquid-crystal materials at the flat-panel peak around 2005 to 2010. It then moved into semiconductors from 2014 through acquisitions. He calls assuming yesterday's success will last "probably the most fatal mistake you can do in business".

2025-03-31 · open the original ↗

• Experts can rent out vetted AI twins of themselves. Monbiot says she now advises a startup (spelled on air W-I-Z-L-Y) that admits experts selectively, lets them train a twin on knowledge they control, use it to generate leads, and monetize it in consulting calls once they trust it to represent them.
• A twin's value is augmented skill, not just bought time. Because the twin is cheaper, always available and speaks any language, Monbiot argues it can serve clients abroad at a price point they can afford, which she calls the "10x" of the idea.
• A CEO's twin can carry company culture across an organization. Monbiot describes building one for a healthcare CEO, first inside Slack, where employees can consult it on his values, have it interject, and get feedback on work, including staff at a company recently acquired; voice and other modalities are to follow. The host flags liability as an open question when people act on the twin's advice.

2025-03-30 · open the original ↗

• Each step from books to Google to LLMs removes cognitive work. The host argues that looking things up in a dictionary or encyclopedia exposed him to incidental information that built a web of connections and aided memory, while a search page surfaces ads and an LLM output does the comparing for you.
• Humans should double down on embodied experience to stay competitive with AI. Monbiot holds that AI is part of human cognitive evolution the way language is, but not all of us: being embodied and embedded in the world, noticing things and making real-world connections is the human advantage AI lacks. Her prescription is to use AI to solve big problems and free ourselves "to be more human".

2025-03-29 · open the original ↗

• Outsourcing whole cognitive processes to LLMs makes us dumber, unlike Google. Monbiot answers commenters who compare the worry to early fears about Google: search returns facts that feed into your own reasoning, whereas an LLM can tempt you to hand over the entire process without reviewing or challenging the result. She expects this to change how children are educated and how adults reskill to avoid that trajectory.

2025-03-28 · open the original ↗

• Baseball analytics has gone from Moneyball experiment to AI arms race. The host says every MLB team now employs data scientists, and that Statcast, introduced in 2015 with high-speed cameras and radar in every ballpark, generates seven terabytes of data per game, far more than human analysts can sift.
• Machine-learning scouting finds players that traditional stats undervalue. Models ingest college and minor-league stats, biomechanics and video, and weight metrics such as exit velocity, launch angle and spin rate; the host's example is a prospect with a low batting average who consistently hits the ball hard. Teams also use the analysis to tailor drills, such as for a hitter who struggles with high fastballs.
• In-game decisions now lean on simulated matchups and positioning models. Managers consult win-probability tables and pitcher-versus-batter predictions on tablets in the dugout, and data-driven defensive shifts proved effective enough that MLB introduced rules limiting extreme shifts. The host stresses the tools augment managers rather than replace them.
• AI is reaching fans and officiating as well. Broadcasts show live win-probability graphs and catch probability on replays, and the host reports that during 2025 spring training MLB let players challenge human umpires' ball-strike calls to an automated system, which drew mostly positive reviews.
• Any traditional industry can be transformed by pairing domain expertise with AI. The host's lesson is that if America's oldest major sport can reinvent itself, finance, healthcare, manufacturing and retail can too; early adopters gain an edge until others catch up. He forecasts AI won't replace product managers, physicians or supply-chain planners for the foreseeable future, but will arm them with better information.

2025-03-27 · open the original ↗

• Step-by-step reasoning models could scale to many Nobel-level minds. The host, prompted by OpenAI's o1 release, argues synthetic data could broaden reasoning beyond math, chemistry and code, and forecasts that within "a matter of years" a data center could hold a million Nobel-prize-level machines, the "powerful AI" of Anthropic CEO Dario Amodei's essay.
• Relying on LLMs for cognitive work makes people dumber. Monbiot cites studies showing users' cognitive capabilities decline when LLMs do their thinking, and warns of a "great intelligence divide" in which a few build ever more powerful AI while everyday users degrade. Her focus is on how AI can augment human agency rather than on alignment.
• AI's real payoff is freed capacity for ingenuity, not a modest speed-up. She calls the roughly 15% average performance gain for white-collar workers found by Ethan Mollick and others uncompelling, and prefers Andrew Ng's framing of the 10x engineer as exponentially more effective because AI frees time and mental capacity.
• Used well, AI expects more of people, not less. Monbiot cites the former CEO of Drift building an AI-agent-first customer-success startup whose first hire after engineers was a lawyer who also oversees HR. She argues people must think laterally and drop ego: engineers whose work is democratized, and non-technical people who can no longer claim "I'm not a coder".
• The "anyone can now code" narrative isn't fully true yet. Monbiot, a linguist by training, says she got halfway through a Python course, tried building, and had to go back and learn more. The host argues the best learning pairs a specific goal with just enough fundamentals, with the balance differing by person.

2025-03-26 · open the original ↗

• Prototype with AI tools to test demand before building. Colin Matthews, a PM and founder of Tech for Product, argues that testing users on Figma mocks gives weak feedback, especially for ML features people need to "feel". His fastest side project once took 3 months to ship, and he estimates a web app now takes about a week. The bigger gain, he says, is that you hit the real problem of getting users sooner.
• AI speeds up work you already understand and compounds errors otherwise. Non-technical builders lack a mental model of client, server, database, hosting and authentication, so they cannot spot the AI's mistakes or describe a change correctly. Vibe coders also risk shipping broken auth, security or Stripe integrations that only look like they work. He expects smaller teams with 2–3x output, starting at startups and spreading to enterprises, but not the end of engineers, because system design is the real work and "coding is a medium for your thoughts".
• Choose the tool by your technical skill. If you can already ship, he recommends Cursor with a JavaScript stack (React, Node, Postgres, hosted on Railway), because models perform best on JavaScript. Otherwise he recommends Replit, which handles client, server, database and auth in one place and is the only tool he names that runs Python. Bolt, Lovable and v0 lean on Supabase for the back end, which he finds less consistent, so he uses them for client-only prototypes, with AI responses mocked as JSON. He runs a few simple production apps built this way and moves complex ones out to Cursor.
• Plan in a spec file, revert on errors, diagnose before fixing. A global prompt makes his tools start every project with a PRD.md spec listing components, styling and implementation phases, so the agent works from that file rather than a long chat history. When bugs pile up he reverts to the last working commit instead of debugging in chat. He asks the AI to "explain the issue, don't write any code" first, because the tools, all running on Sonnet 3.7 at the time, write code by default. Non-technical users should retest every feature after each change.
• The Replit demo built working apps, on an unusually easy case. From a one-line prompt and no uploaded data, Replit built a Python/Streamlit analysis of the Iris dataset (box plots, scatter plots, histograms, correlations, a pair plot, PCA), then added decision tree, random forest and gradient boosting models. A deliberately vague "billion-dollar app" prompt produced "Money Maker", a data-cleaning, charting and report app of about 3,000 lines that worked on fake CSV data, although its upgrade button went to a random domain. Colin warns that long autonomous runs like this are rare, that the run worked because Streamlit is simple (Next.js "would have 100% broken"), and a host notes that thousands of Iris analyses already exist online. Pricing as he gave it: $25 a month for $25 of credit, 25 cents per agent message and 5 cents per assistant message, so he put this demo at roughly $1–2.

2025-03-26 · open the original ↗

• Small changes to a technology like language can reshape civilization. Monbiot cites a scholar's argument that the ancient Greeks' addition of vowels made expression more fluid and helped language spread, contributing to the birth of democracy, an impact only visible in hindsight. She argues AI's effect on humanity is similarly impossible to predict yet but probably huge.
• Advanced language arrived long after brains capable of it, the host says. He puts sophisticated language at only tens of thousands of years old while saying Homo sapiens have been genetically "pretty much the same for millions of years", and tells listeners to double-check that fact rather than quote him.

2025-03-25 · open the original ↗

• Humans should be AI's meaning makers, not lemmings off a cliff. Monbiot traces her recent writing to the book God, Human, Animal, Machine, which she recommended on her previous appearance (episode 823), and to an op-ed in the Polish magazine Przekrój asking whether we are ceding humanity to machines. Her answer: humans created the opportunity and the mess, and are good at cooperating to fix messes.
• AI is part of human cognitive evolution, as language was. Drawing on John Vervaeke's lecture series The Meaning Crisis, she calls language a "psychotechnology" that enabled self-reflection and cites his argument that Greek vowels helped give birth to democracy. AI is part of us the way language is, but not all of us, because humans are embodied and embedded in the world.
• Leaning on LLMs risks a "great intelligence divide". Monbiot cites studies that users' cognitive capabilities decline when LLMs do their thinking, and dismisses the roughly 15% average gain for white-collar workers as uncompelling. She favours Andrew Ng's 10x-engineer framing, and the example of an AI-native startup whose first post-engineer hire was a lawyer also running HR, as signs AI should demand more of people.
• LLM drafting drags writing toward the mean. Monbiot says the more iterations she does with an LLM, the more her "aha" insights get averaged out and she has to reinsert them, so she is unsure it saves her time. She and the host agree the states worth optimizing for are flow and peace; the host argues free, ad-funded platforms are built to keep users in a frenetic, fearful state instead.
• Everyone will have AI twins working on their behalf, she predicts. Monbiot's "virtual human economy" thesis drives her advisory work for a startup offering vetted experts AI twins that can earn consulting income in any language, and a custom twin of a healthcare CEO deployed in Slack to carry company culture. The host describes his own early idea for an embodied AI that would steer him back toward flow and peace.

2025-03-24 · open the original ↗

• Build AI strategy on pillars and objective truths, then derive tactics. Alake, who leads and mentors AI teams at MongoDB, calls this the year of "show me the money" for AI and says his method is first-principles: identify the pillars around a persona, then the things within each pillar that will not change short of a platform shift. His example for a developer-focused strategy is GitHub as a pillar, asking what developers actually do there, then forming convictions, assumptions and tactics from that.

2025-03-23 · open the original ↗

• This is the "multi" era, not just the year of agents. Alake calls 2025 the "multi" era: agents went mainstream this year, so leaders should look further ahead: to multi-agent architectures, multimodal embeddings covering text, images, video and audio in one space, multilingual embeddings, multi-step reasoning, and multiple retrieval mechanisms, since "vector search is not all you need".
• Voice agents will be widely built but won't dominate, he predicts. Alake expects everyone to experiment with voice interfaces given AI's experimental nature, while holding a strong opinion that voice will not become the main way people engage with agents.
• Memory management will become a central agent-design problem. He recounts that Andrew Ng's first question to him, in April of last year, was how to model memory in agentic systems, and argues agents will need several forms of memory and recall, as human brains do.

2025-03-22 · open the original ↗

• The AI stack is described differently to executives and to developers. Alake, whose blog post on AI stacks the host says was the top Google result for the term at recording, gives executives three layers: application (e.g. Cursor), tooling (e.g. MongoDB) and compute (e.g. Nvidia). For developers he breaks it into more layers, including programming language, model provider and database.
• Choice of programming language constrains the rest of the AI stack. Alake stresses that many libraries further up the stack exist only in Python or only in JavaScript, though some are evolving to support both, so the language decision comes early and matters.

2025-03-21 · open the original ↗

• This upload holds a different episode than its title names. The video posted as 872 is a copy of episode 870 on OpenAI's Deep Research: both run 17:10, their captions are identical, and neither mentions the Majorana 1 chip. Episode 870's summary, dated 2025-03-14, covers what the video says.

2025-03-20 · open the original ↗

• MongoDB Atlas exists because no developer wants to be a database administrator. Alake describes Atlas as a managed, distributed MongoDB running across AWS, Azure and Google Cloud in multiple regions, bundled with streaming and dedicated search nodes.
• Vector spaces place similar meanings close together. The host explains embeddings from Tomas Mikolov's word2vec, around 2011–2012, which learns a word's meaning from surrounding words without labels; in a trained space, spelling variants coincide, NoSQL sits near MongoDB, and dimensions can run to hundreds or thousands. Documents embedded the same way let RAG systems retrieve relevant items from, say, a million documents by a query's meaning.
• MongoDB folded search and vector search into one database. Alake says customers had been bolting search engines, extensions or separate databases onto MongoDB, so it added Atlas Search and then vector search, allowing lexical, vector and hybrid search in a single database where he has seen some teams run two or three.

2025-03-19 · open the original ↗

• AI on both sides has made the job market noisier. Karun Thankachan, a senior data scientist at Walmart and former Amazon applied scientist who mentors job seekers, says recruiters screen with AI and ATS keyword search while candidates mass-apply. Some services do four applications for a dollar, so a candidate who might once have applied to about 100 jobs can now apply to close to 1,000. He splits the search in half: a portfolio and resume that show the role's skills, and referrals.
• Treat the search as a funnel with weekly targets. He has mentees send 100 LinkedIn connection requests a week, the cap on the lowest Premium tier. Fewer than four to six responses means the profile needs work. He also sets 20 referred applications a week, which new users won't reach for the first three weeks or so, and notes that a referrer will typically cover up to three roles, a norm he believes Google set. His mentees' rates have fallen: callbacks from about 10% in 2019–2020 to about half that, and referred-application callbacks from about 5% to 2–3%, which he partly blames on referral-for-hire groups. Cold email found through tools like Apollo and Hunter has done no better than LinkedIn requests.
• Keep a few templated resumes rather than tailoring each one. He kept three versions (time series/e-commerce/finance, NLP, and a mix), and he pastes each job description into an AI to pull keywords into the competencies section. In his experience the ATS is a bar to clear, not a ranking, and a human then spends 6–7 seconds on the resume. Tailoring every resume costs volume and introduces errors. For applied data science, the resume needs to show three things: data analysis and feature engineering, fitting and tuning models, and productionizing them. Freshers with scattered projects should target companies under about 2,000 employees or pivot to data analyst roles. Grad students should pair an NLP project with a generic prediction-at-scale project and a time-series project.
• Mid-career candidates fail on ambiguity and ownership, not technique. A senior analyst can usually move to product data science without leveling down. Applied science adds a LeetCode-style coding bar and proof of models built at scale, which he says should come from personal projects or shadowing work at the current employer. The gaps he sees most are the lack of a STAR story about an ambiguous problem and never having led work end to end, which shows as missing nuance. A host adds that candidates should spell out the complexity and challenges, or the work sounds easy.
• Recommenders run in stages, and LLMs help at the front. He describes candidate generation (often matrix factorization on Spark, retrained daily by a central team), then real-time ranking, then scoring with business rules, and says implicit signals are the most researched despite their noise. He reports LLMs helping with candidate generation and cold start, citing an Alibaba-linked paper whose name he says he is unsure of, and a 2024 "hierarchical LLM" paper using LLM item embeddings for sequential recommendation, which he says to "give maybe a year" before it is refuted. He doesn't expect AI to replace data scientists because so much knowledge is never written down, and even the best documentation he has seen is about 30% incomplete. His most underrated skill is making connections: his biggest gains on mature products came from finding new data sources, not better models, so he advises at least one coffee chat a month outside your team.

2025-03-19 · open the original ↗

• MongoDB's document model mirrors how developers think. Alake contrasts relational databases' tabular rows and columns with MongoDB's key-value, JSON-like documents, and says seeing data represented the same way in the application and data layers made him a full-stack MEAN developer (MongoDB, Express, Angular, Node.js) after he had disliked programming.
• Flexible schemas suit fast-moving AI experimentation. The host says his startups chose MongoDB around 2014 because pages' data needs were uncertain; Alake argues AI work is the same, with schemas evolving during experimentation and lockable once stable in production. The host's example is storing extra output fields from a Llama model that the OpenAI API lacks, without a schema migration.
• JSON unifies the AI stack from LLM to database. Alake argues that with LLMs now offering JSON mode and structured output, JSON runs through the application layer, the data layer and the model, and that avoiding schema migrations matters because developer speed is a competitive advantage.

2025-03-18 · open the original ↗

• A flexible document schema suits AI's constant experimentation. Alake, a staff developer advocate for AI/ML at MongoDB, argues the JSON-like document model lets the data layer evolve while requirements are unclear and be locked once in production, avoiding schema migrations; with LLMs offering JSON mode and structured output, JSON now unifies application, data and model. The host, who says MongoDB does not sponsor the episode, gives the example of storing an extra output field from a Llama model that the OpenAI API lacks.
• One database can handle operational, lexical, vector and hybrid search. Alake says customers had bolted search engines, extensions or separate vector databases onto MongoDB, sometimes running two or three databases, so MongoDB built Atlas Search and then vector search into the managed, multi-cloud Atlas service. The host adds that a single vendor avoids version mismatches between linked systems.
• Store an agent's tools in the database; retrieve only a few. Alake's open-source library Memorizz implements a "MongoDB as a toolbox" pattern: since OpenAI's guidance is to put only 10 to 20 tool schemas in context, store perhaps a thousand tools with embeddings of their docstrings, then fetch two or three before each LLM call. He jokes that stuffing a million-token context window on every call depends on "how deep your pocket goes".
• The "multi" era is here, and memory is the next frontier. Calling 2025 the "multi" era, Alake expects, beyond single agents, multi-agent architectures, multimodal and multilingual embeddings, multi-step reasoning and multiple retrieval mechanisms, since "vector search is not all you need". He predicts voice agents will be widely tried but not dominant, and that memory management, as pursued by MemGPT's maker Letta and the HippoRAG paper, will become central.
• Ground AI strategy in objective truths; test tactics in the market. Alake's "arena framework" pictures a coliseum whose pillars are segments of objective truth (for developers, GitHub and what they always do there), with tactics battling it out in the market. For teams he prescribes being aggressive, centralized, coordinated, resourceful, efficient and data-driven.

2025-03-17 · open the original ↗

• Extrapolate current curves a little rather than predict AGI. Godbole, recently retired from Google, says his post "From Knowledge to Wisdom" starts from observable trends: each year models get much better while inference cost for the same functionality falls by orders of magnitude, context windows grow and latency improves, as with Tesla's Roadster preceding cheaper models. He notes there is no consensus definition of AGI.
• AI coding tools will produce "full-stack employees". Because tools such as Cursor, Lovable, Replit and Copilot make software accessible to non-engineers, Godbole expects a PM to arrive with generated mocks, a prototype and early user research before an engineer spends a second on it. He suggests roles are bundles of skills that may be unbundled and dynamically rebundled, leaving the shape of future organizations unclear.
• Cheap reproduction could make software resemble the music industry. If an undifferentiated SaaS product can be cheaply copied, Godbole reasons through a chain of "ifs" that most offerings face near-perfect competition while a few highly differentiated winners take outsized returns. He expects the effects to be discontinuous, such as small teams simply not hiring a PM.
• When knowledge is commoditized, wisdom becomes the differentiator. Godbole argues good ideas still come from humans, and that cultivating wisdom resembles cultivating curiosity, open-mindedness and adaptability. His aim is not more agentic AI but using AI to make himself more agentic, self-authored and pro-social, which he says also improved how he wields models, since language makes human assumptions the models do not share.

2025-03-16 · open the original ↗

• LLMs have turned much of machine learning into prompting plus evals. Godbole, a core member of Google's Gemini team until he recently left, calls LLMs an "arbitrage of ML talent": where adding intelligence to a product once meant PyTorch or JAX and data collection, it now means articulating the behavior you want in clear English and measuring it with good evals, a very different skill set.
• The mental model in his prompting playbook outlasts its prescriptions. Godbole says the second half of the playbook he published before leaving Google, examples of good and bad prompts, reflects today's models, since prompting o1 or Gemini Flash Thinking feels different from GPT-4o or Gemini 1.5. The first half, on pre-training versus post-training, he considers more future-proof.
• Pre-training learns every "cinematic universe"; post-training picks one. He argues that for an LLM there is no objective fact: "Bilbo Baggins is a hobbit" is true only in Tolkien's universe, and web text approximates the union of all such universes. Post-training narrows the model to an assistant-and-user universe, so post-trained models are statistical models role-playing as human raters role-playing as AI assistants.
• Prompt as if briefing an all-knowing stranger who doesn't know you. Godbole says complaints about why a prompt is failing usually work better when pasted into the prompt, and that models respond best to extremely explicit, low-context communication. He recommends the book Nonviolent Communication, learned in couples therapy, because being explicit about the observable behavior you want works on people and models alike.

2025-03-15 · open the original ↗

• Google bet on ML code completion before ChatGPT made it obvious. The host points to a July 2022 Google Research post, "ML-Enhanced Code Completion Improves Developer Productivity", noting that at the time such tools might have seemed more trouble than their errors were worth. Godbole, who moved from medical imaging to this work, says early results were promising and that steadily improving hardware and model size made engineering productivity a natural use case for sequence-to-sequence models.

2025-03-14 · open the original ↗

• OpenAI's Deep Research does hours or days of literature review in minutes. The host, who notes OpenAI does not sponsor him, explains that it breaks a query into sub-tasks, searches the web for each, and iteratively synthesizes a cited report, pivoting as it learns. OpenAI trained it with end-to-end reinforcement learning on browsing and reasoning tasks, and it can read uploaded files, plot with Python, embed images and cite specific passages.
• Deep Research jumped far ahead on Humanity's Last Exam. On the 3,000-question benchmark spanning over 100 subjects, the host's table at recording shows GPT-4o at 3%, Grok 2 and Claude 3.5 Sonnet around 4%, Gemini Thinking about 6%, o1 and DeepSeek R1 around 9%, o3-mini (high) about 13%, and Deep Research at 27%. He predicts that, as with past coding and math benchmarks, the exam won't take long to be conquered once AI gains traction.
• In the host's use, it asked sharp questions and saved hours. Drafting the syllabus, title and abstract for his four-hour agentic AI workshop at ODSC East, he supplied partial modules and past abstracts; it asked about audience level, hands-on coding and tone, then spent three minutes across eight sources, including his own website, to produce a draft he only lightly edited.
• Google's and Perplexity's versions trade depth for other strengths, the host judges. He says Google's Gemini-based tool shows a user-guided research plan first and is reliable but less nuanced, while Perplexity's is fast and free but shallower; for mission-critical analysis he gives OpenAI's the edge despite its cost.
• The catches are cost and occasional overconfidence. The host pays $200 a month for Pro, which includes 100 Deep Research queries a month, and warns it can hallucinate or present rumor as fact, though he has caught neither and says OpenAI reports markedly lower hallucination rates. He forecasts efficiencies such as smaller models like o3-mini will bring more queries to all paid users and eventually free access.

2025-03-13 · open the original ↗

• The Deep Learning Tuning Playbook aims to turn alchemy into chemistry. Godbole, its first author, says it pooled his Google Brain co-authors' decades of experience into a systematic, architecture-agnostic process: set up a good initial experiment, then decide each next experiment from the data collected, given a compute budget and constraints. He credits the co-authors as the real brains and calls it their "two cents", not the final answer.
• Releasing it as a forkable markdown file made it a community resource. Rather than a PDF on arXiv, the team published it on GitHub under a permissive license to invite forks and pull requests; community members crowdsourced translations the authors do not endorse. The host reports 28,000 GitHub stars at recording and says it answers the question his deep learning students always ask about why to add layers or neurons.

2025-03-12 · open the original ↗

• Open models win on control; closed models on performance and ease. Maarten Grootendorst, co-author with Jay Alammar of Hands-On Large Language Models and maintainer of BERTopic and KeyBERT, works at a cancer research institute that will not send patient data to any cloud provider. He adds other reasons to run models yourself: not knowing which version or business rules sit behind a proprietary API, content restrictions, outages, and fine-tuning on domain data no lab can see. Open tooling, often built by hobbyists, will "never ever be as perfect" as closed providers' interfaces. He names Llama, Mistral, Qwen and DeepSeek as strong open models.
• Small models will improve fast through distillation and serve agents. Large models generate training conversations that teach cheap small models, which already run on phones. In his agent picture, one large model steers many tiny ones that summarize or transcribe, so a model about 90% as accurate handles basic work without the cost of a large one.
• He predicts open source will always catch up with proprietary. He traces scaling from train-time compute to test-time compute ("more so-called thoughts"), and DeepSeek matched that and ranked in the top five, second or sometimes first depending on the evaluation. New architectures such as Mamba, BitNet and Google's Titans show promise, but he says only well-funded labs can test whether they scale, because a university will not "throw 50 million at the problem".
• Representation models still beat generative ones for many tasks. You embed, say, 10 million documents once and reuse the vectors for classification, clustering, topic modeling, deduplication and retrieval. Running those documents through ChatGPT would be slow and costly, and costly again every time the prompt changes. "We're not going to use GPT reasoning models to classify spam." He sees a recurring cycle of chasing the shiny model when bag of words, word2vec or a cheap BERT classifier would do.
• Communication is the most underrated skill in data work. Stakeholders care about usefulness, not model performance, and many data science teams must keep justifying their existence because they are advisory rather than a product, he argues. He suggests practicing by recording 30–60-second explanations on TikTok until viewers' questions shift from "I don't understand" to how the idea would be applied.

2025-03-12 · open the original ↗

• A deep learning system beat radiologists at predicting breast cancer from mammograms. Godbole, co-first author of the 2020 Nature paper "International evaluation of an AI system for breast cancer screening", says it predicted biopsy-confirmed cancers two or three years ahead, depending on the country's screening guidelines, better than expert radiologists. It was benchmarked on a large retrospective US and UK dataset of what he recalls as a hundred-and-some thousand patients, and beat every radiologist in a separate reader study.
• The bigger promise is combining AI and humans and widening access. Godbole says humans and AI have different strengths, so their synthesis can be powerful, though he is out of date on that literature. He highlights countries with many patients per radiologist, where AI could substantially improve access to potentially life-saving screening.

2025-03-11 · open the original ↗

• Use AI to make yourself more agentic, not to build agents. Godbole, recently retired from Google, argues that once knowledge is commoditized the differentiator is how wise, pro-social and self-authored a person is. He wants AI to support metacognition and a reciprocal loop: acting more, hitting one's limiting beliefs, and growing. He credits Toronto professor John Vervaeke's work on wisdom and relevance as a major influence.
• AI predicted breast cancer from mammograms better than radiologists. Godbole was co-first author of the 2020 Nature paper showing a deep learning system predicted biopsy-confirmed cancers two to three years ahead, depending on the country, on a US and UK dataset of what he recalls as a hundred-and-some thousand patients, and beat every radiologist in a separate reader study. He sees the bigger promise in combining human and AI strengths and in expanding access where patients far outnumber radiologists.
• An LLM must be told which "cinematic universe" it is in. Godbole argues that for an LLM there is no objective fact: pre-training fits the union of all universes in web text, and post-training narrows it to an assistant-and-user universe, so models are statistical models role-playing as human raters role-playing as AI assistants. This first half of his prompting playbook, he says, will outlast its prompt examples.
• Prompt as explicitly as you would brief a stranger. He advises imagining a person off the street with all human knowledge but no knowledge of you, and pasting one's complaints about a failing prompt into the prompt itself. Skills from couples therapy and the book Nonviolent Communication carried over, and he reports an anecdotal inverse correlation between ML expertise and prompting skill.
• Cheaper, better models will unbundle job roles and squeeze undifferentiated products. Extrapolating modestly from current curves, Godbole expects "full-stack employees" such as PMs who arrive with prototypes and user research before engineers are involved. Because copying an undifferentiated product will be cheap, he speculates software could come to resemble the music industry's winner-take-most economics, with discontinuous effects such as small teams not hiring PMs.

2025-03-10 · open the original ↗

• DeepSeek erased the frontier labs' moat, not merely their benchmark lead. Burkov calls it probably the most important event for language models since ChatGPT: a state-of-the-art model trained for what he puts at 5 to 10 million rather than the 20 to 50 million previously assumed, with a published step-by-step recipe that several independent R1-Zero and R1 implementations have already reproduced.
• It is also far cheaper to run. Burkov compares OpenAI's o1 at $60 per million output tokens with DeepSeek's $2, roughly a 30-fold reduction, so that anyone, "your grandmother", can now run state-of-the-art AI.
• Automated verification replaced costly expert-written training data. Burkov argues OpenAI and Anthropic invested heavily, by his estimate billions, in subject-matter experts writing problem-solving dialogues, whereas R1 used binary rewards from checks: does the code compile or produce the right output, does the math answer equal the known value, does the logic puzzle name the right culprit. Hiding a class in a GitHub repo and rewarding a compiling rewrite is one way he says close to a million examples can be generated with no experts.
• Open-weight models are not open source without their data. Burkov's stricter definition, unlike Yann LeCun's, says a model is like a binary: reproducing it requires the training data. The few models released with full datasets are no longer competitive, since today's pre-training uses roughly 15 to 20 trillion tokens versus perhaps four or five trillion in open datasets.

2025-03-09 · open the original ↗

• To eliminate hallucination, don't let the LLM generate answers. Burkov says RAG cuts hallucination sharply, by his rough figures from perhaps 20–30% of answers to 1–2%, but not to zero. For his company's closed-domain labor-market product, the team instead fills predefined templates ("the average salary for a nurse in San Francisco is…") with numbers pulled straight from internal APIs.
• Showing the system's interpretation makes remaining errors visible. The LLM only parses the user's string into structured fields, which are normalized to internal taxonomies (e.g. "JS" to JavaScript) and displayed as labels before any number appears. Errors can still come from interpretation or imperfect scraped data, but Burkov says the chatbot's answer matches what the conventional UI would show; the LLM is used "to understand but not to inform".
• Bolted-on uncertainty warnings can be overcautious. The host reports that Claude now often says a topic is niche and should be double-checked, including on questions he doesn't consider niche, which he takes as a layer on top of the core model that still needs tuning.

2025-03-08 · open the original ↗

• AGI is possible but as speculative as teleportation. Burkov declines to give a timeline, saying only that since evolution produced human minds, something analogous might be built.
• Infinite-horizon planning is what LLMs, and other animals, lack. He contrasts crows that use cars to crack nuts and apes that keep a stick overnight but discard it after three or four days without bananas, with humans who plan for generations, as with climate change. LLMs are reactive: an "agent's" agency comes from its system prompt, not itself, so it can be violated unpredictably. The host adds his own hypothesis that AGI may need brain-like modules, such as a prefrontal-cortex analogue, rather than a scaled-up single architecture.
• LLMs cannot tell what they don't know. Burkov asked OpenAI's o3-mini whether his new book, The Hundred-Page Language Models Book, released after its training cutoff, is good; its visible chain of thought admitted knowing nothing about it, yet the answer confidently said experts and readers praise it. He says a Qwen model fine-tuned to answer "I don't know" about obscure people denied knowing him, then named him as author of his earlier book, showing such fixes are patches, not self-knowledge.

2025-03-07 · open the original ↗

• dbt's real value is documented, agreed business logic. In a clip from episode 861, a former pro athlete turned data engineer says dbt let her team embed business logic agreed with stakeholders, with field definitions (dbt docs, now auto-generated, plus reusable doc blocks) that force agreement on what terms like "active customer" mean and help new engineers. She describes Sigma, sitting on Snowflake, as easing adoption for Excel-minded PMs.
• LLM prompts need a compiler the way HTML needed React. Gupta, CEO of Boundary (episode 859), says BAML ("basic-ass machine learning") was inspired by React catching an intern's missing closing div that could break a news feed. Because every codebase will hold more prompts each year, he argues developers consuming models need type-safe tooling with instant error feedback, as TypeScript gave JavaScript.
• A tabular foundation model is spreading across scientific applications. Hutter (episode 863) and the host note v1 was already used in a Science paper on chemical proteomics, and an "awesome TabPFN" repo Hutter had just created lists about a dozen applications, including insurance, fault classification, finance, wildfire propagation and algal blooms. Version 2 handles more data types, missing data, outliers and larger datasets, and can be fine-tuned for domains its authors know nothing about.
• A mandatory paid discovery phase made Pandata scalable. Al-Dhubaib (episode 865) says returning clients valued plain speaking and a 30-day to six-week "discovery and design" engagement, so he made it compulsory, priced then at 12,000 and now 50,000. The pipeline briefly dried up, one of three times Pandata's bank account fell below a month's expenses, but he then landed three of his biggest clients, two of whom stayed about six years until the exit; his lesson is the art of saying no.

2025-03-06 · open the original ↗

• Executives' agent hype reflects their shareholders' interests. Against the host's list of boosters, Burkov says Nvidia's CEO talks up agents because it implies buying more GPUs, and that Salesforce once declared traditional software dead in favour of SaaS, which did not happen, and now says the same of SaaS. He distinguishes Andrej Karpathy as careful to describe possibilities rather than predict them.
• A single agent is a rebranded LLM, useful only in-distribution. Burkov's best case for agents is gathering and summarizing web information, because web data is closest to what LLMs were trained on; he questions how big that market is, likening some of it to a cron job running Google searches. Dropped into a company's intranet of legacy systems they have never seen, he expects agents to "break their teeth" quickly.
• LLM-based multi-agent systems are effectively undebuggable. Burkov, whose PhD was in multi-agent systems, explains that distributed systems are already hard to debug because you cannot pause all asynchronous actors at once, and LLM agents add black-box internals you cannot step through. Reliable multi-agent systems need full control of each actor and inspectable communication protocols.
• Agents need human-level trustworthiness before being combined. Burkov argues that even human systems guard against unreliability with redundancy, such as two keys turned far apart, and that today's agents are nowhere near as reliable as a person, so multi-agent systems built on them are "a recipe for disaster". He says no one has a clear idea how to reach a trustworthy AGI.