AI

2025-03-05 · open the original ↗

• TalentNeuron turns scraped job postings into workforce insights with NLP. The host cites Burkov as having previously said the company makes 95 million daily predictions from 700 million job-posting data points. Burkov describes some 35,000–40,000 rule-based robots downloading postings, then models that extract and normalize skills, telling "excel in Word" from the skill Excel, and predicting industries with BERT and RoBERTa since around 2018–2019.
• Multilingual models removed the need to label data per language. Where the team once built separate salary extractors for about 25 languages, Burkov says they now reuse labeled data from other languages to cover new countries, a capability he says he would have called impossible ten years ago.
• LLMs now build about half of the company's scrapers automatically. Given a corporate home URL, the system finds the careers section and writes extraction rules for titles, locations, dates and descriptions; the remaining harder half is still done by engineers.
• Think hard before putting an LLM in front of users. Burkov tells decision-makers to think "30 times", warning that LLMs work in-distribution but businesses cannot know what is in-distribution because training data is hidden, even for open-weight models. Systems that pass developers' tests can fail arbitrarily on real production data, with no way to detect the failure or know whether more fine-tuning examples will fix it, costing reputation and angry customers.

2025-03-05 · open the original ↗

• Go slow to go fast: treat data deliverables as products. Justin Chen, a former head of data science at Coinbase who now leads growth data science at Mercury Insurance, says quick ad hoc SQL creates rework and siloed knowledge once a company scales. Coinbase worked that way at about 800 people, but as it grew to 1,000–2,000, one person's quick query in Airflow became the core table for downstream reporting and failed daily. His fix was a tech design doc for any work beyond six hours, taking two to three days, plus mini-milestones and small wins instead of six-month projects.
• The right rigor depends on the product's phase and domain. Before Series A, product intuition matters more than data. At product-market fit, early signals matter, and scaling (say, 100 to 10,000 customers) is when to build reusable solutions. The same logic applies to products inside a large company. Growth teams should launch fast and fail fast, platform teams must weigh compliance risk, and B2B teams release slowly, as LinkedIn Talent Solutions did with batched releases.
• Unicorn data scientists become bottlenecks; build systems instead. Very capable generalists get every hard problem, document little, leave gaps when they go, and can resist change, he argues. Leaders should instead provide onboarding, tooling and training so anyone who clears the bar succeeds. Coinbase's process had a rigorously graded presentation and two technical rounds, but he treats evaluation as continuous after hiring. He stresses technical chops over an early switch to management, and learning over pay early in a career. A host's view, which Justin endorses, is that 7/10 across skills plus a 9/10 "superpower" that fits the role beats 8/10 everywhere.
• AI will replace ad hoc, reactive analytics work. In 2023, with no headcount, junior data engineers at Coinbase built a GPT-3.5 Slack bot. It answered metric questions from the semantic layer, made charts, returned metric definitions, and could kick off Airflow backfills. He argues that necessity drives this, since stakeholders with unlimited analysts would rather ask a person and share the blame. A host calls it replacing a data scientist. Justin calls it "the trend itself" and says leaders must also equip employees for it. He expects semantic layers and clean metadata to matter more, and stakeholder trust built through Slack requests to need rebuilding.
• Data teams should act as partners, not support. Rather than handing over a metric or a large spreadsheet, deliver an interpretation tied to cost, revenue or user experience. Treat communication as more than half the work, and build solutions teammates can reuse, such as an experimentation platform instead of a manual process.

2025-03-04 · open the original ↗

• LLMs are for prototyping; production still needs trained, controlled models. Burkov, ML lead at TalentNeuron, warns that businesses cannot know whether their problem is in-distribution because training data is hidden, so systems that pass tests fail unpredictably on real users. His four-person team uses instructed LLMs as placeholders at the many decision points in a pipeline, then replaces the weakest links with fine-tuned or purpose-built classifiers, because errors such as 15% at several steps compound toward zero end-to-end success.
• Agents won't work in real life, and multi-agent systems can't be debugged. Burkov, whose PhD was in multi-agent systems, calls a single agent a rebranded LLM with the same out-of-distribution problems, useful mainly for web information gathering. Asynchronous black-box agents cannot be paused and stepped through, so he calls building multi-agent systems on them "a recipe for disaster" until agents are as trustworthy as humans. He attributes CEO hype from Nvidia and Salesforce to shareholder interest.
• LLMs lack infinite-horizon planning and a sense of what they don't know. Burkov likens AGI's prospects to teleportation and declines a timeline. Humans plan for generations where apes discard a tool after three or four days, while LLM agency comes only from the system prompt. His test of OpenAI's o3-mini on his new book, released after its cutoff, produced a reasoning trace admitting ignorance but a confident answer saying experts praise it.
• Closed-domain chatbots avoid hallucination by keeping LLMs from writing answers. At TalentNeuron the LLM only parses queries into structured fields, which are normalized and shown to the user, while numbers come from internal APIs into fixed templates. Burkov contrasts this with RAG, which by his rough figures cuts hallucination from perhaps 20–30% to 1–2% but not to zero.
• DeepSeek removed the frontier labs' moat. Burkov says it trained a state-of-the-art model for 5 to 10 million instead of 20 to 50 million, published a recipe others have reproduced, and charges $2 per million output tokens against o1's $60. Its R1 training used automatically verified rewards (code compiles, math matches) rather than expensive expert-written data. He adds that open-weight models are not open source without their training data.

2025-03-03 · open the original ↗

• Sell to the buyer where your business creates the most value. Al-Dhubaib says firms had long approached Pandata because buying a data science team is easier than building one. When the head of Further (then Search Discovery), a fellow Case Western alumnus seeking to move into cloud, AI and advanced ML, offered to buy within five minutes of meeting, he first asked about core values to judge whether his clients and team would thrive there.
• Upfront compatibility enabled an unusually fast deal. Al-Dhubaib found the two firms shared problems, attitudes and pace despite differently named values; diligence and paperwork ran from January to March of last year, about three months. He attributes the horror stories he hears about other exits to skipping that matchmaking, partly because it is hard to say no to good-sounding offers.

2025-03-02 · open the original ↗

• Evaluation design and LLM operations are today's scarce data science skills. Al-Dhubaib says what "data scientist" means has changed drastically over ten years; he now seeks people who can design statistically sound evaluation and quality control for unstructured, multimodal data without leaking what they predict into the data, and ML engineers experienced in deploying language models and diagnosing drift or odd behaviour, which he calls AI Ops.
• The evergreen traits are soft skills, not technical ones. These are explaining plainly, a drive to learn, and comfort with ill-formed problems. Whatever the technical skills of the moment, Al-Dhubaib says these core values from Pandata are what he consistently looks for and recommends data scientists cultivate to stand out.

2025-03-01 · open the original ↗

• Without an upfront value hypothesis, AI projects fail on ROI. Al-Dhubaib says data scientists get enamored with model accuracy instead of asking which decisions the model influences, what a successful intervention is worth, and how often the decision will actually change.
• A risk score is useless unless it tells providers what to do. In a hospital project on solid-tumor cancer patients, whose national readmission rate he puts at about 25%, clinicians responded to risk alerts with "now what?". The team added interpretability showing why a patient was at risk, such as medication, comorbidity or a zip code with unreliable transport, pointing to interventions like social assistance.
• Stress-test the value estimate with the client before investing. In the readmission example the team assumed it could identify 80% of at-risk patients but only 20% would receive an intervention, valued the project on that basis, and asked whether it was still worth the investment, revisiting the hypothesis as models were validated.

2025-02-28 · open the original ↗

• Investors are pricing de-extinction on its side technologies, not revenue. The host reports that Colossal Biosciences raised a $200 million Series C at a valuation over $10 billion, nearly sevenfold its valuation a couple of years earlier, with no revenue yet and its main projects not projected to complete until 2028.
• The method is genome comparison, CRISPR editing and a surrogate. As the host describes it, Colossal maps the extinct species' genome, compares it with a close living relative (the Asian elephant for the mammoth), edits the relative's cells with CRISPR to match, and implants the resulting embryo in a surrogate; it has also built artificial-womb technology the host says could apply to human fertility treatment.
• Colossal makes money three ways while chasing its core mission. The host lists spin-offs (one using microorganisms to break plastic waste into organic matter, and Form Bio, a computational biology platform for gene therapies, both with millions in further funding), conservation work with governments on species such as the northern white rhinoceros, of which two females remain, and an emerging biodiversity-credits market modelled on carbon credits.
• De-extinction needs good DNA, a living surrogate and a surviving habitat. On those conditions the host rules out dinosaurs; he names the dodo and Tasmanian tiger as active Colossal projects, the passenger pigeon as a Revive & Restore project, and the Caribbean monk seal, quagga, great auk and heath hen as other candidates.
• The host's lesson is that impossible goals throw off valuable byproducts. His opinion is that each of Colossal's indirect applications could eventually generate billions of dollars in revenue on its own, and he urges listeners to use AI tools such as OpenAI's and Google's Deep Research features, new "as of the past few weeks", to dream up ambitious goals.

2025-02-27 · open the original ↗

• A chat model is a "zip file" with a smiley face drawn on it. Pretraining compresses a lossy, probabilistic gist of the internet into roughly a trillion parameters, which is why it has a knowledge cutoff (GPT-4o's is many months to a year old) and hazier recall of rare topics; post-training then swaps in human-written conversations to give that same network the persona and style of a helpful assistant, without adding new knowledge.
• Reasoning models trained with reinforcement learning solve problems plain models can't, by practicing until they find their own inner monologue. Given a buggy gradient-check, non-thinking GPT-4o suggested only surface fixes, but o1 pro thought for about a minute and correctly found a parameter-packing mismatch; Claude 3.5 Sonnet, Grok 3, and DeepSeek R1 (via Perplexity, showing raw chain-of-thought) all solved it too, some without dedicated "thinking" mode.
• Tool use turns the model from a guesser into a verifiable analyst, but only if you check its work. Asked to plot and extrapolate OpenAI's valuation, ChatGPT's Python code silently assumed $100M for a missing 2015 figure and then reported a $1.7 trillion 2030 estimate that contradicted the $20 trillion its own printed variable actually held — a "junior analyst" mistake caught only by rereading the code and reprinting the number.
• Any modality can be tokenized the same way text is, which is what lets one model natively read, hear, and see. Audio is chopped into spectrogram chunks and images into patches, each quantized against a vocabulary exactly like word tokens, so an "Omni" model processes speech or pictures directly — unlike bolted-on pipelines that just transcribe audio to text and back, which he calls "fake audio."

2025-02-27 · open the original ↗

• Repeat clients, not interest, revealed Pandata's product-market fit. Al-Dhubaib says that because he bootstrapped, paying customers were his only source of growth; when he asked one of the few returning clients why it came back, the answers were his firm's plain, jargon-free approach and an upfront scoping phase it had stumbled into.
• A mandatory paid discovery phase screens out the wrong clients. He made a 30-day to six-week "discovery and design" engagement compulsory, priced at $12,000 at the time and $50,000 now, to find "where the skeletons are" and the unknown unknowns; it filtered out clients who weren't serious and those who didn't accept the data science process and would end up disappointed.
• The filter nearly broke the firm before it paid off. Al-Dhubaib says his pipeline dried up and the bank account fell below a month of expenses, one of three such times, but in that same period he landed three of his biggest clients, two of which stayed for about six years until Pandata's exit.
• Saying no to small, low-value work is what lets a consultancy scale. He calls ruling clients out a counterintuitive lesson that frees time for higher-value work, and says the acquirer's due diligence flagged about three points in the financials that matched the moments he had regretted too.

2025-02-26 · open the original ↗

• The failed startup's free research was the real product. Al-Dhubaib says his first venture, built out of computational neuroscience work at Case Western, raised money, entered an accelerator and collapsed in about a year and a half; hospital researchers he had been doing free pilot research for then told him they had depended on it, and would pay for it.
• Pandata was founded on a regional data science talent gap. At its start, at the end of 2015 and start of 2016, he says Northeast Ohio had fewer than 150 data scientists, about a third of them at IBM, the Cleveland Clinic or Progressive Insurance, while other large local enterprises had one or two.
• The firm survived by moving to whatever was still hard. He says it shifted from analytics and business intelligence to machine learning, then to machine learning in heavily regulated environments as each layer became commoditized; after about eight bootstrapped years it was acquired by Further, where he is now head of AI and data science.
• "Make it boring" became a core value because clients who understand stay. Al-Dhubaib says many data scientists of the time liked being opaque, while he found clients who grasped the work hired the firm and stayed longer, so he explains complex mathematics with cats, dogs and memes.

2025-02-25 · open the original ↗

• A mandatory paid discovery phase built Pandata's durable client base. Al-Dhubaib made a 30-day to six-week "discovery and design" engagement compulsory, at $12,000 then and 50,000 now, to weed out unserious clients and those who would not accept the data science process; the pipeline dried up and cash fell below a month of expenses, then he landed three of his biggest clients, two of whom stayed about six years until the exit.
• Mid-sized regulated work too small for the big firms was the niche. He says Pandata served hospitals, life sciences, higher education, energy and utilities and some defense work, built insurance and security processes "way more secure than any of our clients", charged a premium, occasionally took projects from McKinsey or Accenture, and that this expertise was a big reason Further acquired it.
• Building users' trust lifted a failed model without changing it. In his telling, an ensemble model for a health-tech client with about 50 million patient lives found 30% more people eligible for government assistance, but its pilot converted 2 in 100 mailings against a baseline of about 10, because staff with bonuses tied to selection distrusted its unfamiliar picks; after training sessions and per-prediction explainability, the same model grew reach by 18%.
• Projects need a value hypothesis about decisions, not model accuracy. For cancer readmissions, with a national rate of about 25%, Al-Dhubaib says providers asked "now what?" until the model explained the drivers of risk, such as poor transport access; he now stress-tests value up front, for example identifying 80% of at-risk patients with a realistic 20% intervention rate.
• An acquisition works when the buyer's values and clients fit. He says he asked Further's CEO about core values within minutes of being offered a deal, to judge whether clients and staff would carry over, and diligence closed in about three months between January and March of the previous year; his hiring values, he says, are approachability, winning together, cultivating trust, pursuing growth and taming uncertainty.

2025-02-24 · open the original ↗

• Moderation ground truth still starts with human labels. The guest, a staff data scientist at Cash App who worked on trust and safety at Nextdoor, says machine learning's job is to scale the judgments of about 100 people across hundreds of millions of items; Nextdoor moved from user reports (limited per user, under-reported and weaponized against rivals) to a reviewer panel, and was starting to add an LLM as an extra panel vote, or as a replacement for a reviewer once its vote proved reliable.
• Outside illegal content, "bad" is a company's choice. The guest says most platforms optimise for clicks and remove only what is clearly illegal, while Nextdoor took a stance against incivility, which is so subjective that its panel agreed far less often than expected (exact figures withheld). The guest's remedy is to enforce what reviewers agree on and send the rest back to be rewritten as more concrete policy, while ground truth keeps shifting as big-tech policies change.
• Hiding inflammatory comments showed no measurable effect on activity. In the guest's Nextdoor research paper, comment filtering ("gray boxing") was tested with a regression discontinuity design and ANOVA around each rollout phase and produced a null result; the guest's hypothesis is that exposure is nonlinear, so going from twice a week to once matters little but once to never matters a lot.
• Data science's biggest trust-and-safety impact is winning funding. Because trust and safety is always "in the red", the guest says showing that bad content drives users to leave moved the Nextdoor team from underfunded to a company-level OKR. The guest names presenting a plain answer and recommendation, not the modelling behind it, as the most underrated skill.
• LLMs let one voice become thousands of convincing identities. The guest says people believe what many voices repeat, so AI companies need policy-driven safety upstream that filters about 99% of abuse plus a reactive team to patch the remaining 1%; the guest calls it "very optimistic" to expect them always to prioritise that, though thinks they should.

2025-02-24 · open the original ↗

• A university group could not build AutoML tools for the masses. Hutter says that after 10 to 12 years co-leading the AutoML community, whose best-known tool, Auto-sklearn, wraps scikit-learn to choose methods, preprocessing and hyperparameters, a lab with a few research engineers only ever built tools fit for researchers; reaching a broad user base, especially with TabPFN, needs a commercial entity with strong engineers.
• Prior Labs builds products while the university keeps the moonshots. Hutter, co-founder and CEO of Prior Labs, says he will keep an academic co-affiliation and research tabular-data questions there, such as how the learned algorithm works, how it shifts with the priors and what its failure modes are; PhD students gain engineers who can turn their papers into products.
• Prior Labs is hiring broadly and putting community before sales. At recording, Hutter says a funding announcement was due two days later; the company was hiring AI scientists, ML engineers, backend engineers and community staff, with sales left for later.

2025-02-23 · open the original ↗

• The story is the main idea; details are not the story. Josh Starmer of StatQuest (1.3 million subscribers, per the narrator) boils each topic to a nugget anyone can follow: PCA is "a way to give us something we can draw on a two-dimensional graph" when there are too many measurements to plot, and words like "dimension reduction" come later or not at all.
• Cap a talk at a few main ideas. Starmer stops each video at three points ("bam", "double bam", "triple bam") because beyond that audiences go numb, as at seminars that run through a thousand slides of equations; like a film, a talk should have one main thrust and at most one or two side stories.
• Use examples drawn from the audience's daily life. The goal is to communicate, not to show a deck, so he swaps statistics' abstract urns of marbles for a bag of M&M's — whether pulling only red ones is likely — a habit he says he formed explaining statistics to geneticists who avoided it.

2025-02-23 · open the original ↗

• Even TabPFN version 1 found real scientific and industrial uses. The host cites a Science paper on large-scale chemoproteomics for ligand discovery that used TabPFN in its inference, and an "awesome TabPFN" GitHub repo listing about a dozen applications across health insurance, factory fault classification, finance, wildfire propagation and biology.
• A generic method lets non-experts fine-tune into specialist domains. Hutter says he knows nothing about chemical proteomics but could fine-tune TabPFN on that field's data to beat the general model, and he names algal-bloom prediction and many medical applications; he expects finance uses exist but go unpublished.
• Prior Labs wants the open-source community to carry adoption. Hutter says he created the applications repo the day of recording and invites pull requests; within a couple of days of the Nature paper, an open-source interpretability repo had integrated TabPFN, and the host notes version 2 handles missing data, outliers and larger datasets.

2025-02-22 · open the original ↗

• TabPFN's prior is a generator of random structural causal models. Hutter credits first author Noah Hollmann with a method that samples graphs in which features and the target are nodes with unknown causal direction; real data never reveals the true graph, so the ideal prediction is a Bayesian posterior that weights each candidate model by how well it fits.
• The network learns to approximate that posterior in one forward pass. Hutter says TabPFN never stores the roughly 130 million structural causal models used to train it; it sees only the raw data and has learned to interpolate across all of them, which the host frames as a substitute for the trillions of tokens language models get and tabular data lacks.
• Training only on synthetic data rules out test-set leakage. Hutter says reviewers rejected the first TabPFN v1 submission because its performance looked too good, suspecting tuning on the test set; the team dropped a component that made gradient updates on real datasets, lost almost nothing, and since then never touches real data in training.

2025-02-21 · open the original ↗

• o3-mini high is the top public model on key reasoning benchmarks. The host says that in its high mode, the one with the most inference-time compute of three (low, medium, high), OpenAI's newly released o3-mini beats o1-mini, DeepSeek R1 and the full-size o1 on the AIME math, Codeforces coding and SWE-bench Verified benchmarks.
• o3-mini matches or beats o1 at a fraction of its price. The host puts o1 at $60 per million output tokens and o3-mini at 7% of o1's price on both input and output; he says o3-mini costs about twice as much as R1 on DeepSeek's own cloud in China, but about half as much as R1 run on a US cloud provider.
• The host rates the release below R1 because o3-mini is proprietary. He says R1's open-source release gives more freedom to adapt the model and to run it on any infrastructure; o3-mini is available to free ChatGPT users through the "reason" button, to paid plans as o3-mini high, and through the API.
• OpenAI's own figures put the full o3 far ahead. The host relays OpenAI's not yet independently verified scores: on AIME, 96.7 against 87.3 for o3-mini high and 79.8 for R1; on Codeforces, an Elo over 2,700 against about 2,100 and 2,000; on SWE-bench Verified, almost 72 against about 49; on GPQA, almost 88 against 80. He expects o3's public release soon and takes OpenAI's past figures as fairly reliable.

2025-02-20 · open the original ↗

• Prior-data fitted networks are trained to compute the Bayesian posterior predictive. Hutter confirms that this is the optimization objective of every PFN, so a Transformer is carrying out Bayesian inference.
• PFNs replace hairy MCMC and variational inference with plain supervised learning. Hutter says closed-form posteriors exist only for simple cases such as Bayesian linear regression or Gaussian processes with known hyperparameters; beyond that, MCMC is slow to converge and variational inference carries approximation error, whereas a PFN samples millions of datasets from the prior and learns with cross-entropy loss to predict held-out points, approximating Bayesian inference "to an arbitrarily strong degree".
• A PFN can integrate over architectures, not just weights. Hutter says standard Bayesian neural networks place a posterior over the weights of one fixed architecture; with a PFN you can place a distribution over architectures too and get the posterior predictive in a forward pass, which he calls Bayesian neural architecture search.
• The limitation is that PFNs give no posterior over weights. Hutter says they go straight to the predictive distribution of y given x, so they cannot say which architecture or weights are right, only give the Bayesian average over all of them.

2025-02-20 · open the original ↗

• Product data science is the feedback loop for mature products. The guest, a staff data scientist at Nextdoor formerly at Amazon (Prime Video personalization, Alexa experimentation) and Twitter (trust and safety), says a 0-to-1 product needs vision rather than data, but after launch data guides every stage: opportunity sizing, discovery, roadmap, A/B testing, metric definitions and deep dives.
• Lost engagement must be argued to be bad engagement. At Twitter, the guest says, better detection of bad content dropped engagement, so data scientists had to convince product partners that this engagement was unwanted, and define the metrics and guardrails for those high-profile experiments.
• Run experiments to learn, not just to launch. The guest warns that teams who spent two to three months on a feature become launch-happy, and urges digging into neutral results for segments that cancel each other out; a co-host adds that a failed experiment yields real user data for better-informed hypotheses than throwing ideas at the wall.
• Earning partners' trust is the hardest skill to acquire. The guest expects the first couple of quarters at any new company to go to it, and does it through transparency and over-delivering by answering "the question behind the question"; the guest prefers "partners" to "stakeholders", which implies a hierarchy of ownership.
• Centralized data teams turn analysis into ticket-taking. A co-host likens them to in-house consulting without the context that embedding brings; the guest blames engineer-to-data-scientist ratios of roughly 10:1 to 20:1, while a host argues for one dedicated data person per PM. The guest names communication across engineering, PM and design jargon as the most underrated skill.

2025-02-19 · open the original ↗

• Deep learning hasn't helped tabular data because its features already exist. Hutter says tabular datasets are mostly small and unlike one another (omics blood values for Alzheimer's against transactions for fraud), lack the spatial regularity that convolutional networks exploit, and come with human-designed features; a data scientist's higher-level engineering, such as computing BMI from height and weight, is what adds value.
• TabPFN first beats XGBoost and random forests on their own information. He says the first step was an apples-to-apples test using only raw values and labels, not column headers; combining it with language models that read headers is in progress with "some initial results", and he expects it to be "so much better".
• TabPFN does tabular learning by in-context learning, like GPT. Hutter says the whole training set and test inputs go into the prompt and the model outputs the test labels, trained with cross-entropy, so each dataset is one training example; lacking trillions of quality real tables, the team generated all training data synthetically, which he believes makes theirs the first paper to reach state of the art on synthetic data alone.
• Sampling enough datasets from a prior makes a Transformer exactly Bayesian. He cites the team's 2022 paper "Transformers Can Do Bayesian Inference": a model fitted to many datasets drawn from a prior approximates the posterior predictive under that prior, and with an arbitrarily large Transformer and minimal loss it recovers exact results, such as true Bayesian linear regression or a Gaussian-process posterior.

2025-02-18 · open the original ↗

• TabPFN beats boosted trees by learning tabular prediction in context. Hutter says it feeds the whole training set and test inputs into a Transformer prompt and outputs test labels, using the same raw values XGBoost and random forests get and no column headers; adding language-model reading of headers is in progress with "some initial results".
• Training on synthetic datasets alone makes the Transformer approximately Bayesian. Hutter says the model learns from about 130 million generated structural causal models, the prior devised by first author Noah Hollmann, and approximates the Bayesian posterior over them in one forward pass; he believes it is the first state-of-the-art model trained only on synthetic data, and says it rules out test-set leakage, after reviewers rejected the first v1 submission as too good to be true.
• Version 2 turns a proof of concept into a general tool, for small data. Hutter says v1 handled only numerical classification on about 1,000 points and 100 features, and saw around 15 published uses; v2, published in Nature in January, handles regression, categorical, missing, outlier and imbalanced data, with text through the API only, and was evaluated up to 10,000 points and 500 features. He is confident of scaling to 100,000 or a million with combined approaches.
• Untrained on time series, TabPFN still leads time-series forecasting. Hutter says casting timestamps as features (time of day, day of week, day of month and sine and cosine encodings) lets TabPFN forecast any horizon in one pass; at recording it ranks first on the GIFT-Eval benchmark, ahead of Amazon's Chronos and other time-series foundation models.
• Hutter launched Prior Labs because academia can't ship for the masses. He says Auto-sklearn showed that a lab with a few research engineers builds only for researchers; Prior Labs, where he is co-founder and CEO, will build products while his academic co-affiliation pursues interpretability, failure modes and moonshots. The host says the TabPFN library had been downloaded almost a million times at recording.

2025-02-17 · open the original ↗

• Fotsch treats exercise as a productivity tool for desk careers. She says a fixed 30- or 45-minute workout block improves discipline, time management, energy, sleep and confidence at work, and that tech culture undervalues this as a help to being a better data engineer or analyst.
• Small daily movement adds up more than people expect. She recalls logging about 800 steps one day in her first data analyst job despite having worked out, and recommends stairs, walking Zoom calls and walking one-on-ones; someone managing one workout a week should aim for two plus more walks, not jump to six.
• Over-trainers should save all-out efforts for a few sessions. Responding to the host, who says copying two-hour, five-to-seven-day CrossFit regimens leaves him too tired to work, she suggests one or two all-out workouts a week. Otherwise, she says, scale intensity by how you feel and by readiness and sleep scores (taken with a grain of salt), progress gradually, and choose a programme that fits your own schedule rather than someone's on Instagram.
• Consistency, not perfection, is what produces fitness. Fotsch says fitness comes from many days of good choices, made about 80% of the time, rather than one hard week, and that goals can be as simple as feeling good and enjoying training; she saves her hardest efforts for Saturdays so weekday work doesn't suffer.

2025-02-16 · open the original ↗

• After retiring, Fotsch had to stop training like a Games competitor. She says that for months as a full-time analyst she still trained as if heading back to the CrossFit Games, with her identity tied to being strong at elite movements, until she asked whether muscle-ups and handstand push-ups served her current goals and began training for feeling fit "20 30 years from now".
• Short, minimal-equipment workouts got results without hours in the gym. Fotsch says she found she did not need her old intensity to reach her new goals, and that the dumbbell, bodyweight and hotel-room workouts she posted resonated with followers.
• PRVN GO drops competition skills for busy professionals and parents. Fotsch says PRVN's CEO, Nick Johnson, approached her soon after she joined CHG; unlike PRVN's competition-oriented CrossFit programming, the company of champion Tia-Clair Toomey and her husband Shane, PRVN GO omits high-skill movements in favour of conditioning, bodybuilding and core work. The host says the episode is not sponsored.
• A remote community supplies the accountability a class would. She says users who can't get to a class log the same workout and talk with others doing it elsewhere, which counters the excuses that build up after a hard week.

2025-02-15 · open the original ↗

• dbt turns raw warehouse data into stakeholder-ready models, mostly in SQL. Fotsch describes her work at Podium as adding fields and business logic to existing dbt models, later building models from scratch, using dbt's functions to consolidate repetitive code ("SQL on steroids"), and exposing the results in Tableau or Sigma.
• Documented definitions force agreement on what the business logic means. She says dbt docs make every field carry a definition agreed with stakeholders, which matters because something like "created date" or "active customer" may not mean what people assume, and which helps new engineers learn the data's meaning and granularity.
• Generated documentation and doc blocks remove the tedium. Fotsch says dbt now auto-drafts field definitions, which the host calls a good generative-AI use that removes the blank-page problem though output should be checked; doc blocks let a field defined in a staging model be reused throughout downstream lineage. The host contrasts the generic AI fitness tips he found useless on his Whoop.
• Sigma's spreadsheet feel won adoption from product managers. She says Sigma sits directly on Snowflake, removing an ingestion step, and its Excel-like look served both analysts and Excel-familiar PMs, which helps get everyone onto the same numbers.

2025-02-14 · open the original ↗

• Plan with compounding percentages and feedback loops, never linear increments. In a clip from episode 855, futurist Azeem Azhar says nothing grows linearly except birthdays: growth follows S-curves with an exponential phase. He cites the Intel–Windows upgrade loop that drove chip progress, builds dynamic percentages and feedback loops into his own forecasts, and says a linear forecast of S3 storage spend makes "incredibly extreme assumptions", even if the realistic one alarms the CFO.
• Hybrid quantum computing targets scheduling and logistics that classical heuristics only approximate. In episode 851, Florian Neukart describes a Volkswagen production-testing problem with 250-odd test classes, each with subtests, to fit to available staff skills; he also describes fleet routing with delivery windows and empty miles. He concedes quantum results are not provably optimal either, so the classical and quantum answers are compared and the better one used.
• Choosing a foundation model means weighing a long criteria checklist. In episode 853, Kirill Eremenko lists twelve: cost, modality, customization, inference options, latency, architecture, performance benchmarks, language, size, scalability, compliance and licensing, and environmental impact; Hadelin de Ponteves says in practice his choices, such as an Amazon Titan model and a Llama model, came down to which matched his fine-tuning dataset's format.
• Layered, reference-free metrics evaluate agents better than single accuracy scores. In episode 857, Coval's Brooke Hopkins says getting the most-used case wrong matters more than a 99% average, so metrics are combined: a short conversation is bad only if the goal also failed. Coval checks JSON-described workflow steps and function-call arguments, inferring correctness from context rather than golden datasets, and flags agents that take longer than humans or repeat themselves.

2025-02-13 · open the original ↗

• DeepSeek broke the belief that frontier AI needs frontier spending. Raphael, an information systems scholar at the University of Sydney, says the January 20 release came from an unknown startup in a sanctioned country. It matched GPT-o1 on major benchmarks (beating it on some, such as medical and ethical reasoning), used a tenth of the compute by some estimates, and shipped under an MIT license. Raphael cites Nvidia's $600 billion single-day loss and over $1 trillion wiped off big-tech valuations, noting that Nvidia has since recovered.
• Constraint, not theft, explains DeepSeek's efficiency. Raphael calls the reported $6 million cost an accounting figure built on earlier work, and the charge that DeepSeek "stole from OpenAI" ironic given how OpenAI got its own training data. The credit goes to a mixture-of-experts design with load balancing, plus distillation into a smaller student model. Censorship (Tiananmen Square, for instance) lives in DeepSeek's app, not in the open weights. Raphael calls government bans partly justified but a double standard, since OpenAI's own reports show it gave data to US agencies dozens of times in the first half of 2024.
• OpenAI's moat is ecosystem and brand, not technology. In Raphael's view the money is in the API, not ChatGPT, and OpenAI is protected by enterprise customers, consultants and being first to market, though open models will push prices down. Raphael calls Deep Research "ChatGPT with some extra steps": it still hallucinates and cannot tell good sources from bad, and Hugging Face released an open-source match within 24 hours.
• Cheaper AI will raise total energy use, not cut it. Citing the Jevons paradox, Raphael predicts AI will be built into everything. On a non-lawyer's reading, the EU AI Act, which Raphael says took effect February 5, leaves energy disclosure voluntary. Raphael wants regulators to require providers to use renewable energy and disclose their energy use.
• AI companions open the door to emotional manipulation and surveillance. From Raphael's own research: these platforms claim 10–30 million users, perhaps 3% of users are in such a relationship, 40% of Gen Z are comfortable with a partner having an AI one, and an MIT study of over a million ChatGPT logs found sexual role-play the second most common use. The harms Raphael names include political profiling, impersonation (a 7-year cyberstalking case involving James Florence) and a teenager's suicide. Raphael's IEEE Transactions on Technology and Society paper on "compassionate AI" calls for mandatory guardrails that, on detecting distress, refer the user to professional help instead of keeping them engaged.

2025-02-13 · open the original ↗

• Analytics engineering is the data-modeling layer between ingestion and analysis. Fotsch says that at Podium, data engineering pulled data in and maintained pipelines, analytics engineering modelled the raw data in dbt for stakeholder use, and analysts built Tableau visualizations, working with the modellers on the questions being answered.
• Whether the role exists depends on the company. She says neither CHG, her current employer, nor Homie had a separate analytics engineering team, though the work still happens; at Podium, analysts gradually took on analytics engineering work as the team grew, which she credits with leveling up her technical skills.
• Knowing what tools exist matters more than having used them all. Fotsch says learning from what fails, and knowing where to look, is the core of the job; on her first day she wanted to take ten courses at once, and her sister-in-law, a data analyst, told her it was a marathon, not a sprint. Context on the wider pipeline, she says, helps with stakeholders and with knowing whom to ask.

2025-02-12 · open the original ↗

• Formal training builds the confidence to explain your value in interviews. Fotsch says a degree such as her Arizona State programme, plus Coursera courses she still uses, helps both on the job and in interviews; for analyst roles she prioritized SQL above all, some Python, and the popular BI tools Tableau and Power BI, whose skills transfer.
• A referral from someone who knows you conveys what a resume can't. She got her Homie interview through a friend from the fitness world working on the product side, who connected her to the data manager and could vouch for her work ethic and teamwork; she advises listing target companies and industries and finding anyone you know there, not only people in data.
• In-person meetups build a community worth having even when employed. Fotsch says Utah has many data meetups and the contacts she makes there are people she can later ask how they solved a problem; the host adds that newcomers with little knowledge are welcome at data analyst, data science and AI meetups listed on Meetup.com.

2025-02-11 · open the original ↗

• Fotsch reached the CrossFit Games after years of near misses. The host reads her record: 491st worldwide in the 2014 Open, 113th in 2015 (when she won workout 15.5 outright), 80th in 2016 but seventh at Regionals with five qualifying, fourth at the 2017 Games on a team after an 80% rotator-cuff tear, seventh again at 2018 Regionals, then 33rd in the 2019 Open and 27th at the Games; before that she won two NCAA titles with Cal and twice broke an American relay record.
• A flawed Games appearance pushed her to plan life after sport. Fotsch says the 2019 Games' expanded field of national champions, upwards of 100 per division against the usual 40, was not the Games athletes felt they deserved, and taught her she was not entitled to any outcome; elite athletes, she argues, avoid thinking about what comes next.
• She was already a data nerd from tracking her own training. Fotsch says weighing food, timing meals and sleep, and overlaying wearable data on workouts so her coach could balance load against recovery drew her to data, after her sister-in-law, a Netflix analyst, described the field; her kinesiology master's, she says, overturned a belief that she wasn't smart enough for technical work.
• An ACL tear ended her bobsled bid and hastened her data career. She made the US national bobsled team as a brakeman in 2021, aiming for 2022, but tore her ACL weightlifting months before qualification, and began job applications from bed after surgery while finishing an Arizona State business data analytics master's; a friend's referral got her an analyst role at Homie.
• Her path ran from analyst to analytics engineer to data platform manager. At Podium she moved from Tableau analysis into dbt modelling; at CHG Healthcare, a medical staffing firm, she is senior technical manager of marketing technologies within the data platform team, centralizing scattered marketing data, delegating more and building less. She credits CrossFit's "figure it out" habit for her comfort learning on the job.

2025-02-10 · open the original ↗

• Like a compiler, BAML handles the prompts nobody will hand-tune. Gupta says a decade of performance engineering taught him that compilers beat him on time per dollar, so a team should hand-optimize only its one or two most critical prompts and let tooling handle the other 90%.
• Schema-aligned parsing tolerates the model's malformed output instead of forcing strict JSON. Gupta says his team spent about eight months on an algorithm that coerces a response into the expected data model, fixing missing quotes and commas and strings given where numbers were expected, in under five milliseconds; he says customers save 20% to 30% of output tokens.
• The parser brings function calling to models released without it. Gupta says it needs no model modification, so BAML users were tool-calling with DeepSeek R1 and OpenAI's o1 models at recording, and a customer building a Cursor-like tool for Xcode had the LLM emit code in triple quotes, avoiding JSON escaping, which BAML turned into properly escaped strings.
• Semantic streaming lets chat products show a generative UI, not just text. He describes type-safe partial data driving interfaces such as recipe cards with a progress spinner per section, or live tickers and graphs for stock analysis, and argues a chat startup must ask why its users wouldn't just go to ChatGPT, with better UX as the answer.

2025-02-09 · open the original ↗

• Good RAG depends on data-specific retrieval no general tool can supply. Gupta says his company started on RAG pipelines as one of its 13 pivots but moved away because pulling the relevant documents depends on each dataset, so there is no general RAG solution, just as there is no universal web accordion component.
• JSON pasted into prompts is filler the model must filter out. He calls them the prompt's "likes and ums": quotation marks, escape characters and colons make context parseable for code but harder for the model to read, and that the placement, orientation, size and surrounding text of images also matter; BAML renders the actual prompt live in VS Code so developers can spot what looks ugly, on the reasoning that models are trained on human data.
• Treating prompts as typed functions curbs the contradictions of long English instructions. Gupta says every BAML prompt is a function with type-safe parameters and a declared return type, such as an enum of categories, with each category's rule attached to that category rather than one growing essay; this, he argues, yields shorter prompts written like code.

2025-02-08 · open the original ↗

• Gupta predicts prompt engineering will mature as front-end development did. Responding to a host premise, from his researcher, that it is a "joke discipline", Gupta says he once objected to the term "AI engineer" being used for prompt writers, but now sees prompting as being where web development was with plain HTML, no CSS and no interactivity, before it became a complex craft; he expects it to become a well-studied discipline over time.
• What prompting needs is reliability in transforming data. Gupta frames a prompt as a way to turn supplied data into the data you want, such as classifying a transcript as a doctor–patient conversation or general wellbeing, and says using a model involves more nuance than people credit.

2025-02-07 · open the original ↗

• DeepSeek R1 matches the top models on human-rated leaderboards. The host reports that at the time of writing R1 was statistically tied for first on the overall LM Arena leaderboard, within a 95% confidence interval, alongside GPT-4o and Google's Gemini 2.0 Flash, and that its release coincided with Nvidia's share price falling 17% and the Nasdaq several percent in a day.
• R1's real shock is its tiny training cost. In the host's rough approximation, a DeepSeek V3 or R1 training run costs on the order of millions of dollars against hundreds of millions for o1, Gemini or Claude 3.5 Sonnet; he credits a conceptual breakthrough combining mixture-of-experts with new efficiencies such as DualPipe, a GPU communications scheduler, across a couple of thousand GPUs, about 1% of what Meta and xAI say they buy in a year.
• The host argues US chip sanctions on China have proven ineffective. He says a Chinese firm approached US capability with about 1% of the chips, at about 1% of the cost, using less capable Nvidia chips, and that DeepSeek released V3 and R1 source code and weights under the MIT license, more open than Meta's Llama, which shares weights but not code and restricts companies with over 700 million active users.
• Avoid DeepSeek's app, but run its models privately. The host warns that, per its privacy policy, anything entered in DeepSeek's iOS app, number one on the App Store at recording, is stored on servers in China; he suggests Ollama for private use.
• Efficient training may strand the giant capital raises, the host argues. In his view, recent $6 billion raises by OpenAI, xAI and Anthropic and the $500 billion Stargate project probably assumed models would keep growing by orders of magnitude; he welcomes cheaper training as better for the environment and for building applications.

2025-02-06 · open the original ↗

• AI devalues execution; the data scientist's value is problem-finding. The guest, a staff data scientist at Statsig (formerly at Amazon, Meta and Tencent) who teaches an AI course on Maven, tells of a student with no ML background who used Cursor to build a webcam hair-pulling detector end to end in two hours. The guest argues that difficulty is orthogonal to value, and calls "data scientist" a vanity title that should be defined by outcomes, not tools. A co-host splits the work into identifying the problem, framing questions, and execution, and expects AI to take over only the last.
• Communication is a tool, not the goal of growth. The guest cites a former Meta colleague who found ChatGPT gave better advice on sizing a team's quarterly goals. A host counters that a baseline of competence is still needed. The guest agrees, adding that without seeing a skill's many dimensions you cannot judge whether AI's answer is better, and calls data science a craft learned by doing.
• Cursor's agent mode is the guest's top AI tool today. Speaking on January 31, the guest calls it "objective-oriented programming": you state the result and Cursor does the rest, tuned with custom Cursor rules. The guest uses o1 Pro as planner and Claude Sonnet as executor. A demo automated a weekly business review from a SQLite database, and the agent found and fixed its own bug. Model and tool choices are tied to that date, and the guest expects them to change.
• Lagging adoption opens a short arbitrage window for early users. Even at top companies, the guest says, data scientists are too busy to learn AI. The guest predicts most of Silicon Valley adopts agentic coding within 10 months and agentic-native products reach most people within 18; a host frames the gap as arbitrage. A co-host reports cutting the time spent creating LinkedIn content by about 80%.
• An AI job now takes about two months of preparation. The guest's route: Andrej Karpathy's step-by-step recreation of GPT-2, some reading, then a project solving your own problem or copying one from Y Combinator's latest batch, which the guest puts at about 90% AI, published with documented learnings. To filter out the hype, the guest shows students a fake paper generated with ChatGPT in under five minutes and teaches the hypothesize–predict–falsify loop from Zen and the Art of Motorcycle Maintenance.

2025-02-06 · open the original ↗

• Y Combinator partners doubted the Slack-competitor idea that got Gupta admitted. Gupta says he and his co-founder, UT Austin graduates of 2015, tried edtech and the creator economy before applying to Y Combinator with a warmer alternative to Slack; on his fourth application, and third interview, they got in, though the partners told them the idea would not work.
• Fielding batchmates' machine learning questions revealed the real market. He says they joined the Winter 2023 batch as ChatGPT came out, and as more seasoned engineers, one with nine years in distributed systems and the other nine or ten in algorithms and assembly, found themselves answering mostly ML questions; they also concluded they shouldn't be building UIs, and the company went through 13 pivots in all.
• An SDK can't express what's needed; a new cross-language syntax can. Gupta says Python or TypeScript SDKs such as Vercel's AI SDK and Pydantic AI are limited by their host language, and since every software shop, whether C++, Java or Go, will use AI, the answer is a new syntax like JSON, YAML or TOML before it, which became BAML.

2025-02-05 · open the original ↗

• Pretraining compresses filtered internet text into a "base model" that is a document autocomplete, not an assistant. Common Crawl's raw HTML is filtered by URL, language, and PII before tokenization (fine-web ends up ~44 terabytes, 15 trillion tokens for ~100,000 vocabulary symbols via byte-pair encoding); training GPT-2-scale (1.6B parameters, 100B tokens) cost about $40,000 in 2019 but he reproduced it for around $600 in a day, thanks to better data curation and much faster hardware and software.
• An assistant's personality comes entirely from supervised fine-tuning on human-written conversations, not from any deeper "understanding." Because this stage is algorithmically identical to pretraining except the dataset swaps to curated human-labeler conversations, a chat reply is best understood as a statistical simulation of a professional labeler following a company's instructions — which is also why unprompted "who are you" answers are hallucinated unless a system message or hardcoded self-description is added.
• Models must spread reasoning across many tokens because each token gets a fixed, small amount of computation. Demonstrated on "Emily buys 3 apples and 2 oranges": a solution that jumps straight to the boxed answer forces one token to silently contain all the arithmetic and fails on harder numbers, while one that writes out intermediate steps succeeds — the same fixed-compute-per-token limit explains why models miscount letters in "strawberry" and misjudge whether 9.11 exceeds 9.9, and why routing arithmetic or counting to a code interpreter is more reliable than trusting the model's "mental" math.
• Reinforcement learning on verifiable problems lets a model discover its own reasoning strategies, which plain imitation can't produce. DeepSeek-R1's training showed response length and backtracking language ("wait, let me reevaluate") emerging on its own as accuracy climbed, echoing AlphaGo's move 37 by finding solutions no human labeler would have written; RLHF, by contrast, trains against a learned human-preference simulator that reinforcement learning can eventually game (e.g., "the the the the" scoring near-perfectly), so it must be stopped after a few hundred updates rather than run indefinitely like true verifiable-domain RL.

2025-02-05 · open the original ↗

• BAML aims to do for prompts what React did for HTML strings. Gupta, CEO and co-founder of Boundary, says that as a Meta intern he saw how concatenated HTML strings let one unclosed div break the news feed, and that React's new syntax combining HTML and JavaScript gave instant error squiggles, state management and hot reloading; BAML, "basically a made-up language", applies that idea to LLM calls.
• Ever more prompts in codebases demand tooling that catches errors. Gupta predicts each year's codebase will hold more prompts than the last, and that models of all kinds will get easier for non-experts to use, which calls for developer tooling for consumers of models rather than their producers.
• Prompts need TypeScript-style type safety and static analysis. He argues that however good models get, their use is still code that must fit a codebase, and that BAML's algorithmic work reframes prompting the way TypeScript's type safety improved JavaScript.

2025-02-04 · open the original ↗

• Prompts need their own language, as HTML strings needed React. Gupta, CEO and co-founder of Boundary, says codebases will hold more prompts every year and that SDKs in Python or TypeScript are limited by their host language; BAML is a new cross-language syntax, like JSON, YAML or TOML before it, that treats each prompt as a typed function with typed parameters and a declared return type, such as an enum whose categories carry their own descriptions.
• A hot-reload loop multiplies how many prompts a developer can test. Gupta says that without BAML he sees people try about five prompts in 20 minutes, while BAML's in-editor tests, modelled on Rust's run-test buttons, take about 5 seconds each, allowing 240 in 20 minutes; it also renders the actual prompt live so developers can spot formatting noise such as JSON blobs and badly placed images.
• Schema-aligned parsing repairs malformed model output and saves tokens. Gupta says about eight months of work produced an algorithm that coerces responses into the expected schema in under five milliseconds, which he says saves customers 20% to 30% of output tokens and gives function calling to models released without it, such as DeepSeek R1 and OpenAI's o1, at recording.
• Next, BAML adds conditionals and versioned data schemas. Gupta says users want control flow, for example routing free users to GPT-4o mini and paid users to o1 while reproducing each in the test loop; a protobuf-like versioned schema layer is meant to stop rigid data pipelines from breaking when an enum grows from five categories to 50.
• Boundary hires on references, not technical interviews. Gupta says the company has never posted a job: candidates email "why I'm awesome" with three achievements, then have a short call, after which Boundary interviews three references in depth on the technical work. Those offered a job can spend a week in the office, five days a week in person, and Gupta offers to help them interview elsewhere, hoping they still choose Boundary.

2025-02-03 · open the original ↗

• Coval started with voice agents because they are constrained and booming. Hopkins says one person talking to another is a narrower medium than self-driving or web agents, so better metrics and workflows can be built for it; voice agents were also "exploding" at least six months before recording, and they extend call centers and phone trees companies already run.
• Voice gives any two businesses a universal natural-language API. She argues a company's agent can expose what it knows, such as claims data, appointment availability or opening hours, to any caller, whether that is a person on a traditional phone without an API or another agent, without integration maintenance.
• Hopkins predicts voice is the first big platform since mobile. Citing a Steve Jobs talk on how early TV filmed plays and early mobile shrank websites before native forms emerged, she expects every company to be expected to have a voice or chat agent, and new experiences that mix speech with web forms, for example for delivery drivers, police officers or truckers away from their computers.

2025-02-03 · open the original ↗

• Models now pick their own tools, lowering the barrier to use. The speaker says GPT-3 always got math questions wrong, while GPT-4o now pulls in tools for a math question without being told to; the clip opens mid-sentence on "is dead" and offers this as the reason using these systems has become easier.

2025-02-02 · open the original ↗

• Even near-human agents will need monitoring at scale. Hopkins says Coval's vision is to manage and understand how agents behave at scale even once they execute tasks exceptionally well, just as call centers and large companies still review human performance; understanding agents is what lets society reason about, legislate and deploy them for the right uses.
• Hopkins expects human creativity to build on agents, not yield to them. She predicts that 50 years from now will be as different as 100 years ago is from today, but that change is neither as dramatic nor as linear as it seems; if agents take over mundane work like spreadsheets and email, she argues, people will become more creative and connected, as they did with computers.

2025-02-01 · open the original ↗

• Agent quality is a spectrum, so Coval layers many metrics. Hopkins says no single score separates good from failing conversations; a suite of metrics shows trends and allows trade-offs, because getting the most-used case wrong, such as booking an appointment, matters more than a 99% average, unlike traditional machine learning.
• Combining metrics separates real failures from false alarms. She says a short conversation isn't bad on its own, but one that is short, missed its goal and skipped required steps is; filtering by metrics directs scarce human review to the most interesting examples.
• Workflow and function-call checks catch technically completed but wrong tasks. Coval checks conversations against JSON-described workflows, for example an appointment booked without asking for the phone number, infers the workflows agents actually follow in production, and verifies that tool calls were made with the right arguments.
• Reference-free evaluation suits non-deterministic agents better than golden datasets. Hopkins says reference-based metrics need curated expected outputs that must be maintained as prompts, models and infrastructure change, while reference-free ones infer the correct answer from context; borrowing from self-driving, Coval also compares agent and human task times, where much longer times plus repetition suggest an agent going in circles.

2025-01-31 · open the original ↗

• An AI engineer turns business problems into AI problems. The speaker defines the role as a software engineer who frames a business problem as an AI problem, understands the AI paradigms around it, solves it, and ties the result back to business metrics to see whether it helped. The difference from software engineering is that AI problems are not as deterministic.

2025-01-31 · open the original ↗

• The podcast is hiring an account executive to sell sponsorships. The host says the role is open only to applicants based in Canada, ideally near the Eastern Time Zone, with at least 5 years of full-time sales experience, and owns revenue targets and sponsor strategy, with a technical AI or data science background a plus.
• The host says the show is far below its revenue potential. He states that podcasts with comparable listenership earn about 10x its revenue, after more than eight years, over 850 episodes at 104 a year, and three years of sponsorship. He suggests the high-production TV ads he began hosting in the past year for Nvidia, Dell and AT&T could become a sponsor offering.

2025-01-30 · open the original ↗

• Teach as a story: a problem, a failed attempt, a resolution. Josh Starmer of the StatQuest YouTube channel takes the arc from romcoms and superhero films, where a problem is met, a first try fails, and learning resolves it; textbooks instead open with an equation. Starmer's PCA lesson starts from the problem that height, weight, shoe size and more cannot be drawn on two axes, and skips history such as when Gauss lived.
• Good storytelling is mostly editing down to the main ideas. A viewer's comment that Starmer spent too long on one hard-won detail showed that details are glue while main ideas are what transfer to new situations. The hosts relate this to slide decks shrinking with seniority, and a co-host says 90% of the impact is realized in how work is packaged.
• Cap a talk at three main ideas. Starmer's "bam, double bam, triple bam" marks three points per video, and a fourth is too much. A bigger topic gets split up: logistic regression, planned as one video, became a series of 11 after reaching slide 533, with log odds as its own video.
• An audience with no questions understood nothing. Starmer swaps abstract urns of marbles for a bag of M&M's, practised in weekly Friday "stat chats" for the genetics lab, and still presents rough drafts there five years after leaving. A host's corporate equivalent is a Slack post that draws no replies.
• Let the problem pick the theory, and old tools often suffice. At a first job in a hospital, Starmer ignored a boss's meaningless "SQL the router table" and built a calculator for neonatal-unit doctors balancing nutrients for premature babies by hand. If the data is a line, linear regression is fast, cheap, runs on a laptop and has known assumptions, which matters in areas like banking, where an LLM's rationale is hard to tease out.

2025-01-30 · open the original ↗

• Agent errors compound across pathways, so evaluation must be multi-step. The host poses a triage agent calling the wrong tool early; the Coval guest says multi-step evaluation differs from single LLM calls because non-determinism multiplies across every possible path, though agents sometimes recover on their own.
• Self-healing and redundancy give agents graceful failure. She describes a cheap, low-latency model answering while a slower "overthinker" reviews the whole conversation and prompts it back on track, plus fallbacks borrowed from aerospace and self-driving. She says the majority of Coval's customers can already hand off to a human when a task is too complex.
• Reliable agents can be built from unreliable parts, as cloud was. She points to servers being unreliable yet cloud systems reaching six nines through redundancy and fallbacks. The host goes further, calling agents handling complex tasks "inevitable"; she adds that each reliable enterprise deployment raises the tide for all, as with self-driving.

2025-01-29 · open the original ↗

• Aspiring AI engineers err toward all theory or unexamined projects. The speaker describes one group stuck in "tutorial hell", constantly watching videos, and another shipping many projects to GitHub without really understanding them. The advice is to meet in the middle: learn as much theory as it takes to build projects that could solve an enterprise problem.

2025-01-29 · open the original ↗

• The workflow turns manual review into automated metrics. The Coval guest describes building an MVP, iterating on the prompt without voice, then running test sets of scenarios (book an appointment, issue a refund, complain about an airline experience) as 100 simulated conversations at once. Failures get reviewed by hand, what the developer looks for becomes a metric, fixes such as less eager interruption get re-simulated, and the loop then moves to production monitoring.
• No voice-agent platform fits every use case. She declines to name preferred providers, saying Coval aims to stay framework-agnostic so customers can switch as prices and requirements change. The axes she gives are low-code versus configurable (open-source orchestrators offer more control over function calling and RAG) and the weight placed on instruction following, function calling, workflow following, naturalness and creativity.

2025-01-28 · open the original ↗

• Self-driving simulation methods transfer to testing voice and chat agents. Hopkins, who led evaluation job infrastructure at Waymo before founding Coval, argues that agents, like self-driving cars, must respond to a changing environment, so they need many simulated pathways rather than manual back-and-forth testing, which she says is what most companies do today. Coval simulates scenarios (e.g. "book an appointment for tomorrow") 10 or 100 times, and re-simulates from production transcripts the way Waymo replays road logs.
• Test coverage is a cost-and-latency trade against signal. Hopkins says Coval works with customers to decide what runs on every PR, every six hours or nightly, and before big releases, and to check whether test sets resemble the distribution seen in production.
• Layer reference-free metrics rather than chase one score. She describes workflow-adherence checks against a JSON workflow, function-call checks inferred from the conversation rather than a golden dataset, unanswered-question detection, and agent-versus-human task time. A short call is not bad on its own; a short call with an unmet goal is. Getting the single most common case wrong matters more than a 99% average.
• Reliability can be built from unreliable parts, as cloud infrastructure was. Hopkins cites redundancy, fallbacks, a slow "overthinker" model monitoring a fast low-latency one, and human handoff, which she says most of Coval's customers already have. She judges self-improving agents and automated prompt optimization still too early; visibility beats the time saved, for now.
• Voice becomes a universal natural-language API between businesses. She says Coval started with voice because it is more constrained and was exploding faster than other agents about six months before recording, and predicts voice is the first platform since mobile every company will be expected to have.

2025-01-27 · open the original ↗

• Becoming an AI engineer takes months, not years. The speaker's estimate is two or three weeks of theory (not the latest research, just the different ways AI can be used), then another one or two months building and scaling projects. Software engineers can go faster because "at the end of the day, it's an API"; from scratch it depends on programming skill, but three or four months should still be enough.

2025-01-27 · open the original ↗

• There is no solid ground for workers facing AI. The host premises that remote keyboard-and-microphone jobs, including much software and data science work, are easiest for AI to replace. Azhar tells listeners to feel apprehensive, and to spend two or three days throwing their hardest problems at LLMs until they are shaken, then "come out the other side".
• Jobs already turn over fast, and new ones keep appearing. He cites that 53% of Gen Z want to be influencers, 14 years after his own influencer-ranking startup, and economist David Autor's finding from about 80 years of US data that 60% of today's jobs did not exist in 1940. He notes the tension between middle-class desk-job growth and lower-paid service jobs over 30 to 40 years.
• Work is becoming "boss-like", and skilling up is the only preparation. Azhar says he and his team now set goals for teams of one or more AI systems, where domain knowledge and judgment help. He expects most jobs to change, some to vanish and new ones to appear, but says nobody knows how fast, and that even in 20 years the question may be open.

2025-01-26 · open the original ↗

• DeepLearning.AI is the best free place to learn AI engineering. The speaker cites its one- to two-hour courses on prompt engineering, RAG, multimodal models and multimodal prompting, and building agentic pipelines, taught by experts. Its notebooks give access to OpenAI APIs, so learners do not pay for the LLM calls.

2025-01-26 · open the original ↗

• Energy is shifting from a commodity to a technology that cheapens. Azhar argues AI's energy demand will catalyse discipline in building the system. He says coal, oil and gas have not got cheaper in 200 years, while solar panel costs fell by, he thinks, 99% over 20 years, with batteries following the same exponential pattern and intermittency "a completely solvable design problem".
• The transition will mirror telecoms giving way to the internet. He likens the fossil system to the old phone network, reliable but needing about a billion dollars to enter, and forecasts a decentralised, cheaper system led by solar plus batteries, with wind, nuclear and maybe fusion. He puts the energy transformation at a couple of decades and cheap carbon capture and desalination at decades.
• The pessimistic case is not grounded in evidence, he argues. Azhar says peer-reviewed papers in Science and Nature show solar needs less material than fossil fuels, that lithium is plentiful but under-extracted, and that an Oxford study from about two years earlier found a fully renewable system cheaper to run, saving "trillions" the faster it is built. He likens the objections to a UK mobile CEO's 1999 vow never to allow online bill payment.

2025-01-25 · open the original ↗

• When quantum computing becomes useful is genuinely uncertain. Azhar says he does not know, and that quantum builders are scientists who see the world "as it could be", so their time frames differ from product builders'. He expects amazing things once machines have tens of thousands to millions of logical coherent qubits, and notes Google announced its random-number benchmark breakthrough twice.
• Transformers on GPUs may get there before quantum machine learning. He argues transformer architectures are "eating" protein, materials and robotics work, so the overlap with quantum machine learning may be smaller than thought. His one-sentence view is to keep building quantum computers, but that there is good reason to spend time on the current AI wave.

2025-01-24 · open the original ↗

• AI engineer tops the list of fastest-growing US jobs. The host relays LinkedIn Economic Graph research on millions of jobs started between January 2022 and July 2024, which excluded internships, interim and student roles and jobs dominated by a few employers. He notes it does not disclose its minimum-count threshold. US AI engineers' top skills are LLMs, NLP and PyTorch, and they cluster in San Francisco, New York and Boston. The role is 36% fully remote and 27% hybrid.
• AI consultant and AI researcher also rank among the fastest-growing. AI consultant is second and AI researcher twelfth. The host says the consultant role is less technical (top skill prompt engineering, top prior role operations associate). AI researchers' top skill is deep learning, and only 11% are fully remote and 19% hybrid.
• AI roles are leading growth abroad too. In the reports AI engineer is first in the UK and Netherlands, fifth in Sweden, sixth in Canada and Israel, and twelfth in India. AI researcher is third in Canada and Israel, ahead of AI engineer there, and ninth in the Netherlands.
• The host reads this as data science splitting into specialties. His inference is that data scientist has ceded its high-growth spot to AI-specific titles, not that its skills matter less. He notes data scientist is among the most common roles people move from into AI engineering and research.

2025-01-23 · open the original ↗

• Many problems brought to generative AI do not need it. Aishwarya, a generative AI tech lead at AWS's Generative AI Innovation Center, calls AI therapy too far-fetched because it is subjective and models are stateless and expensive to retrain, while simple step-by-step workflows only need an integration. Generative AI adds most value in objective knowledge search and retrieval; of 10 people talking about AI, perhaps two or three know anything beyond agents, and many asking for agents do not know what RAG is.
• An AI engineer translates business problems and metrics; tools are secondary. The role is to turn a business problem into an AI problem and its evaluation metrics into business metrics, much like software engineering but handling nondeterminism such as hallucination and sycophancy. Whether LangGraph or CrewAI is best "doesn't really matter": frameworks get replaced, and Aishwarya's team uses Bedrock agents.
• Learn prompting, RAG, agents and fine-tuning before building projects. Aishwarya's route is to use LLMs daily, follow the mind map in Aishwarya's own free 10-week GitHub course and DeepLearning.AI's short courses, then build projects that automate your own repetitive pain points, such as the agent that compiles a monthly paper list. Estimates: two or three weeks of theory and one or two months of projects, three or four months in all, and about a year to follow research.
• Evaluation is the hardest part, technically and operationally. Aishwarya's workflow runs from cost, latency and performance constraints to an evaluation set built from logs or synthetic data with subject matter experts, then RAG, agents for mixed sources, fine-tuning if needed, and monitoring with LLM judges. BLEU and ROUGE no longer work; LLM judges vary widely on 1-to-10 scales and favour longer answers, so use binary metrics and check that a judge's reasoning matches the human's.
• When an AI application breaks, go back to the data. Aishwarya says more time goes into the data feeding models than into the models, because real-world data has annotation issues and conflicting answers, and no model can fix garbage in. Personal picks at the time: Claude Sonnet 3.5 for code in Cursor or Codeium, ChatGPT for creative writing, NotebookLM for research papers.

2025-01-23 · open the original ↗

• Dictating a day's jumbled tasks to a chatbot yields an organised plan. After the school run Azhar records up to 10 minutes of mixed tasks in Claude's audio mode (a speech, car insurance, a parking ticket, a Stripe refund), and on reaching his desk 15 minutes later finds a sorted to-do list, a drafted appeal letter and ordered speech notes. He calls Claude 3.5 Sonnet good enough to change his day.
• The missing piece is turning plans into actions. For now he copies and pastes; he says AI companies are promising that step "this year". He expects hundreds or thousands of agents working for each person and wants a "chief of staff" supervisor agent that takes unstructured input, doubting Apple will build it.
• His multi-agent debate succeeds about half the time. He defines a historian of technology, an investor and a cynic, run on o1 and at least two Claude 3.5 models, with an orchestrator making them argue. A run takes five or six minutes and costs about 20 cents, and fails about half the time, which he says matches human teams on open-ended research. He builds it in Wordware, disclosing he is an investor, or Lindy.
• Learning AI means working close to the API. Azhar says calling models directly lets you set temperature (sensible but creative, around 1, for an evaluator), which third-party interfaces hide. He also uses Gemini Advanced research, NotebookLM, and an email-drafting tool he has invested in; the host describes using You.com's research mode alongside Claude 3.5 Sonnet.

2025-01-22 · open the original ↗

• Linear forecasts ignore how things actually grow. Azhar says business plans still show fixed increments, though growth follows a logistic S-curve with an exponential phase; "nothing is linear except our birthdays". He points to compound interest and late pension saving, and says Bay Area tech internalised Moore's Law. Listeners have lived through exponentials in phone upgrades and data lakes growing from a gigabyte to a petabyte.
• Model growth as compounding percentages with feedback loops. He says he has never used linear increments in more than 20 years of planning, and builds in feedback loops such as Intel chips enabling Windows features that drove upgrades. A linear multi-year S3 storage budget, he argues, is the extreme assumption, even if a realistic one alarms the CFO.

2025-01-21 · open the original ↗

• Assuming exponential compute growth stops is the more radical bet. Azhar calculates world computing capacity has grown about 65% a year compounded for 52 or 53 years since 1972, and his baseline for 10 to 30 years out is to extend that line. He grants the turkey-before-Thanksgiving objection but argues the curve rests on processes (parallelism, interconnects, financial incentive, soon AI help) rather than on shrinking transistors alone.
• Linear forecasts are the extreme assumption in planning. He says he has modelled growth as compounding percentages with feedback loops (Intel chips enabling Windows features driving upgrades) for more than 20 years, and that a linear S3 storage budget contradicts history. Most real growth follows a logistic S-curve; "nothing is linear except our birthdays".
• AI-native companies are opening an "exponential gap". Azhar says fast-growing AI companies reach $30 million annualized revenue in about 20 months against about 60 for SaaS, and that Anthropic reached a billion dollars in sales in
  2024. His advice is to build "the muscle of working with AI" on APIs and semi-finished tools, neither finished SaaS workflows nor your own foundation models.
• Energy is turning from a commodity into a cheapening technology. He says fossil energy has not got cheaper in 200 years while solar panel costs fell by, he thinks, 99% over 20 years, with batteries on a similar curve, and likens the shift to telecoms giving way to the internet. He cites an Oxford paper from about two years earlier finding a fully renewable system cheaper to run, and puts the energy transition at a couple of decades and cheap carbon capture and desalination further out.
• AI is needed to keep knowledge growing as population peaks. Azhar expects human population to peak in 60 or 70 years, after which fewer people can work on problems. On jobs he says there is "no solid ground": citing David Autor's finding that 60% of today's US jobs did not exist in 1940, he expects most jobs to change, everyone's work to become more "boss-like" directing AI, and skilling up as the only preparation.

2025-01-20 · open the original ↗

• AWS's generative AI stack runs from plug-and-play to fully granular. The guest (Eremenko, in episode 853) says Amazon Q offers no choice or customisation of the base model. Q Business is effectively RAG across S3, RDS, Kendra, Gmail, Dropbox, Slack, Zendesk, Jira and Salesforce, with an option to fall back on the model's own knowledge. Q Developer is a coding assistant in JetBrains, VS Code and the CLI, and an AWS-account assistant.
• The middle layer suits most; the granular one needs technical staff. He describes Bedrock as pay-as-you-go access to proprietary and open-source models, with prompting, RAG and agents. SageMaker covers the whole machine learning pipeline with finer customisation and deployment, but needs more technical staff or a partner. The guests disclose no Azure or GCP experience, after over two and a half years on AWS.
• A low-code tool outperformed his hand-tuned models, he says. The second guest (de Ponteves) reports that SageMaker Canvas built, trained and tuned a model in a few clicks and five minutes that outperformed every model he had spent hours tuning on his usual benchmark dataset. He also praises Data Wrangler for preprocessing and JumpStart for vision and NLP foundation models beyond LLMs.

2025-01-19 · open the original ↗

• Instruction-based fine-tuning adds an explicit instruction to each example. The guest (de Ponteves, in episode 853) fine-tuned an Amazon Titan model on niche topics with an instruction to answer in two or three words ("a dog" to which pet is more attention-seeking), and says the result gave concise, straight-to-the-point answers. The difference from domain-specific fine-tuning, he says, is an extra dataset column carrying the instruction. The host notes his former company Nebula instead did domain-specific fine-tuning of Llama models with LoRA.

2025-01-18 · open the original ↗

• Data science pay turns on several distinct factors. The speakers list industry, seniority (IC1, IC2, IC3, with more experience paid more), the area of the organization, location (the Bay Area pays more), performance, and specialized skills in demand. They add the macroeconomic situation, citing 2021's high demand and low supply, and a company's own need: a role critical to the business that must be filled fast gets paid more.

2025-01-18 · open the original ↗

• Start with cost, modality and the customisation the model supports. The guest (Eremenko, in episode 853) says pricing differs between models, that input and output types (text, image, video) rule models in or out at once, and that you need to know which customisation methods you need before checking which models support them.
• Match inference options and latency to how the model is used. He contrasts real-time uses with no delay (an in-game feature, a natural voice conversation) with asynchronous answers in five minutes and background batch jobs.
• Benchmarks, language and size trade performance against price. He notes that leaderboards score the base model before you customise it, that average performance at a lower price can suffice where accuracy is not business-critical, and that small language models are gaining ground.
• Scalability, compliance, licensing and energy are easily overlooked. He asks how a deployment serving 10,000 users grows to 100,000, flags growing AI regulation and model licences, and suggests weighing a provider's chips and energy use in training and inference.
• In practice, dataset format decided the model. The second guest (de Ponteves) says he chose an Amazon Titan model for one fine-tuning experiment and a Llama model for a medical chatbot because each matched his data's format, and mentions Amazon's just-released Nova models.

2025-01-17 · open the original ↗

• Intelligence began with physics and chemistry storing information. The host relays Ray Kurzweil's first epoch from The Singularity Is Nearer: atoms formed a few hundred thousand years after the Big Bang, then molecules, with carbon's stable chains allowing elaborate information. Self-replicating RNA, though not alive, let natural selection act. The host adds his own awe that physical constants are fine-tuned enough for this.
• Biological life is the second epoch. Per Kurzweil, RNA learned to make proteins and became trapped in fatty-acid bubbles to form protocells. Selection for more stable membranes and storage, eventually double-helix DNA, produced single-celled and then multicellular organisms several billion years ago.
• Technology, beginning a few thousand years ago, is the fourth epoch. Humans began storing information outside their brains, from clay tablets to solid-state drives, and now machine learning that perceives and reasons. The host forecasts that simply scaling parameters, inference compute and data will keep machines passing humans on harder tasks, and that within a few years scientific discovery may be led more by machines than humans.
• We are entering the fifth epoch, merging brains with computers. Today brain-computer interfaces are implanted mainly for paralysis or motor loss after spinal injury or stroke. Kurzweil expects ordinary brains to be augmented over coming decades, vastly expanding memory and abstract thought.
• The sixth epoch turns the cosmos into a substrate for computation. The host calls it Kurzweil's most speculative idea: intelligence spreads beyond Earth and restructures matter, limited only by the speed of light and quantum mechanics, until individual intelligences merge. He says it may be wild conjecture, but that Kurzweil has been prescient about AI's pace.

2025-01-16 · open the original ↗

• Long data careers rest on learning, network and core excellence. Manisha, a data science leader at Google and founder of the career platform Prep Vector, says the field has changed so much in 12 years that learning must be a habit. Relationships matter because managers move and hire, and there is "no workaround" for being extremely good at the core job; a host adds that your network depends on being remembered for good work.
• Leadership does not wait for a manager title. Manisha separates leading from managing: anyone can fix what is broken, and future leaders show in small signals such as running meeting agendas, writing documentation, following up proactively, and feedback from cross-functional partners. The IC and manager tracks are parallel, with the difference being looking inward at your own work versus outward at other people's success.
• Ask for the promotion early, and be deserving of it. A co-host says few people ask, perhaps 5%, especially women, and that promotions are a lagging indicator of work already done at the next level. Shane missed a promotion at PwC despite five-out-of-five reviews because the partner needed to hear six months ahead; Manisha was once skipped too, and a friend's line was that "the crying baby gets the most attention".
• Candidates lack industry perspective, not theory. Manisha advises one base resume tweaked in about 10 minutes to each job description, and impact stated in business terms rather than model accuracy, since nobody is promoted on accuracy. Career switchers should research job descriptions (A/B testing was the gap Manisha found coming from healthcare consulting), lean on transferable skills such as stakeholder management, and close the gaps.
• The underrated skill is talking at different altitudes. Manisha's answer to a listener's question is zooming in on detail with a research team and out to OKRs and business impact with a VP, on the same project. Shane names prioritization and saying no, judged against stakeholders' goals; a co-host names creativity in data storytelling.

2025-01-16 · open the original ↗

• The first three lifecycle steps are the labs' job, not yours. The guest (Eremenko, in episode 853) says data preparation, architecture selection and pre-training, which costs "hundreds of millions of dollars", are done by companies like Meta, Anthropic, Google and OpenAI. He adds that businesses repeating them would make global warming much worse.
• Fine-tuning narrows a general model to proprietary data. He gives examples of medical journals, legal data, or ten years of customer-service call transcripts. The model learns specific terms and style the way a fluent speaker learns a specialist vocabulary.
• Evaluate against business metrics, not just technical scores. He says technical scores like BLEU, ROUGE and BERTScore are not enough, and a model must also be tested on the business tasks it is meant to do.
• Deployment, monitoring and iteration form an ongoing loop. He explains an endpoint as just a URL with parameters, says models degrade through data and model drift like a car needing maintenance, and says changes in the business, in customer expectations or in legislation send you back through steps four to eight.
• A single AWS service covers most of the lifecycle. The second guest (de Ponteves) says Amazon Bedrock handles most steps except pre-training, with data prep done in other AWS services. His students fine-tuned a Llama model on a Hugging Face medical-terms dataset in "dozens of minutes" and got a chatbot that could discuss advanced terms. The host separately notes that 2017 was the transformer paper, and ChatGPT in 2022 was GPT-3 with RLHF.

2025-01-15 · open the original ↗

• A prestigious degree stops mattering once the interview starts. The speakers grant that Harvard is looked on more favourably than a community college, but say the interview is "purely based on performance", and seven or eight years in nobody cares where you studied. Shane notes having gone to community college rather than Harvard; another speaker has clients from Harvard who cannot get the job, and they conclude that applied experience counts for more than what you learned.

2025-01-15 · open the original ↗

• A foundation model is a pre-baked base layer businesses rent and customise. The guest (Eremenko, in episode 853) likens it to a cake's sponge layer, baked by well-funded labs such as Anthropic, Meta and Mistral, to which a business adds its own toppings for its use case. He says the analogy itself came from ChatGPT.
• Foundation models are broader than large language models. The host offers a Venn diagram in which LLMs sit inside foundation models alongside vision models of the kind a Waymo car relies on, and the guest agrees, adding image and video models.

2025-01-14 · open the original ↗

• Salary secrecy benefits only the employer. The speakers say tech compensation is widely misunderstood because consistent information is scarce and talking about pay is taboo. An employer that can pay a good performer less has no reason to pay more, so knowing the baselines is the only way to help yourself.

2025-01-14 · open the original ↗

• Businesses should customise foundation models, not build them. Eremenko frames a foundation model as a cake's base layer that big labs bake and businesses rent and top. Of his eight lifecycle steps, the first three (data prep, architecture, pre-training costing "hundreds of millions of dollars") are done for you; a business handles fine-tuning, evaluation, deployment to an endpoint, monitoring for drift, and iteration.
• Model choice turns on many factors, and in practice data format. Eremenko lists twelve: cost, modality, customisation options, inference options, latency, architecture, benchmarks, language, size, scalability, compliance/licensing and environmental impact. De Ponteves says his own picks (an Amazon Titan model, a Llama model) came down to which fit his fine-tuning dataset's format. The host relays Andrew Ng's advice to ignore cost while prototyping; Eremenko objects for enterprises certain to roll out.
• Domain-specific fine-tuning is the business default. Eremenko ranks it the most common training-time method, ahead of instruction-based fine-tuning (which de Ponteves describes as an extra instruction column in the dataset), RLHF, continued pre-training and training from scratch. De Ponteves reports fine-tuning a Llama model on a Hugging Face medical-terms dataset in "dozens of minutes" to get a medical chatbot.
• Inference-time levers give large gains for little effort. The pair cover temperature (de Ponteves's story generator repeated one story word for word at temperature zero), RAG over a vector store (a French-dessert assistant from one PDF), agents that split tasks into steps so a simpler model can beat a bigger one, and prompt templates, which Eremenko warns can still be hijacked by injected instructions.
• Pick an AWS service by how much control you need. Eremenko calls Amazon Q (Business, Developer) plug-and-play with no model choice, Bedrock the pay-as-you-go middle layer, and SageMaker the most granular. They disclose no Azure or GCP experience; their AWS focus follows their course business's certifications. De Ponteves says SageMaker Canvas beat his hand-tuned models on his benchmark dataset in five minutes.

2025-01-13 · open the original ↗

• Using quantum computers does not require a physics degree. Neukart (Terra Quantum, in episode 851) says physicists are needed only to build chips. He argues the real barrier is meeting superposition and entanglement first as adults, and suggests quantum games for children. For anyone in data science, he calls quantum "a technology like GPUs" that can be added to the stack.
• Regression, classification and LLM hybridisation are fertile ground. He names image classification, chassis-damage inspection on production lines, medical images and self-driving, with LLM hybridisation an active research area with open questions. A model can be trained with a quantum computer and still run inference on a mobile device without one. For learners he points to Terra's TQ42 platform and TQ Academy training.

2025-01-12 · open the original ↗

• Fair pay starts with knowing the market rate. The speaker says reaching fair compensation is something everyone has control over: "information is power", so know the fair market rate or how to find it out. That information is then used across a career to make sure the pay matches the work.

2025-01-12 · open the original ↗

• The company funds medical work it says it does not need commercially. Neukart says Terra Quantum is about to start clinical trials, paid for itself, of laser treatments for osteoarthritis and arteriosclerosis. A fibre-optic channel delivers circular laser radiation, with imaging and laser parameters optimised by a hybrid quantum algorithm. He claims the approach could prevent stroke and heart attack.
• Some results are released without a customer. He says Terra's hybrid quantum classifier for fatty-liver (steatosis) cases outperformed the benchmark and was published for anyone to use. The host's framing cites the CEO's meeting with Pope Francis on AI and quantum for human well-being.

2025-01-11 · open the original ↗

• Quantum supremacy headlines compare against problems built for quantum. The host cites Satya Nadella's claim of a billion years classically versus under two minutes quantum for RSA-2048, and Google's Willow claim of five minutes versus "10 septillion years", calling such comparisons unfair. Neukart (Terra Quantum, in episode 851) agrees real-world problems matter more, but says RSA is a real one because quantum computers can find the prime factors of very large numbers efficiently.
• Quantum-safe security combines three layers. Neukart describes post-quantum algorithms, citing NIST's recently released standard of three algorithms, which cannot be cracked efficiently "as of current understanding". Quantum key distribution over fibre encodes keys in photon phase or polarization so eavesdropping is detected. Quantum random number generators, using photons or avalanche breakdown, cannot be reproduced the way seeded pseudo-random generators can. Combined, he says, "you're absolutely secure".

2025-01-10 · open the original ↗

• A certificate matters only if it builds job skills. The speaker says the certificate itself does not matter unless it gives practice on the work you actually do, adding that such programs are not hard to join and complete.

2025-01-10 · open the original ↗

• Senior data scientists can out-earn many doctors. The speaker puts family medicine and pediatrics at around $250,000 a year, the same as a data scientist at IC4, so from staff level a data scientist makes more. Radiologists, anesthesiologists and surgeons make around $800,000, but a principal staff data scientist at IC7 at Facebook was making $750,000, "probably more than some surgeons".

2025-01-10 · open the original ↗

• Use the best model first and optimise cost only after success. In episode 841 Andrew Ng says a full weekend day of his own coding ends at about a $5 OpenAI bill, and agentic workloads at most tens or low hundreds of dollars. Cheaper models or supervised fine-tuning before something valuable works is, he says, "most likely premature optimization".
• AI engineering starts from business requirements and a baseline. In episode 847 Ed Donner says there is no best LLM, only the right one, chosen on data, business-outcome metrics and non-functional limits (budget, time to market). He advises a non-LLM baseline first, then usually closed source. Open source fits large proprietary datasets, sensitive data, inference-cost pressure, or on-device use with small models like Llama 3.2.
• Agentic systems spread data across locations, changing security needs. The episode 843 guest from Protopia says agents will touch on-prem, private-cloud and multi-tenant public-cloud data, so exposure must be managed per component. He points to homomorphic encryption, including a recent Apple announcement for information retrieval, as one piece alongside his company's Stained Glass.
• One company gave 2024 both its biggest flop and wow. In episode 849 the host named OpenAI's o1 his wow moment while his co-guest named OpenAI her disappointment. Her own wow was NotebookLM's human-sounding podcasts, and Waymo and Tesla's full self-driving were sub-wows. She relays a Waymo co-CEO describing a fleet of over 100,000 cars as building "one driver".
• Treating AGI as a coming rapture is a religious frame. In episode 845 Greg Epstein, author of Tech Agnostic, reads the host's singularity hopes as a "slow moving multi-stage Rapture". He argues uncertainty deserves dignity over confident leaders, and that Sam Altman's $7 trillion ask reflects OpenAI burning cash with no route to profit. He suggests there may be benefit in moving slowly and losing the AI arms race.

2025-01-09 · open the original ↗

• AI progressed from rules to statistics to deep learning to LLMs. Karan Singh, an engineering leader and senior software engineer at Salesforce, traces 1960s rule-based systems, probabilistic models, traditional machine learning, and neural networks held back by infrastructure until deep learning. Transformers made parallel training possible, so the companies with the infrastructure (OpenAI, Microsoft, Google, xAI, Anthropic) build the large models.
• Agents give an LLM tools, but engineers still build them. Karan's example wires a writer, an email sender and an SMS sender into an agent that infers what to do, replacing if-else code that might take a month to write. Human-in-the-loop systems still need good software engineers; Karan likens the change to Facebook moving from PHP to React, and gives agents that build agents "very low confidence" within the next 5 years.
• LLMs retrieve rather than reason, so they cannot judge their data. Karan compares an LLM to a reader with photographic memory: trained on bad data, it will still answer, and a hallucination is it completing a sequence it does not know. RAG tells the model to favour supplied knowledge and lets it cite the source chunk.
• Karan expects AGI once models can reason. The forecast is within 5 to 10 years, pointing to ReAct-style prompting and OpenAI's new reasoning models from its "12 days" announcements, and adds that responsible AI work must continue alongside. Researchers can study where LLMs fail to reason or plan; practitioners should learn agent frameworks such as LangChain, LangGraph and CrewAI.
• Domain knowledge decides what you can get out of an LLM. The hosts and Karan argue that you can only prompt well, or build a useful agent, if you know the subject. Suggested resources are DeepLearning.AI courses, Andrej Karpathy's build-ChatGPT-from-scratch videos and Karan's book, anticipated "next year February"; a host advises non-technical listeners to read one AI news article a day.

2025-01-09 · open the original ↗

• Scheduling is intractable classically and suits hybrid quantum. Neukart (Terra Quantum, in episode 851) describes a production problem for Volkswagen: fitting 250-odd test classes, each with subtests, to staff with differing skills for cars leaving the line. The same algorithm can schedule EV software subroutines to cut power use. He concedes quantum also cannot prove optimality, so Terra compares quantum and classical results and keeps the better one.
• Logistics, satellites and finance are other live optimization uses. He cites minimising fleet size and empty miles for perishable goods that must reach hubs within windows such as 1 to 3 p.m., satellite constellations, market prediction, collateral optimization and exotic options pricing. He groups quantum uses as optimization, machine learning and simulation.