Dr. Jon Krohn1011: The Real AI Frontier Isn't Smarter Machines (with Catherine Williams)
synopsis
Catherine Williams, chief data officer at the nonprofit Candid and a data scientist since 2012, ends on a single piece of advice for the LLM era: keep building your own mental models while you hand work to machines. Deep technical understanding still matters even when AI does the math, she argues, because it is what lets a person reason at the next level up. The frontier she finds most interesting is not smarter machines but the interface where machines help human thinking.
From black holes to mental models. Williams was a pen-and-paper mathematician working on spherically symmetric black hole spacetimes. Her thesis showed that, under reasonable assumptions, a quasi-local characterisation of a black hole's boundary coincides asymptotically with Penrose's event horizon. She took two habits from that work into industry. One is fearlessness about going deep into hard material. The other is deliberately building a mental model of each business, checked by repeating it back to colleagues. The habit she had to unlearn was bottom-up thinking: needing every detail before trusting the big picture works for a math PhD, she says, and wastes time in business.
Does math still matter? The host notes that engineering has displaced math as the field's core skill, and that programming and math are exactly what LLMs do best. Williams answers with a hierarchy of complexity. As machines absorb one layer, people move to the layer above, but they still need fluency in the layers below, as children learn arithmetic despite calculators. The best professionals will move up and down the stack rather than specialise. She sees the interface between machine and human thinking as an underexplored frontier and uses LLMs to sharpen her own models. A career that tracked the field. AppNexus hired her in 2012 for probability math, despite her lack of data skills, because its bidding data was stored in aggregate and modelled with Bayesian methods. Once Hadoop exposed log-level data, her team taught itself logistic regression and random forests from papers and conferences. She later became marketplace czar, treating buy and sell sides as coupled rather than zero-sum and building a joint revenue forecast. After AT&T bought the company and it became Xandr, she moved to Qualtrics. There she built a machine-learning function just as BERT arrived, and embeddings swept aside the keyword and rule-based text analysis.
Fine-tuning and cost. Both agree that fine-tuning open models to beat off-the-shelf ones has largely gone the way of the dinosaur. Williams expects LLMs' true costs to come due and spark a new wave of creativity. The host counters with his own claim that a fixed level of capability has fallen to about 1% of its cost within two years. She agrees, saying that cost pressure, through distillation and cheaper ways to run frontier models, is the creativity she means.
Candid. Candid formed about six years ago when Foundation Center and GuideStar merged, and it supplies data on nonprofits, foundations and grants. For the first time in her career the data is not instrumented by her own organisation; it comes largely from IRS forms and self-filled profiles, so cleaning and extracting meaning are the core job. The organisation moved from printed books to CDs to SaaS, and now offers an MCP connector, built with Anthropic, that serves its data inside tools like Claude. Advice. To aspiring leaders she says to learn the problems one level above your own: that is what got her into the rooms where decisions were made. For the future she claims no crystal ball, only the rule to grow along with the tools rather than outsource to them.
- Deep math still matters even when machines do the math. Williams argues knowledge is a hierarchy of layers, and as AI takes over one layer practitioners move up to the next, but still need fluency in the layers below, as children still learn arithmetic despite calculators. The best data professionals, she says, will move up and down that stack rather than specialise in one slice.
- The real frontier is the machine-to-human interface, not smarter machines. Against the billions raised for world models, she says the more interesting problem is using LLMs to improve one's own mental models. Her advice for the LLM era is to keep building those models while delegating to agents: "Don't outsource. Grow along with."
- A math PhD's habits help, except the bottom-up instinct. She credits the PhD with fearlessness about hard technical material and with the habit of deliberately building a mental model of each business's data flows, but says she had to unlearn needing every detail before trusting the big picture. Her leadership advice is to understand the problems and constraints one level above your own role.
- Each data-science era was wiped out by the next. She was hired at AppNexus in 2012 for probability math on aggregated bidding data, moved to logistic regression and random forests once Hadoop exposed log-level data, and at Qualtrics saw embeddings blow away keyword and rules-based text analysis. Fine-tuning has largely gone "the way of the dinosaur" to zero-shot models, but she expects pressure to cut LLM costs, once their true bill comes due, to drive a new wave of creativity in distillation and cost optimisation.
- Candid's hardest data problem is data it did not instrument. The nonprofit, formed about six years ago by merging Foundation Center and GuideStar, draws largely on IRS forms and self-reported profiles, so cleaning and interpretation matter more than in her earlier roles. Its MCP connector, launched with Anthropic, puts that data directly inside AI tools like Claude.