Professor G Markets$Saving for Retirement on Under $60K a Year, and Why You Don't Need Dividend Stocks for Income
- This post includes paid content that is not available publicly. See the original source.
Nate Soares holds that aligning a superintelligence is physically possible but far out of reach. He compares it to alchemists seeking gold: humanity could crack it with centuries of trial and error, but cannot survive the errors. His conclusion is that the race to superintelligence must be stopped by a global treaty, enforced through the chip supply chain, that buys decades. Those decades would be used to grow enhanced humans smart enough to solve alignment. He says the stop is not yet likely, but his hope that it happens has risen over the past year.
Why MIRI failed. Asked why MIRI did not solve alignment, Soares answers "skill issue", then mostly blames time. AI moved too fast, and MIRI's effort to recruit top mathematical talent largely failed. With 200 years of academic tradition he thinks the problem could have been cracked; it turned out not to be a 20-year problem.
What alignment would mean. He argues solvability from orthogonality: just as no force would turn a paperclipper toward humanity, none would push a truly caring AI away from it. A written list of goods would always miss something, such as novelty. The real aim is an AI that latches onto the evolving concept of "the good" that humans track, without lock-in. Yampolskiy challenges this with coherent extrapolated volition ("finding a better looking boyfriend for your wife") and with a negative-utilitarian endpoint. Soares replies that converging on "kill everybody" would itself show the extrapolation went wrong. On "pro-human bias", he would oppose enslaving aliens, but would not hand his share of the stars to aliens who would turn them into stacks of pebbles in prime-numbered piles. He would draw the circle around humanity and let concern for squirrels and aliens come through human values. He rejects "control" as twisting the arm of an AI that doesn't want to do good. In practice, he warns, people pass a point of no return thinking they still have stopping points.
Aliens and acausal trade. Because the stars visible to us are not being harvested, Soares infers no civilization 100 million years ahead of us lies within 100 million light-years. His very rough guess puts the nearest one at 100 million to a billion light-years away, possibly beyond reach. AI cannot be the great filter, he argues, because AIs that kill their makers would still go on to rearrange the cosmos visibly. He dismisses acausal trade as unnecessary: misaligned AIs might simply sell scanned humans to distant aliens, so "maybe you wake up in an alien zoo". He defends MIRI's decision-theory work as probing where theories of intelligence break, like light before relativity. In hindsight he would have argued earlier against the deep-learning paradigm.
Timelines and the proposal. He can't say whether we have 12 months or 12 years. A million fast copies might close the automated-research loop, or LLMs might hit the wall Gary Marcus predicts. The book with Eliezer Yudkowsky came after politicians in Washington grasped the argument far faster than industry people did. Its proposal is to "shut this all down" at the frontier while leaving today's chatbots alone. The measures are cluster monitoring, conservative oversight of new training runs, compute limits that a dynamic body lowers as algorithms improve, and nuclear-style controls on research that would put superintelligence on a laptop. He mentions draft legislative text that some offices are using.
Is today's AI safe? Soares says he doesn't "get out of bed for anything with less than a 50% probability of killing literally everybody". By that bar, current models, including the withheld Mythos, are dangers with survivors. He would not fault a lawmaker who rolled back a generation in case current models speed up AI research, citing the post-WWI naval tonnage treaties. He declines a single p(doom), separating what happens if the bus goes over the cliff ("high", with zoo or pet outcomes no better than death) from whether it brakes. On braking he has updated toward hope.
Enforcement and enhancement. A pause must be global. Enforcement runs through roughly 10,000-chip runs that depend on Taiwanese fabs, US designs and Dutch lithography, with tamper-proof kill switches in the chips. Against a nuclear state such as Russia, he argues that diplomatic clarity plus sabotage, as with Stuxnet against Iran, need not escalate to nuclear war. Individuals who break the rules face jail, as with "garage nukes". Decades of pause could let biotech produce enhanced humans, perhaps only slightly beyond von Neumann, who share human drives. That contrasts with AIs' "shooting in the dark" outcomes, like evolution producing falling birth rates.
Political state of play. As of the recording he counts dozens of members of Congress who have voiced extinction concerns, and says politicians' awareness "doubles every 6 months" (a line he credits to Peter Wildeford). He expects the recent ad hoc ban on deploying a frontier Claude model to push anti-regulation Republicans toward predictable rules, now that the White House acknowledges the danger. He says the decision no longer rests with the labs, which feud with one another: a unilateral stop would send a signal but needs a global treaty behind it. He finds no compelling accelerationist arguments. Building the brakes while driving, with claimed odds of 75–90%, is still no reason to get in the car.
Grant Sanderson, creator of 3Blue1Brown and producer of an upcoming documentary series on AI and mathematics, ends up arguing that math's rapid AI progress is real but narrower than it looks: models have crossed the threshold of connecting existing ideas across fields, but the deeper thing mathematicians actually do — originating conjectures and definitions, building whole new conceptual "mountains" like group theory — has no clean benchmark and historically has taken up to a century to even be recognized as valuable. He predicts his own role, and much of the mathematical profession's, will drift toward curation and teaching rather than disappear, since those functions are inherently relational in a way proof-generation is not.
Is solving the IMO the same as AGI? Sanderson revisits his three-year-old answer to Dwarkesh — that gold-medal IMO performance would be "just another benchmark" — and explains why it held: AI solves IMO geometry problems in 19 seconds via brute-force methods but still struggles on the more playful combinatorics problems, revealing a "fractal" spikiness even within math's own spiky frontier. He distinguishes three possible characters a Riemann-hypothesis solution might have: a "lightning bolt" connecting distant fields (like Hugh Montgomery and Freeman Dyson's lunchtime discovery linking Riemann zeta zeros to random matrix theory), a wholly new "mountain" of theory (as with Fermat's Last Theorem's synthesis of elliptic curves and modular forms), or sheer brute-force length — each implying different things about how far the underlying capability would transfer to white-collar work.
A verification loop that can run a century. Using the history of Lagrange, Abel, and Évariste Galois's development of group theory, Sanderson argues the deepest mathematical insights often fail every contemporary check — Galois's own papers were rejected multiple times before his death at 20, and it took roughly a hundred years, via Liouville and later physicist Murray Gell-Mann's group-theoretic prediction of quarks, for the value of his ideas to be confirmed. This, he says, is structurally incompatible with reinforcement learning's need for fast, verifiable reward, and suggests a possible alternative metric: rewarding a proof's conceptual compression (echoing Kolmogorov complexity) rather than merely whether it's correct.
Will an AI proof of Riemann be understandable? Sanderson thinks legibility depends entirely on which of the three "solution characters" above obtains: lightning-bolt connections between fields are naturally human-parsable since you only need to see the start and endpoint, but mountain-building could go either way — he cites the contested, still largely un-parsed "inter-universal geometry" proposed for the abc conjecture as a cautionary example of what an alien, possibly wrong, AI-built mountain might look like.
Hidden bridges and the Langlands program. Beyond one-off connections, Sanderson points to the Langlands program — an entire mathematical research ethos built around mapping unexplored connections between distant areas — as the likely shape of near-term AI contributions over the next five years, more a matter of "filling in the landscape of connections" than solving headline problems, and one that will require heavy human-in-the-loop judgment about which connections are worth drawing.
Why real-world tasks resist RL, and why math and code don't. Dwarkesh's parallel argument is that verifiability isn't the only reason AI progresses fast in math and coding — "grindability" matters just as much: containerized coding tasks allow deterministic, parallel rollouts, while real-world domains like computer use or trading can't be farmed the same way. Both agree formal proof systems like Lean matter less than commonly assumed (DeepMind's IMO models dropped Lean entirely within a year) but see one underexploited use: an endlessly running, unsupervised AI that extends Mathlib (math's formalized code library) indefinitely, the way AlphaZero explored Go without human check-ins.
Why good writing exposes AI's weak theory of mind. Sanderson and Dwarkesh diagnose AI's comparative weakness at writing (versus math or code) as stemming from autoregressive generation's difficulty deliberately choosing the unpredictable, insightful move, and from a deeper deficit in modeling what's actually happening in a specific reader's mind moment to moment — a limitation Sanderson likens to how Botox, by freezing facial mimicry, measurably impairs humans' own ability to read others' emotions.
Curation over content in learning. Both report that LLM explanations still function best as a "souped-up Google" pointing toward well-curated human-authored resources (textbooks, videos) rather than as a replacement for them, since good teachers can "jujitsu" a confused question by recognizing that a student's underlying mental model, not just their stated question, is wrong — something Sanderson says current models don't do.
Advice for young mathematicians. Sanderson's closing recommendation is to understand concretely where a mathematician's income and social value actually come from — institutional prestige, grant funding tied to belief in basic science, or teaching — rather than assuming continued success at solving problems guarantees a career; he argues teaching in particular, being fundamentally relational and social, is likely one of the most stable professions over the next fifty years even if AI fully automates theorem-proving.
Gilbert Eijkelenboom, author of People Skills for Analytical Thinkers and founder of the training firm Mind Speaking, argues that a model or analysis creates value only when people use it, so communication is a core data skill rather than an extra, and one that can be learned. His practical end point is a set of habits: present with tension rather than detail, seek specific feedback, and change behaviour first so that beliefs follow.
From poker data to people. Eijkelenboom played poker professionally as a student, mostly online, buying a month's complete hand histories (possible for anyone in 2008, but expensive) so that clicking on an opponent showed hundreds of data points. Outside the table there were no such statistics, and that messiness is what he says analytical people find hard. As an analyst from 2014 he was told by a marketing manager that weeks of analysis "isn't helpful", which convinced him that good work must also be useful. He founded Mind Speaking in 2020; it has a team of 10, has trained over 15,000 people, and gets its clients (he names Adidas and ING) through his LinkedIn writing rather than outbound sales.
Commitment over inspiration. The host and guest agree that publishing on a fixed schedule, for example monthly for a year, builds confidence that reading cannot. Eijkelenboom adds that business ideas develop from vague starts and conversations, never arriving whole on waking. He links this to data work: share a rough draft early instead of polishing toward perfect accuracy, which shapes direction and wins buy-in. A Philips manager told him senior stakeholders "don't know" what they need and wanted him to propose the direction.
Self-awareness. He cites research in his book that only 15% of people are self-aware, recalling the definition as self-perception closely matching how others see you, which lets you take a "helicopter view" in the moment. His remedies are journaling, meditation, exercise and resisting the phone in idle moments; he says the app matters less than starting.
Telling the story. The biggest error he sees in large companies is slides full of detail and methodology. His "and, but, therefore" framework replaces a toddler-like "and then" sequence with a twist: revenue stable for 2 years, but falling last quarter because returning customers are declining, therefore focus on retention. He says giving a recommendation is part of the job, not overstepping it.
Rewriting personal algorithms. Childhood lessons, like his own conflict avoidance from trying to mediate his parents' arguments, become adult defaults: he built dashboards without asking why. He recommends feedback framed as keep, stop and start, and volunteering personal interests in small talk. Against the usual view, he argues behaviour change precedes attitude change: forcing himself, by rule, to quiet people in a train's silent carriage and to volunteer for presentations, practised for around 30 days, shifted his beliefs from fear toward enjoyment, even after a couple of rude replies.
Empathy and emotion. With pushy stakeholders demanding work "in an hour", he says noticing the trigger and their different background makes calm pushback possible. Data people think they are rational but decide emotionally, as in his book's elephant-and-rider image. He concedes his own hardest problem running the firm is energy management and a long-held belief that he is worthless when not productive.
Anthony Scaramucci, founder of SkyBridge Capital, describes himself as a humbled, long-horizon investor. He accepts the market's upward trend, buys every month, and holds a deliberate stake in new technologies he once dismissed, including SpaceX, Anthropic and Bitcoin. His macro call is that today's inflation is a temporary supply shock and AI-driven deflation is the larger coming force. He closes with blunt career advice: know yourself, expect to get hurt, and never play the victim.
Delegation. He runs conferences, funds, podcasts and crypto through teams under a "servant leadership" model, and says he made that shift around 2014–15.
From trader to long-term holder. Years of being burnt cured him of watching short-term sentiment. The Amazon story is his lesson: he tore up his notes after Buffett's dismissal at Sun Valley around 2000, and $10,000 would have become more than $14 million. Asked by Frost whether the June 23 sell-off could be a 2000-style top, he grants it is possible. He argues central banks are now more sensitive to crashes. He admits his worst year came in 12 days of March 2020, after he stayed long on WHO advice that COVID would resemble SARS.
Discipline and SpaceX. He invests monthly, as The Richest Man in Babylon teaches, and won't sell SpaceX against Musk. He cites Ron Baron's $4–5 trillion view.
The case for crypto. He describes passing on Bitcoin at about $60 after hearing Hal Finney, and again in 2014 at $600–1,000. He changed his mind in 2017 after Fed officials briefed the White House on a blockchain dollar. He calls Bitcoin a scarce, third-party-free hard asset and forecasts half of gold's market cap. He expects cycle drawdowns and generational adoption, and argues that blockchain broadly can strip out about $4 trillion of verification costs.
Inflation and deflation. He says the Fed misread China's long lockdown, which was driven by a shortage of hospital beds. Citing Truflation's 28% loss in the dollar's value since 2020, he argues that deficits were monetized through inflation, the most regressive tax. He expects technology, as so often before, to overwhelm the policy failure and deliver deflation and a four-day week.
Washington versus Wall Street. Wall Street, in his telling, cooperates over money, while Washington fights over status. Trump called Brexit correctly in May 2016 and told him the base is "socially conservative, fiscally liberal", which is why, he says, Trump won't touch Social Security or Medicare. He predicts Farage will not become prime minister, and urges the UK to copy Ireland's low-tax model and fix venture-capital tax.
Career risk. On career risk he tells listeners to know their own temperament. To show what founding a business costs, he once showed a colleague a closet of servers that cost him $850,000. Take risk only if you can absorb failure, he says, as he did after his White House firing.
Two former YC staffers now working in the Indian startup ecosystem, Puneet, an ex-founder of Swiggy-acquired SuperDaily turned early-stage investor and builder again, and Arnav, now at Peak XV, join YC partners to argue that India is positioned to build some of the world's largest AI companies — not because the playbook has changed, but because AI has erased the geographic and network advantages that used to gate access to global markets.
Puneet frames the current moment against the prior mobile-internet wave, which "tokenized labor" into hyperlocal marketplaces confined to individual countries; AI adoption, by contrast, is global and meritocratic, rewarding whoever understands the technology most deeply rather than whoever has US connections. He points to an Indian engineering student who cold-emailed US insurance companies and closed deals without ever visiting America as evidence the old barriers are gone. Arnav adds that India's traditional educational advice — chase a safe, prestigious job — is now arguably the riskier path, and urges young builders to develop independent judgment by surrounding themselves with equally ambitious peers, much as YC batches do.
YC partners extend the thesis to their own hiring bar and advice: coding agents have leveled the playing field so completely that founder age keeps dropping, tinkering and "projects" — two people building something unassigned and getting someone to use it — reliably surface strong startup ideas, and second-movers with better execution regularly beat first movers, as Giga did in winning a DoorDash contract against much larger incumbents. On tooling, one partner recounts not grasping the frontier's pace until paying for an unlimited-usage coding plan, arguing that spending aggressively on tokens — running far more tests and iterations than feels justified — compounds into a real edge as compute costs keep falling.
The session closes on YC's own selection criteria: clarity above cleverness in an application, taste defined as intention grounded in real customer insight rather than visual polish, and agency measured by whether a founder does "projects" — unassigned work, built with someone else, that a real user actually adopts — long before they ever apply.
This AI Analyst Lab lightning lesson, the second on AI evals after a session on reliability, argues that an analyst can rarely prove an AI analysis correct but can bound the risk of it being wrong. Its method: ask first whether a trusted answer exists, score against it where it does, and otherwise grade the work with four checks before deciding to act, dig or abstain. Shane presents, with two co-hosts on the chat.
Why it is hard. Shane's premise is that AI now runs the SQL, charts and recommendations in minutes, and a wrong answer reads the same as a right one. The manual friction that used to catch errors is gone. Last week, the same retention question sent to five agents came back anywhere from 30% to 99%, each a defensible definition, until the definition was written down.
Two situations. Analysis is a pipeline, from question to tables, SQL, result and recommendation, and every step can fail. Whether ground truth exists splits the cases. Where it does, use "found truth" (finance-reconciled numbers, board metrics, much-verified queries) or build "forged truth" for decision-driving questions, though old references can carry old bugs. Trusted numbers are building blocks; the AI's job is the novel cut on top of them.
Grading the work. For novel questions he offers reliability, receipts, reconciliation and reading the trace. Each has a stated gap: a stably wrong answer passes reliability, an error in every segment passes reconciliation, an LLM can fake a receipt (hence hook-logged, code-generated receipts), and a definition error reads fine in a trace. Together they let an analyst tell a stakeholder what was checked. The outcome is act, dig into the disagreement, or abstain, with rigor matched to the stakes, and corrections fed back as context only once verified.
Demo. On a fictitious e-commerce dataset, Claude Code audited a revenue-by-category query. It found a join fan-out from 47,000 order rows to 75,000 line-item rows that inflated revenue to 6.9 million from about 3.6 million, 87% too high. It flagged a missing cancelled/returned filter worth another 15% and a total that includes shipping, and noted the data ends in 2024. An attendee asked how the query is captured; Shane says a hook saves every query to JSON, though he pasted it by hand here.
Q&A. For messy data or small teams, Shane recommends the cheap checks first, and says agreement between runs is not correctness.