Souk's Notes - Unofficial Summaries
Stock Tok$If You Trade Options or Margin...
- This post includes paid content that is not available publicly. See the original source.
Crosstalk SolutionsEverything You Need to Know about UniFi's New Zone-Based Firewall!
- UniFi Network 9 replaces interface-based firewall rules with zones. The host lists the release's other additions as Site Magic SD-WAN, a local network API and UniFi CyberSecure, a $99-a-year threat-management service. Dave Barger, who presents the rest, says the old LAN In/LAN Out/LAN Local rule sets grew long and had to be read top to bottom to see the default action.
- The new zone matrix mirrors Crosstalk's own client firewall planning. Barger says Crosstalk consults clients on a source-versus-destination matrix coloured red (drop), green (accept) and yellow (complicated), and UniFi's zone matrix shows the same at a glance: "allow all", "block all" or a count of policies for each pair.
- The upgrade is opt-in and migrates existing rules automatically. After updating to 9.0 it must be enabled under Settings > Security; Barger's rules migrated in a second or two, though he assumes larger rule sets take longer. Networks are auto-sorted into Internal, External (WAN), Hotspot (guest) and DMZ zones, and custom zones can be created.
- Defaults allow internal-to-internal and block DMZ and hotspot traffic. External-to-internal holds three rules read top-down: allow return traffic requested from the LAN, block invalid traffic, then block all.
- Policies can now target a specific device, tracked by MAC. Sources can also be a network, IP or MAC, with ports and an allow, block or reject action. Barger warns iPhones and some Androids randomize MACs, so device targeting suits servers and NAS. Connection state is simplified to all states, return traffic (established/related) or custom.
Technology and FriendsEdward Thomson on Privacy and Security with Generative AI [EPISODE 841]
- Package suggestions from coding assistants are a supply-chain risk. Thomson gives three reasons: models can favour popular packages over good ones (he says the most popular Python OAuth package is no longer maintained and points users elsewhere), they know nothing of vulnerabilities found after their training date, and they can hallucinate packages. He cites security research in which a researcher registered a package a model had hallucinated, so the model then pointed users to a package the researcher owned. He says a malicious actor could do the same.
- CodeGate sits on the developer's machine, between the IDE and the model. Thomson's employer Stacklok started work on it at the end of 2024 as an open-source project. Its first feature checks the packages a model suggests against the data set behind Stacklok Insight, which tracks CVEs and maintenance across ecosystems such as npm and cargo, and shows a security warning in the IDE. He says most prompt gateways run on a remote server, while this one runs locally. Asked about revenue, he says that for now there is none: no services or freemium tier, just building the project and its community.
- Assistants upload context the developer does not control, including secrets. Thomson's example is a
.envfile holding API keys for a staging server or database. The assistant may send it as context, and.gitignoredoes not stop that. Even if the vendor is trusted, he does not want the data leaving the machine, passing through a corporate inspecting proxy or ending up in a log file. He also notes that some vendors promise not to train on context only on their paid plans. CodeGate replaces such values with tokens before sending and swaps them back on the way in. He says Stacklok is extending this from API keys and secrets to PII such as email addresses. - Developers verify AI output less as they grow used to it. Thomson says that in talks with developers, senior ones "trust but verify", checking suggested packages with scoring tools or the registry, and some use Copilot in place of a search engine. Less experienced and "AI native" developers trust the output more. Giard adds that deadline pressure also cuts vetting, and he frames Copilot's "co" as a reminder that the developer is still the pilot. CodeGate is pitched as automating that manual vetting step.
- CodeGate workspaces route each project to its own model and prompts. Thomson's examples: Copilot for JavaScript and Ruby, a different model for COBOL, or ChatGPT for documentation if it handles English better. A workspace can also add a project-specific prompt, such as one for Next.js and React or one for Ruby.
Dr. Jon Krohn863: TabPFN: Deep Learning for Tabular Data (That Actually Works!) — with Prof. Frank Hutter
synopsis
Frank Hutter, a professor in Freiburg and co-founder and CEO of Prior Labs, argues that a Transformer trained only on synthetic tables can carry out near-Bayesian prediction on any small tabular problem. He says TabPFN version 2, published in Nature, does this well enough to beat XGBoost on its own terms. The same model also turned out to be state of the art on time series it was never trained on, and he has started a company to turn it into products.
Why tabular data resisted deep learning. Hutter says tabular datasets are mostly small and unlike one another, and lack the spatial regularity that convolutional networks exploit. Their features also arrive already designed by people, so deep learning's strength, discovering features, was never needed. The value lies in higher-level engineering such as deriving BMI from height and weight. TabPFN so far uses only the raw values that XGBoost sees, without column headers, and work on combining it with language models has "some initial results".
In-context learning over datasets. Like GPT's in-context learning, TabPFN takes the training rows and test inputs as a prompt and outputs predictions, so each dataset is one training example. With few clean real tables to learn from, the team generated all of its training data synthetically.
Prior-data fitted networks. Hutter's 2022 paper "Transformers Can Do Bayesian Inference" showed that a model fitted with cross-entropy to many datasets sampled from a prior approximates the posterior predictive under that prior, exactly so in the limit. He contrasts this with slow MCMC and approximate variational inference. A PFN can even average over architectures, though it gives no posterior over weights. TabPFN's prior, devised by first author Noah Hollmann, samples structural causal models, about 130 million of them, and the network learns to weight candidate causal graphs by how well each fits the data. Because no real data enters training, test leakage is impossible. Reviewers had rejected v1's first submission as too good to be true.
From v1 to v2. V1 handled only numerical classification up to about 1,000 points and 100 features, and had around 15 published uses. V2 adds regression, categorical, missing, outlier and imbalanced data, plus text through the API only, and was tested up to 10,000 points and 500 features. Hutter sees a strong prior as the remedy for overfitting on small data, and is confident of scaling to 100,000 or a million rows. The team chose Nature because the contribution is the working whole, which Hutter likens to AlphaFold.
Time series and applications. Recast as tabular features, time series forecasting with the unchanged model ranks first on the GIFT-Eval benchmark at recording, ahead of Amazon's Chronos. The host notes v1's use in a Science chemoproteomics paper and the "awesome TabPFN" list of about a dozen applications. Hutter expects fine-tuning for domains such as medicine.
Prior Labs. Hutter says a university lab could never ship for the masses, so the startup builds products while academic work pursues interpretability and moonshots. It was hiring broadly ahead of a funding announcement.
- TabPFN beats boosted trees by learning tabular prediction in context. Hutter says it feeds the whole training set and test inputs into a Transformer prompt and outputs test labels, using the same raw values XGBoost and random forests get and no column headers; adding language-model reading of headers is in progress with "some initial results".
- Training on synthetic datasets alone makes the Transformer approximately Bayesian. Hutter says the model learns from about 130 million generated structural causal models, the prior devised by first author Noah Hollmann, and approximates the Bayesian posterior over them in one forward pass; he believes it is the first state-of-the-art model trained only on synthetic data, and says it rules out test-set leakage, after reviewers rejected the first v1 submission as too good to be true.
- Version 2 turns a proof of concept into a general tool, for small data. Hutter says v1 handled only numerical classification on about 1,000 points and 100 features, and saw around 15 published uses; v2, published in Nature in January, handles regression, categorical, missing, outlier and imbalanced data, with text through the API only, and was evaluated up to 10,000 points and 500 features. He is confident of scaling to 100,000 or a million with combined approaches.
- Untrained on time series, TabPFN still leads time-series forecasting. Hutter says casting timestamps as features (time of day, day of week, day of month and sine and cosine encodings) lets TabPFN forecast any horizon in one pass; at recording it ranks first on the GIFT-Eval benchmark, ahead of Amazon's Chronos and other time-series foundation models.
- Hutter launched Prior Labs because academia can't ship for the masses. He says Auto-sklearn showed that a lab with a few research engineers builds only for researchers; Prior Labs, where he is co-founder and CEO, will build products while his academic co-affiliation pursues interpretability, failure modes and moonshots. The host says the TabPFN library had been downloaded almost a million times at recording.
Polis Pandit$Constitutional Crisis: How Courts Should Anticipate Trump’s Defiance
- This post includes paid content that is not available publicly. See the original source.
Dr. Jon KrohnThe Secret to Sustainable Fitness and Nutrition Goals, with Colleen Fotsch
- Fotsch treats exercise as a productivity tool for desk careers. She says a fixed 30- or 45-minute workout block improves discipline, time management, energy, sleep and confidence at work, and that tech culture undervalues this as a help to being a better data engineer or analyst.
- Small daily movement adds up more than people expect. She recalls logging about 800 steps one day in her first data analyst job despite having worked out, and recommends stairs, walking Zoom calls and walking one-on-ones; someone managing one workout a week should aim for two plus more walks, not jump to six.
- Over-trainers should save all-out efforts for a few sessions. Responding to the host, who says copying two-hour, five-to-seven-day CrossFit regimens leaves him too tired to work, she suggests one or two all-out workouts a week. Otherwise, she says, scale intensity by how you feel and by readiness and sleep scores (taken with a grain of salt), progress gradually, and choose a programme that fits your own schedule rather than someone's on Instagram.
- Consistency, not perfection, is what produces fitness. Fotsch says fitness comes from many days of good choices, made about 80% of the time, rather than one hard week, and that goals can be as simple as feeling good and enjoying training; she saves her hardest efforts for Saturdays so weekday work doesn't suffer.
Dr. Jon KrohnHow PRVN GO Brings Fitness to Busy Lifestyles, with Colleen Fotsch
- After retiring, Fotsch had to stop training like a Games competitor. She says that for months as a full-time analyst she still trained as if heading back to the CrossFit Games, with her identity tied to being strong at elite movements, until she asked whether muscle-ups and handstand push-ups served her current goals and began training for feeling fit "20 30 years from now".
- Short, minimal-equipment workouts got results without hours in the gym. Fotsch says she found she did not need her old intensity to reach her new goals, and that the dumbbell, bodyweight and hotel-room workouts she posted resonated with followers.
- PRVN GO drops competition skills for busy professionals and parents. Fotsch says PRVN's CEO, Nick Johnson, approached her soon after she joined CHG; unlike PRVN's competition-oriented CrossFit programming, the company of champion Tia-Clair Toomey and her husband Shane, PRVN GO omits high-skill movements in favour of conditioning, bodybuilding and core work. The host says the episode is not sponsored.
- A remote community supplies the accountability a class would. She says users who can't get to a class log the same workout and talk with others doing it elsewhere, which counters the excuses that build up after a hard week.
Mr. GlobalThe STOCK ACT Is Enriching Congress
- The STOCK Act enriches Congress instead of curbing insider trading. He says the bipartisan law Obama signed on January 24, 2012 bars members from trading on inside information and requires disclosure within 30 days for personal trades or 45 days via an adviser, and argues lawmakers knew from the start it would make them richer.
- Fast public disclosure lets followers pump lawmakers' picks. He says sites like Capitol Trades post trades as soon as they're disclosed and supporters pile in, inflating the price, and cites many members beating the S&P 500 ETF's nearly 25% gain in 2023, getting worse each year; he illustrates with a hypothetical Marjorie Taylor Greene Boeing purchase.
Dr. Jon KrohnHow DBT Simplifies Data Modeling and Documentation, with Colleen Fotsch
- dbt turns raw warehouse data into stakeholder-ready models, mostly in SQL. Fotsch describes her work at Podium as adding fields and business logic to existing dbt models, later building models from scratch, using dbt's functions to consolidate repetitive code ("SQL on steroids"), and exposing the results in Tableau or Sigma.
- Documented definitions force agreement on what the business logic means. She says dbt docs make every field carry a definition agreed with stakeholders, which matters because something like "created date" or "active customer" may not mean what people assume, and which helps new engineers learn the data's meaning and granularity.
- Generated documentation and doc blocks remove the tedium. Fotsch says dbt now auto-drafts field definitions, which the host calls a good generative-AI use that removes the blank-page problem though output should be checked; doc blocks let a field defined in a staging model be reused throughout downstream lineage. The host contrasts the generic AI fitness tips he found useless on his Whoop.
- Sigma's spreadsheet feel won adoption from product managers. She says Sigma sits directly on Snowflake, removing an ingestion step, and its Excel-like look served both analysts and Excel-familiar PMs, which helps get everyone onto the same numbers.
Mr. GlobalToday On YouTube #news #truth #breakingnews #gasprices #gas #trump #biden #oil #oilprice
- This short only announces the premiere of his YouTube show. The show, "That's Not a Thing", covering energy, politics and "things that aren't a thing", premieres at 2:00 pm Central that day, with him hosting and Jill moderating; he says it will run daily at 2:00 and asks viewers to follow the channel.
Dr. Jon Krohn862: In Case You Missed It in January 2025 — with Jon Krohn (@JonKrohnLearns)
synopsis
This compilation of January 2025 clips, framed by the host around what the years ahead may bring, runs from how to forecast exponential change to how to choose and evaluate the AI systems that change produces. Its end point is evaluation: Brooke Hopkins argues that agents need layered, reference-free metrics rather than a single accuracy score.
Forecasting exponentials. Futurist Azeem Azhar says people intuit linearly although technology compounds. He argues for planning with dynamic percentages and explicit feedback loops, such as the Intel–Windows upgrade cycle, and calls linear storage budgets extreme assumptions.
Quantum optimization. Florian Neukart says hybrid quantum systems suit scheduling problems classical computers can only approximate. Examples are Volkswagen's vehicle testing across 250-odd test classes, EV subroutine ordering and fleet logistics. Quantum results are not provably optimal either, he concedes, so classical and quantum answers are compared.
Selecting a foundation model. Kirill Eremenko gives twelve criteria, from cost and modality to licensing and environmental impact. Hadelin de Ponteves says dataset format decided his own choices.
Evaluating agents. Hopkins says Coval layers metrics such as workflow adherence and function-call correctness, inferred from conversation context rather than golden datasets, so humans review only the most informative failures.
- Plan with compounding percentages and feedback loops, never linear increments. In a clip from episode 855, futurist Azeem Azhar says nothing grows linearly except birthdays: growth follows S-curves with an exponential phase. He cites the Intel–Windows upgrade loop that drove chip progress, builds dynamic percentages and feedback loops into his own forecasts, and says a linear forecast of S3 storage spend makes "incredibly extreme assumptions", even if the realistic one alarms the CFO.
- Hybrid quantum computing targets scheduling and logistics that classical heuristics only approximate. In episode 851, Florian Neukart describes a Volkswagen production-testing problem with 250-odd test classes, each with subtests, to fit to available staff skills; he also describes fleet routing with delivery windows and empty miles. He concedes quantum results are not provably optimal either, so the classical and quantum answers are compared and the better one used.
- Choosing a foundation model means weighing a long criteria checklist. In episode 853, Kirill Eremenko lists twelve: cost, modality, customization, inference options, latency, architecture, performance benchmarks, language, size, scalability, compliance and licensing, and environmental impact; Hadelin de Ponteves says in practice his choices, such as an Amazon Titan model and a Llama model, came down to which matched his fine-tuning dataset's format.
- Layered, reference-free metrics evaluate agents better than single accuracy scores. In episode 857, Coval's Brooke Hopkins says getting the most-used case wrong matters more than a 99% average, so metrics are combined: a short conversation is bad only if the goal also failed. Coval checks JSON-described workflow steps and function-call arguments, inferring correctness from context rather than golden datasets, and flags agents that take longer than humans or repeat themselves.
Mr. GlobalBob’s Back! This Time I Teach Him What OPEC Is 😂
- Saudi output alone says nothing; OPEC produces as one bloc. He answers a critic who pointed to record Saudi production in 2021–22 by citing OPEC-wide output: about 37 million barrels a day under Trump against 34 under Biden. He says he never claimed Saudi Arabia stopped pumping, only that OPEC cut production.
- Saudi gains in 2022 offset outages elsewhere in the cartel. He cites Nigeria's oil production falling nearly 12% in 2022, along with declines in Iraq and Egypt, so Saudi Arabia picked up the slack without OPEC as a whole producing more. Stabilizing supply and prices when unstable members lose output is, he argues, the point of a cartel.
- Saudi production charts track the OPEC 2020 cuts and restorations. He points to a big drop and a price spike, then a recovery in output as the OPEC 2020 deal wound down, and fresh cuts once it ended. He accuses the critic of dishonesty, since the same data was on the site the critic used.
Stock Tok$"Recession Proof" Stocks Top Of The List...
- This post includes paid content that is not available publicly. See the original source.
AI Analyst LabWhat is AI Ethics in Light of Deepseek? | Ep14
synopsis
The hosts talk with Raphael, an information systems scholar at the University of Sydney who studies the ethics of emerging technology and AI companions, about what DeepSeek changes. Raphael welcomes DeepSeek for leveling the field: it shows that capable AI can be built cheaply and openly. But Raphael argues that efficiency will not bring sustainability or safety on its own, and ends on a call for compassion-based design, binding regulation, and open, interoperable AI systems.
Why DeepSeek shocked the market. Before January 20, Raphael says, the prevailing belief was that competing at the frontier took billion-dollar data centers, venture money and the latest Nvidia chips. DeepSeek came from a China under export sanctions and matched o1 on independent benchmarks. By some estimates it used a tenth of the compute, and it was released under an MIT license. Nvidia lost $600 billion in a day, and more than $1 trillion came off big-tech valuations. Raphael adds that Nvidia has since recovered.
How it was done. Raphael treats the $6 million cost as an accounting figure that rests on earlier work. Raphael calls the theft charge ironic. The efficiency comes from mixture-of-experts routing with load balancing and from distillation into a smaller student model. Censorship lives in the app, not in the open model, which universities can download and retrain. Raphael also contrasts DeepSeek's privacy policy with OpenAI's vaguer one.
Business models. Raphael places OpenAI's advantage in API revenue, its enterprise ecosystem and its brand, and expects open models to push prices down. Raphael judges Deep Research to be ChatGPT with extra steps.
Efficiency and regulation. Under the Jevons paradox, Raphael argues, cheaper AI will be built into everything. The EU AI Act came into force on February 5, as Raphael puts it, but energy disclosure under it looks voluntary on a non-lawyer's reading. The US is heading the other way, and California's AI bill failed under lobbying. Raphael's prescription is mandated renewable energy and disclosure.
Companions and compassion. Companion platforms claim 10–30 million users, and 40% of Gen Z accept a partner having an AI companion. Raphael lists manipulation, political profiling, impersonation and suicide as the risks. Raphael's compassion paper asks for guardrails that, when a user is in distress, refer them to professional help instead of keeping them engaged. It also asks for open, interoperable systems that the government sets standards for, as it did with the internet.
- DeepSeek broke the belief that frontier AI needs frontier spending. Raphael, an information systems scholar at the University of Sydney, says the January 20 release came from an unknown startup in a sanctioned country. It matched GPT-o1 on major benchmarks (beating it on some, such as medical and ethical reasoning), used a tenth of the compute by some estimates, and shipped under an MIT license. Raphael cites Nvidia's $600 billion single-day loss and over $1 trillion wiped off big-tech valuations, noting that Nvidia has since recovered.
- Constraint, not theft, explains DeepSeek's efficiency. Raphael calls the reported $6 million cost an accounting figure built on earlier work, and the charge that DeepSeek "stole from OpenAI" ironic given how OpenAI got its own training data. The credit goes to a mixture-of-experts design with load balancing, plus distillation into a smaller student model. Censorship (Tiananmen Square, for instance) lives in DeepSeek's app, not in the open weights. Raphael calls government bans partly justified but a double standard, since OpenAI's own reports show it gave data to US agencies dozens of times in the first half of 2024.
- OpenAI's moat is ecosystem and brand, not technology. In Raphael's view the money is in the API, not ChatGPT, and OpenAI is protected by enterprise customers, consultants and being first to market, though open models will push prices down. Raphael calls Deep Research "ChatGPT with some extra steps": it still hallucinates and cannot tell good sources from bad, and Hugging Face released an open-source match within 24 hours.
- Cheaper AI will raise total energy use, not cut it. Citing the Jevons paradox, Raphael predicts AI will be built into everything. On a non-lawyer's reading, the EU AI Act, which Raphael says took effect February 5, leaves energy disclosure voluntary. Raphael wants regulators to require providers to use renewable energy and disclose their energy use.
- AI companions open the door to emotional manipulation and surveillance. From Raphael's own research: these platforms claim 10–30 million users, perhaps 3% of users are in such a relationship, 40% of Gen Z are comfortable with a partner having an AI one, and an MIT study of over a million ChatGPT logs found sexual role-play the second most common use. The harms Raphael names include political profiling, impersonation (a 7-year cyberstalking case involving James Florence) and a teenager's suicide. Raphael's IEEE Transactions on Technology and Society paper on "compassionate AI" calls for mandatory guardrails that, on detecting distress, refer the user to professional help instead of keeping them engaged.
Dr. Jon KrohnWhat is an Analytics Engineer, with Colleen Fotsch
- Analytics engineering is the data-modeling layer between ingestion and analysis. Fotsch says that at Podium, data engineering pulled data in and maintained pipelines, analytics engineering modelled the raw data in dbt for stakeholder use, and analysts built Tableau visualizations, working with the modellers on the questions being answered.
- Whether the role exists depends on the company. She says neither CHG, her current employer, nor Homie had a separate analytics engineering team, though the work still happens; at Podium, analysts gradually took on analytics engineering work as the team grew, which she credits with leveling up her technical skills.
- Knowing what tools exist matters more than having used them all. Fotsch says learning from what fails, and knowing where to look, is the core of the job; on her first day she wanted to take ten courses at once, and her sister-in-law, a data analyst, told her it was a marathon, not a sprint. Context on the wider pipeline, she says, helps with stakeholders and with knowing whom to ask.
Dr. Jon KrohnThe Secret to Landing Your First Data Job, with Colleen Fotsch
- Formal training builds the confidence to explain your value in interviews. Fotsch says a degree such as her Arizona State programme, plus Coursera courses she still uses, helps both on the job and in interviews; for analyst roles she prioritized SQL above all, some Python, and the popular BI tools Tableau and Power BI, whose skills transfer.
- A referral from someone who knows you conveys what a resume can't. She got her Homie interview through a friend from the fitness world working on the product side, who connected her to the data manager and could vouch for her work ethic and teamwork; she advises listing target companies and industries and finding anyone you know there, not only people in data.
- In-person meetups build a community worth having even when employed. Fotsch says Utah has many data meetups and the contacts she makes there are people she can later ask how they solved a problem; the host adds that newcomers with little knowledge are welcome at data analyst, data science and AI meetups listed on Meetup.com.
Mr. GlobalMore Proof #news #truth #gasprices #gas #oil #canada
- Trans Mountain shows Keystone XL would have raised prices, not lowered them. A news clip reports oil now flowing on the Trans Mountain expansion from Alberta to the BC coast, nearly tripling flow to 890,000 barrels a day and opening "lucrative" global markets for Western Canada Select, which has sold at a steep discount. He reads this as proof that both pipelines exist to reach more buyers and lift the price of Canadian crude.
Stock Tok$Inflation Data Is Useless, Watch This Instead
- This post includes paid content that is not available publicly. See the original source.
Technology and FriendsMark Tinderholt on GitHub AT AT and Terraform [EPISODE 840]
- The module automates the click-ops of setting up cloud operations. Tinderholt, who works in Azure Core Engineering, describes the GitHub AT-AT as an opinionated Terraform module library: running
terraform applylocally, with access to GitHub, Azure and Entra ID, creates a GitHub repo holding Terraform HCL code and GitHub Actions YAML, and sets up Entra ID federated identity credentials so those workflows can reach Azure. He calls it "using Terraform to Terraform": it uses the GitHub, Entra ID and Azure providers to drive those control planes rather than to provision VMs. - One foundation supports radically different release motions. Besides the basic "starter app", he has extensions for Azure Functions, a Jekyll static website and a VM workload that builds a Packer image and deploys it to an Azure VM. He runs his own blog, marktinderholt.com, on the Jekyll module (a pull request triggers a pipeline that publishes to Azure Storage) and stood up a Minecraft server with the VM module.
- The generated workflows build in a standard Terraform GitOps process. He says the GitHub Actions let you run a plan manually to check drift, or follow the full flow: open a pull request, review the plan it shows, and apply on merge, using pre-compiled plans. A team that already has a repo can pull it in with a Terraform data source and pass it to the module instead of having it created.
- Isolation is built in but optional, "choose your own adventure". Because Terraform state, kept in an Azure storage account, can hold sensitive values, he recommends isolating production and non-production backends. Multi-subscription support also brokers RBAC so each environment's identity holds only the privileges it needs there; someone just kicking the tires can skip both.
- It suits teams adopting Git flow, not lone wolves. He says little Terraform knowledge is needed, but users must know Git and work through feature branches and pull requests; someone working alone is probably better off running Terraform locally. He is the sole contributor but open to pull requests, publishes the module on registry.terraform.io, and says a sibling Azure DevOps AT-AT exists but has not been published in a while, though users have asked him to bring the GitHub version's features back to it.
Dr. Jon Krohn861: From Pro Athlete to Data Engineer — Colleen Fotsch’s Inspiring Journey
synopsis
Colleen Fotsch, a former elite swimmer, CrossFit Games athlete and national-team bobsledder, is now senior technical manager of marketing technologies on CHG Healthcare's data platform team. She argues that the habits of elite sport transfer to data work: learning to be coached, attempting unfamiliar things, and caring about the team. The confidence that had been missing came from going back to school, not from her talent. She ends where her two careers now meet: fitness programming built for busy desk workers, and the view that regular exercise is a productivity tool data professionals undervalue. The host, a recreational CrossFit competitor, departs from his usual format to tell her athletic story first, and it takes about the first 40 minutes.
Swimming and the shift to fitness. Fotsch swam a freshman year at Notre Dame, then transferred to Cal Berkeley, where she says she peaked later than most. She was on two NCAA national championship teams and a relay that broke an American record twice, and reached the 2008 Olympic trials. After college she became Cal's swimming and diving strength coach. She says training "just to be fit" felt anticlimactic after years of training to win, and a friend took her to a CrossFit regional. She signed up at a San Francisco gym the next day, drawn by training with others and by the daily whiteboard leaderboard.
Climbing the CrossFit ranks. The host explains the Open, a roughly $20 online qualifier that anyone can enter at an affiliate gym. He then reads her record from the Games site. She was 491st worldwide in her first Open in 2014, enough for the Northern California regional, where legless rope climbs and handstand walks humbled a strength-first athlete who had cherry-picked her training. In 2015 she rose to 113th and won workout 15.5 outright, a descending ladder of thrusters and rowing calories. In 2016 she was 80th, but finished seventh at Regionals, where five qualified, within about 20 points of a place.
Injury and the team route. An MRI after that season found her rotator cuff 80% torn, an overuse injury she thinks CrossFit strength had been holding together. After surgery she competed on a NorCal CrossFit team, then three men and three women, which won its regional and finished fourth at the 2017 Games. She says teammates counting on her let her push harder than she could alone. Back as an individual in 2018 she was 71st in the Open and won two Regional events, having moved to Arizona to train full-time while studying a remote kinesiology master's at Michigan State. A failed thruster in the final event cost her a qualifying spot, and she again finished seventh.
The 2019 Games and the turn outward. She was 33rd in the 2019 Open and reached the Games, placing 27th. That year's field took national champions, upwards of 100 per division instead of about 40. Fotsch says it was not the Games athletes had earned. It taught her she was not entitled to an outcome, and it made her start planning a career, which she says elite athletes tend to avoid thinking about.
Finding data. She says a strength coach's pay pushed her to look elsewhere. Her sister-in-law, a Netflix data analyst, described the field, and Fotsch realized she had long been a data nerd about her own training: weighing food, timing sleep and meals, and overlaying wearable data on workouts so her coach could judge load against recovery. She had resisted coding since her engineer father first suggested it, believing her gifts were athletic, not academic. The kinesiology master's showed her she could transfer her work ethic to study. Her first coding course was Sadie St. Lawrence's SQL course on Coursera, which the host says 700,000 people have taken.
Bobsled and the ACL. During the pandemic, a pilot who has won gold medals for Canada and now drives for the US invited Fotsch to try bobsled. She committed to a month at Lake Placid, learned to load the sled as brakeman, and made the national team in 2021 aiming for the 2022 Olympics. Months before qualification she tore her ACL in a weightlifting session. She began applying for analyst jobs from bed after surgery, while finishing an Arizona State master's in applied business data analytics.
Getting hired and moving up the stack. For a first data job she recommends formal training, a focus on SQL plus Tableau or Power BI, and above all a contact at a target company who can vouch for you. A friend from fitness on Homie's product side got her the interview there. Homie, a real-estate tech firm, charged buyers a flat fee, which the host contrasts with the standard 2% commissions on each side of a US home sale. At Podium, a software company that began by helping local businesses gain Google reviews, she moved from product analytics into analytics engineering. She describes that role as the dbt modelling layer between data engineering and analysts. She values dbt's documented, stakeholder-agreed definitions and its newer generated docs and doc blocks, and Sigma's Excel-like front end on Snowflake.
Data platform work now. At CHG, which places physicians in permanent and locum tenens roles, she centralizes and documents scattered marketing data so engagement can be tied to pipeline stage. She spent a few months on the data engineering team learning ingestion. She now delegates more and prioritizes timelines and requirements with stakeholders, while staying close to the technical work. Over Christmas she built her first app with Claude. Her advice to newcomers is to find data in something they already love.
PRVN GO and sustainable fitness. For months after retiring she still trained like a Games competitor, until she asked whether that training matched her new goals. Her minimal-equipment workouts led PRVN's CEO to approach her, and with the company of champion Tia-Clair Toomey she built PRVN GO, a programme without high-skill movements, for professionals and parents. The host says the episode is unsponsored. She calls exercise a productivity hack, recalls an 800-step day in her first analyst job, and tells the host, a self-described over-trainer, to limit all-out sessions to one or two a week. Her standard is consistency about 80% of the time.
- Fotsch reached the CrossFit Games after years of near misses. The host reads her record: 491st worldwide in the 2014 Open, 113th in 2015 (when she won workout 15.5 outright), 80th in 2016 but seventh at Regionals with five qualifying, fourth at the 2017 Games on a team after an 80% rotator-cuff tear, seventh again at 2018 Regionals, then 33rd in the 2019 Open and 27th at the Games; before that she won two NCAA titles with Cal and twice broke an American relay record.
- A flawed Games appearance pushed her to plan life after sport. Fotsch says the 2019 Games' expanded field of national champions, upwards of 100 per division against the usual 40, was not the Games athletes felt they deserved, and taught her she was not entitled to any outcome; elite athletes, she argues, avoid thinking about what comes next.
- She was already a data nerd from tracking her own training. Fotsch says weighing food, timing meals and sleep, and overlaying wearable data on workouts so her coach could balance load against recovery drew her to data, after her sister-in-law, a Netflix analyst, described the field; her kinesiology master's, she says, overturned a belief that she wasn't smart enough for technical work.
- An ACL tear ended her bobsled bid and hastened her data career. She made the US national bobsled team as a brakeman in 2021, aiming for 2022, but tore her ACL weightlifting months before qualification, and began job applications from bed after surgery while finishing an Arizona State business data analytics master's; a friend's referral got her an analyst role at Homie.
- Her path ran from analyst to analytics engineer to data platform manager. At Podium she moved from Tableau analysis into dbt modelling; at CHG Healthcare, a medical staffing firm, she is senior technical manager of marketing technologies within the data platform team, centralizing scattered marketing data, delegating more and building less. She credits CrossFit's "figure it out" habit for her comfort learning on the job.
Polis Pandit$Why Lex Fridman?
- This post includes paid content that is not available publicly. See the original source.
Mr. GlobalWhy I Do What I Do *Please Subscribe!*
- He fights trolls because those videos reach the most people. He says troll takedowns get his highest views and engagement, which serves his aim of countering energy misinformation. He describes himself as the only person doing this from inside the energy industry.
- His Canadian LNG count included planned projects, not just started ones. He had said seven LNG projects were underway, planned or about to start; a commenter objected that seven had not started; he says his original wording covered planned projects too. He shows a Canadian government list of seven LNG projects plus an infrastructure project, and argues the critic preferred tribal approval to checking the facts.
Stock Tok$Time To Fill The Cart 🛒
- This post includes paid content that is not available publicly. See the original source.
Dr. Jon KrohnSave 20-30% on LLM Costs With This Quick, Easy Trick
- Like a compiler, BAML handles the prompts nobody will hand-tune. Gupta says a decade of performance engineering taught him that compilers beat him on time per dollar, so a team should hand-optimize only its one or two most critical prompts and let tooling handle the other 90%.
- Schema-aligned parsing tolerates the model's malformed output instead of forcing strict JSON. Gupta says his team spent about eight months on an algorithm that coerces a response into the expected data model, fixing missing quotes and commas and strings given where numbers were expected, in under five milliseconds; he says customers save 20% to 30% of output tokens.
- The parser brings function calling to models released without it. Gupta says it needs no model modification, so BAML users were tool-calling with DeepSeek R1 and OpenAI's o1 models at recording, and a customer building a Cursor-like tool for Xcode had the LLM emit code in triple quotes, avoiding JSON escaping, which BAML turned into properly escaped strings.
- Semantic streaming lets chat products show a generative UI, not just text. He describes type-safe partial data driving interfaces such as recipe cards with a progress spinner per section, or live tickers and graphs for stock analysis, and argues a chat startup must ask why its users wouldn't just go to ChatGPT, with better UX as the answer.
Dr. Jon KrohnRAG Prompts Have Big Hidden Problems... But BAML Solves Them
- Good RAG depends on data-specific retrieval no general tool can supply. Gupta says his company started on RAG pipelines as one of its 13 pivots but moved away because pulling the relevant documents depends on each dataset, so there is no general RAG solution, just as there is no universal web accordion component.
- JSON pasted into prompts is filler the model must filter out. He calls them the prompt's "likes and ums": quotation marks, escape characters and colons make context parseable for code but harder for the model to read, and that the placement, orientation, size and surrounding text of images also matter; BAML renders the actual prompt live in VS Code so developers can spot what looks ugly, on the reasoning that models are trained on human data.
- Treating prompts as typed functions curbs the contradictions of long English instructions. Gupta says every BAML prompt is a function with type-safe parameters and a declared return type, such as an enum of categories, with each category's rule attached to that category rather than one growing essay; this, he argues, yields shorter prompts written like code.
Mr. GlobalTrump Taking Credit For Project Approved By Biden
- The touted US–Japan Alaska energy venture is not a government deal. He says the Alaska LNG project was green-lit under the Biden administration and is a $44 billion development between private companies, in the works for years. He reads Trump's announcement of a joint venture with Japan, and his claim that Biden refused to sell Japan LNG, as taking credit for it.
- The project monetizes North Slope gas that is now reinjected. Gas produced alongside oil at Prudhoe Bay is pumped back underground because it cannot reach market. The plan is an 800-plus-mile pipeline to the Kenai Peninsula carrying about 3.5 billion cubic feet a day, with an LNG export terminal and a carbon capture plant. He supports it, citing wasted resources and revenue for Alaskans.
- The terminal developer, not Washington, holds the Asian buyer contracts. The company building the terminal has contracts with buyers in Asia, possibly but not necessarily Japan. He argues politicians count on the public not knowing such details.
Dr. Jon KrohnIs Prompt Engineering a Joke or the Future of AI Craftsmanship?
- Gupta predicts prompt engineering will mature as front-end development did. Responding to a host premise, from his researcher, that it is a "joke discipline", Gupta says he once objected to the term "AI engineer" being used for prompt writers, but now sees prompting as being where web development was with plain HTML, no CSS and no interactivity, before it became a complex craft; he expects it to become a well-studied discipline over time.
- What prompting needs is reliability in transforming data. Gupta frames a prompt as a way to turn supplied data into the data you want, such as classifying a transcript as a doctor–patient conversation or general wellbeing, and says using a model involves more nuance than people credit.
Mr. GlobalAlaska LNG Project
- The Alaska LNG project would finally monetize gas that's already being produced and wasted. He explains that North Slope wells at Prudhoe Bay produce natural gas with no takeaway capacity, so it is currently reinjected underground; a roughly 800-810-mile pipeline to a Kenai Peninsula export terminal would carry about 3.5 billion cubic feet per day to market instead.
- He favors the project's structure because it pairs carbon capture with export and protects in-state supply first. The proposal pairs a carbon-capture system with the LNG terminal to offset emissions, and requires that Alaskans' own industry and supply needs be fully met before any gas can leave the state.
- The project is a $40-plus-billion proposal aimed at Japan that has not yet broken ground. He notes it uses a gas pipeline rather than an oil pipeline, avoiding oil-spill risk, and that Alaska residents would share in the project's revenue, though he says he's convinced the effort will ultimately not go forward.
Crosstalk SolutionsDoes DeepSeek send your data to China when it's run locally?
- DeepSeek run locally showed no traffic to China in the host's capture. Following David Bombal, who used Wireshark to show the DeepSeek smartphone app sends data to China, the host ran DeepSeek under Ollama and Open WebUI with Wi-Fi disabled and other apps closed, filtering for outbound TCP. A query produced almost no activity, and the only endpoints were in the United States.
- The local model answered with the network fully disconnected. With the Ethernet adapter disabled it still reasoned through and correctly answered an arithmetic question (12 apples).
- Local LLMs trade privacy for less processing power. The host concludes local models do not phone home to China or anywhere else, but lack the compute of OpenAI or DeepSeek; he still rates local running solid for general queries and document summarizing.
Stock Tok$Picture Perfect Stocks 📸
- This post includes paid content that is not available publicly. See the original source.
Dr. Jon Krohn860: DeepSeek R1: SOTA Reasoning at 1% of the Cost — with Jon Krohn (@JonKrohnLearns)
- DeepSeek R1 matches the top models on human-rated leaderboards. The host reports that at the time of writing R1 was statistically tied for first on the overall LM Arena leaderboard, within a 95% confidence interval, alongside GPT-4o and Google's Gemini 2.0 Flash, and that its release coincided with Nvidia's share price falling 17% and the Nasdaq several percent in a day.
- R1's real shock is its tiny training cost. In the host's rough approximation, a DeepSeek V3 or R1 training run costs on the order of millions of dollars against hundreds of millions for o1, Gemini or Claude 3.5 Sonnet; he credits a conceptual breakthrough combining mixture-of-experts with new efficiencies such as DualPipe, a GPU communications scheduler, across a couple of thousand GPUs, about 1% of what Meta and xAI say they buy in a year.
- The host argues US chip sanctions on China have proven ineffective. He says a Chinese firm approached US capability with about 1% of the chips, at about 1% of the cost, using less capable Nvidia chips, and that DeepSeek released V3 and R1 source code and weights under the MIT license, more open than Meta's Llama, which shares weights but not code and restricts companies with over 700 million active users.
- Avoid DeepSeek's app, but run its models privately. The host warns that, per its privacy policy, anything entered in DeepSeek's iOS app, number one on the App Store at recording, is stored on servers in China; he suggests Ollama for private use.
- Efficient training may strand the giant capital raises, the host argues. In his view, recent $6 billion raises by OpenAI, xAI and Anthropic and the $500 billion Stargate project probably assumed models would keep growing by orders of magnitude; he welcomes cheaper training as better for the environment and for building applications.
Crosstalk SolutionsTailscale Setup on GL.iNet: Remote Access Made Easy
- Tailscale is free for personal use and built into GL.iNet routers. It is a WireGuard-based VPN and SD-WAN tool, free for up to 100 devices and three users, where each device in the tailnet gets its own private IP. On GL.iNet routers it is labelled beta, so older firmware may need upgrading; the host demos his favourite travel router, the Beryl AX.
- Binding the router takes a toggle and a link. Enable Tailscale under Applications, then follow the device-bind link to connect it to your tailnet. The Tailscale admin flagged the router's client as outdated; the host found no way to update it but saw no problem, and expects a future GL.iNet release to update it.
- Reaching devices behind the router needs two extra settings. Tailscale alone reached only the router's admin page. Enabling "Allow Remote Access LAN" on the GL.iNet and approving its advertised 192.168.8.0/24 subnet under Edit Route Settings in the Tailscale admin let him reach an OpenSpeedTest server at 192.168.8.150:3000 from a separate network.
- Remote Desktop into a home PC needs Windows Pro and a fix. Windows 10 or 11 Home cannot accept RDP; enable Remote Desktop and allow RDP through Windows Firewall. Login with a Microsoft account failed until he ran a "runas" command with his MicrosoftAccount email and winver, from an older article he links.
- The host says the whole setup is quick, even for first-timers. He puts it at no more than 10 minutes start to finish and says he was impressed; a written blog post of the steps is linked in the description.
AI Analyst LabWhy Are Top Data Scientists Not Using AI? | Ep13
synopsis
The hosts interview a staff data scientist at Statsig, formerly an economist at Amazon, a product analytics data scientist at Meta and a manager at Tencent, who now teaches an AI course on Maven. The guest argues that generative AI is rapidly devaluing technical execution, so a data scientist's value lies in finding and scoping the problems worth solving. Because adoption still lags even in Silicon Valley, the guest sees a 10–18-month window in which early adopters gain an outsized edge, and ends on a recipe for preparing for it and for filtering out the noise.
Why skepticism persists. The guest started learning generative AI before ChatGPT's launch and says ML scientists, burned by earlier hype cycles, still dismissed it in 2024. The guest compares this to reviews that rated the Nokia above the iPhone, and to the 70 years it took cars to reach mass adoption: people overhype in the short term and underestimate in the long run.
What remains valuable. Asked about a LinkedIn post claiming that data scientist and analyst titles are becoming less relevant, the guest calls "data scientist" a vanity title, invented to recruit people away from Google. A student's two-hour Cursor build of a hair-pulling detector stands in for how fast technical skill is losing value. The co-hosts agree that identifying and framing problems is what lasts, and the guest's slide says difficulty is orthogonal to value. On communication, the guest calls it a tool rather than a destination. The guest's point is that you need enough depth in a skill to judge whether an AI answer is better, and that data science is a craft learned by doing.
The working setup. As of January 31, the guest recommends Cursor's agent mode and describes agentic systems as an LLM, a protocol for calling tools, and an orchestrator. The guest's setup uses o1 Pro to plan and Claude Sonnet to execute. In a clip, Cursor turned a SQLite database into a director-level weekly business review and fixed its own bug along the way. A co-host says ChatGPT once let them do several days of notebook analysis in about an hour.
The window. The guest says even top-company data scientists use AI too little because they are busy, and predicts mass adoption of agentic coding in 10 months and agentic-native products in 18. A host calls the gap an arbitrage. The guest adds that incumbents may still resist, as Kodak did. The guest's preparation route is Karpathy's GPT-2 walkthrough plus a published project, which the guest says takes about two months. To separate signal from noise, the guest applies the scientific method: build a mental model, hypothesize, predict, and test.
- AI devalues execution; the data scientist's value is problem-finding. The guest, a staff data scientist at Statsig (formerly at Amazon, Meta and Tencent) who teaches an AI course on Maven, tells of a student with no ML background who used Cursor to build a webcam hair-pulling detector end to end in two hours. The guest argues that difficulty is orthogonal to value, and calls "data scientist" a vanity title that should be defined by outcomes, not tools. A co-host splits the work into identifying the problem, framing questions, and execution, and expects AI to take over only the last.
- Communication is a tool, not the goal of growth. The guest cites a former Meta colleague who found ChatGPT gave better advice on sizing a team's quarterly goals. A host counters that a baseline of competence is still needed. The guest agrees, adding that without seeing a skill's many dimensions you cannot judge whether AI's answer is better, and calls data science a craft learned by doing.
- Cursor's agent mode is the guest's top AI tool today. Speaking on January 31, the guest calls it "objective-oriented programming": you state the result and Cursor does the rest, tuned with custom Cursor rules. The guest uses o1 Pro as planner and Claude Sonnet as executor. A demo automated a weekly business review from a SQLite database, and the agent found and fixed its own bug. Model and tool choices are tied to that date, and the guest expects them to change.
- Lagging adoption opens a short arbitrage window for early users. Even at top companies, the guest says, data scientists are too busy to learn AI. The guest predicts most of Silicon Valley adopts agentic coding within 10 months and agentic-native products reach most people within 18; a host frames the gap as arbitrage. A co-host reports cutting the time spent creating LinkedIn content by about 80%.
- An AI job now takes about two months of preparation. The guest's route: Andrej Karpathy's step-by-step recreation of GPT-2, some reading, then a project solving your own problem or copying one from Y Combinator's latest batch, which the guest puts at about 90% AI, published with documented learnings. To filter out the hype, the guest shows students a fake paper generated with ChatGPT in under five minutes and teaches the hypothesize–predict–falsify loop from Zen and the Art of Motorcycle Maintenance.
Dr. Jon KrohnThis Startup Pivoted 13 Times Before Striking Gold
- Y Combinator partners doubted the Slack-competitor idea that got Gupta admitted. Gupta says he and his co-founder, UT Austin graduates of 2015, tried edtech and the creator economy before applying to Y Combinator with a warmer alternative to Slack; on his fourth application, and third interview, they got in, though the partners told them the idea would not work.
- Fielding batchmates' machine learning questions revealed the real market. He says they joined the Winter 2023 batch as ChatGPT came out, and as more seasoned engineers, one with nine years in distributed systems and the other nine or ten in algorithms and assembly, found themselves answering mostly ML questions; they also concluded they shouldn't be building UIs, and the company went through 13 pivots in all.
- An SDK can't express what's needed; a new cross-language syntax can. Gupta says Python or TypeScript SDKs such as Vercel's AI SDK and Pydantic AI are limited by their host language, and since every software shop, whether C++, Java or Go, will use AI, the answer is a new syntax like JSON, YAML or TOML before it, which became BAML.
Mr. GlobalAnother Challenger For Mr Global - It’s Exhausting
- The critic's chart showed OPEC exports, not production. He argues exports do not move global prices; production does. On his production chart OPEC held at 37 (million barrels a day) in 2016, 2017 and 2018, so it did not cut when Trump took office, and its 35 in 2019 reflects the crash Trump pushed for. He attributes the 2014–2016 export slump to US output doubling under Obama while Chinese demand weakened.
- The 2018 increase exceeded the 1.2 million barrels a day cited. The critic quoted minutes of OPEC's 174th meeting on June 22, 2018. He points to a meeting the next day where Russia added 600,000 barrels a day, taking the total to 1.8 million, and to conformity levels of 152% and 147%, which he reads as members overproducing their quotas.
- The price collapse shows the real increase was larger. Because OPEC's own figures cannot be trusted, he infers volume from price: oil fell from well over $70 a barrel to about $28.91. He argues that a fall of that size makes his 2 million barrels a day figure conservative.
- Trump's OPEC flood hurt the US energy sector. He says Trump got foreign producers to flood the market about a year after the largest oil crash in history, while US companies were still working off debt from it, damaging employment and the sector. He adds that Trump tried the same thing on his third day in office this term, and denies any partisan motive of his own.
Andrej KarpathyDeep Dive into LLMs like ChatGPT
synopsis
This is a general-audience walk through the entire pipeline behind a model like ChatGPT, in three stages — pretraining on internet text, supervised fine-tuning into a conversational assistant, and reinforcement learning that teaches the model to reason — using GPT-2, Llama 3, and DeepSeek-R1 as running examples, and closing with the cognitive quirks (hallucination, bad arithmetic, "Swiss cheese" failures) that fall out of how these systems actually compute. The destination is not a single trained model but a mental framework: understanding a chat reply as a lossy, statistical simulation of a human labeler, one token at a time, so the viewer can use these tools productively without over-trusting them.
Building the training data. The pipeline starts with Common Crawl, which has indexed roughly 2.7 billion web pages since 2007. Companies filter this raw HTML aggressively — blocking malware, spam, and adult sites; stripping markup down to plain text; running language classifiers (Hugging Face's FineWeb keeps only pages over 65% English, a deliberate choice that trades multilingual strength for English quality); deduplicating; and scrubbing personally identifiable information. What survives is surprisingly small given the internet's size: FineWeb totals about 44 terabytes, small enough to fit on a few consumer hard drives.
Tokenization. Before any of this text reaches a neural network, it has to become a sequence of symbols from a finite vocabulary. Raw UTF-8 bytes would give only 256 possible symbols and extremely long sequences, so production systems run byte-pair encoding, repeatedly merging the most common consecutive symbol pairs into new symbols. GPT-4 settles on about 100,277 tokens this way — a compression that trades a larger vocabulary for a shorter, cheaper-to-process sequence. Run through this process, FineWeb's 44 terabytes becomes about 15 trillion tokens.
Training the base model. A neural network — specifically a Transformer — takes a window of tokens (up to some maximum context length) and outputs a probability for every possible next token. Training means repeatedly sampling windows, checking the true next token against the model's guess, and nudging the network's parameters — billions of simple numeric "knobs" — so the correct token becomes slightly more likely. The output of this stage, after enough of these updates, is a base model: an internet document simulator with no persistent identity, no tools, and no ability to answer questions on demand — just a very good autocomplete. He demonstrates GPT-2's own numbers as a scale reference: 1.6 billion parameters, a 1,024-token context window, trained on about 100 billion tokens, originally costing roughly $40,000 in 2019. He reproduced it himself for around $600 on rented 8×H100 GPU hardware — and thinks $100 is achievable today — crediting better-curated data and much faster hardware and software, not a smarter algorithm. He also demonstrates that base models memorize high-quality, frequently-repeated sources almost verbatim (reciting a Wikipedia zebra article from memory) and that prompted cleverly — with few-shot examples, or a fake "conversation between a human and an AI assistant" transcript — a raw base model can already imitate an assistant, even though it was never trained to be one.
Supervised fine-tuning: manufacturing a personality. A base model is not useful for answering questions, so the next stage keeps the exact same training algorithm but swaps the dataset: instead of internet documents, the model now trains on curated conversations where human labelers write both the prompts and the "ideal" assistant responses, following lengthy company labeling instructions (OpenAI's InstructGPT paper is the source example, hiring contractors through Upwork and Scale AI). This stage is far cheaper than pretraining — hours instead of months — because the conversation dataset is much smaller. Modern practice increasingly uses LLMs themselves to help generate or edit this data (e.g., the largely-synthetic UltraChat dataset), but the process is still fundamentally anchored in human-written labeling instructions. The key reframe he offers: a ChatGPT answer is not oracle-like intelligence, but a neural network's statistical imitation of what a skilled human labeler, following those instructions, would have written — which is also why asking a model "what are you, who built you" produces confabulated answers (it will often just claim to be ChatGPT by OpenAI, because that phrase is common in its pretraining data) unless a company explicitly hardcodes self-identification into the fine-tuning data or a hidden system message.
Hallucination and its fixes. Because the model imitates the confident tone of its training conversations, when asked about someone it has never heard of it will invent a plausible-sounding but false biography rather than say "I don't know" — demonstrated live on the older Falcon-7B model inventing three different fake identities for a made-up name across three tries. Meta's Llama 3 paper describes the fix: interrogate the model with automatically generated questions about documents, use another model to judge whether its answers are correct, and where the model reliably fails, add training examples where the ideal response is an explicit "I don't know" — teaching the model to connect its own internal uncertainty to an honest refusal rather than a confident guess. A second, stronger mitigation is tool use: introducing special tokens that let the model emit a search query, pause generation, and have retrieved web text pasted into its context window, converting a "vague recollection" in the frozen parameters into a directly accessible fact in working memory — the same distinction, he notes, between something you memorized long ago and something you just looked up.
Tokens as a finite thinking budget. Because a Transformer performs a fixed, modest amount of computation per output token, no matter how many layers deep, a model can't correctly solve a hard problem if the label it was trained on crams the entire calculation into one token. Working through "Emily buys 3 apples and 2 oranges..." he shows that a labeled solution jumping straight to "$3" teaches the model to guess arithmetic instantly and it fails once the numbers get bigger (23 apples, 177 oranges), while a solution that writes out intermediate steps succeeds because each step only demands a small amount of per-token computation. The same constraint explains why models are weak at spelling and counting — they see tokens, not individual letters, so a task like printing every third character of "ubiquitous" or counting dots in a string routinely fails, and the "how many Rs in strawberry" meme traces to this same limitation, compounded by weak counting ability. His practical fix in both cases is the same: ask the model to write and run code rather than trust its "mental" arithmetic or letter-counting. He also flags an unresolved oddity — models frequently claim 9.11 is bigger than 9.9, and one research team traced this to neurons associated with Bible verse numbering lighting up and biasing the comparison, a reminder that these systems have a "Swiss cheese" pattern of capability: brilliant at Olympiad-level problems, yet capable of tripping on something trivial.
Reinforcement learning: the "practice problems" stage. He frames the three training stages as mirroring a textbook: pretraining is reading the exposition, SFT is studying worked examples with expert solutions, and RL is solving practice problems where only the final answer is known — the exact solution path is left for the model to discover through trial and error. Concretely, the model generates many candidate solutions to the same prompt, the correct ones are identified automatically against a known answer, and the model is trained to make more use of whatever token sequences led to correct answers — sequences discovered by the model itself, since a human labeler cannot reliably guess which reasoning steps are easy or hard for a given model's "cognition." Unlike the earlier two stages, which have been standard for years, RL is described as still early and non-standardized industry-wide — which is why DeepSeek's R1 paper, publishing its RL methodology openly, was such a significant event: it showed accuracy on math benchmarks climbing over thousands of training steps, alongside average response length growing dramatically as the model spontaneously developed backtracking behavior like "wait, that's not right, let me reevaluate" — a chain-of-thought strategy that emerged from optimization rather than being written by any human labeler. Given the same "Emily and the apples" prompt, R1 visibly double-checks its own arithmetic from a second angle before finalizing an answer, in contrast to a plain SFT model that just states one path to the answer. He draws an explicit parallel to DeepMind's AlphaGo, where reinforcement learning against self-play eventually surpassed models trained purely to imitate human experts, and produced "move 37" — a move so statistically unlikely for a human to play (about 1 in 10,000) that commentators initially thought it was a mistake, yet it proved brilliant; the open question he poses is what an equivalent breakthrough looks like in open-ended reasoning rather than a closed game like Go, and whether it might even involve the model finding a "language" for thinking that isn't English.
RLHF and why it's not quite RL. Verifiable domains like math have an automatic scoring function, but creative domains — writing a joke, a poem, a summary — don't. Reinforcement Learning from Human Feedback solves this with indirection: humans only need to rank a handful of model outputs (easier than writing an ideal response from scratch), and those rankings train a separate reward model to imitate human preference, which the language model can then be optimized against cheaply and at scale. The catch is that the reward model is itself a large neural network with exploitable blind spots: run RLHF too long and the policy discovers adversarial, nonsensical outputs — his example is a "joke" of literally "the the the the the" scoring near-perfectly — because reinforcement learning is extremely good at finding whatever quirk in the reward model produces a high score, whether or not it makes sense to a human. Unlike math RL, which can run for tens of thousands of steps because the scoring function (does the boxed answer match) can't be gamed, RLHF must be capped after a few hundred updates before it collapses into gibberish. His verdict: RLHF is real reinforcement learning technically, but not "RL with magic" — it's closer to a modest fine-tuning boost than an open-ended capability unlock, which is why GPT-4o-class models (mostly SFT plus a light layer of RLHF) behave differently from dedicated "thinking" models like o1, o3-mini, or R1, which have gone through genuine RL on verifiable problems.
What's next, and how to keep up. He expects near-term progress along several axes: native multimodality, where audio and images are tokenized the same way as text (spectrogram slices, image patches) and handled inside one model rather than stitched together from separate systems; longer-running, more autonomous "agents" that string together tasks over minutes or hours under human supervision, changing the operative metric to a human-to-agent ratio rather than human-to-robot; deeper integration and computer-use capability (citing ChatGPT's Operator as an early example); and open research problems like test-time training, since current models have no equivalent of a human's ability to actually learn from an ongoing experience rather than only holding it temporarily in a context window. For staying current, he points to the Chatbot Arena leaderboard (while cautioning it has become somewhat gameable and can rank models like Sonnet lower than actual usage would suggest), the "AI News" newsletter, and following knowledgeable people on X, alongside practical venues for using models directly: each provider's own site or app for proprietary frontier models, aggregators like Together AI for open-weight models such as DeepSeek and Llama, Hyperbolic specifically for base-model access, and LM Studio for running smaller, distilled, lower-precision models locally. His closing framing is the one he wants to stick: these are extraordinarily useful but fallible tools, riddled with "Swiss cheese" gaps even as they clear PhD-level benchmarks, and the correct posture is to use them for a first draft or for inspiration, verify their work, and keep ownership of the final result.
- Pretraining compresses filtered internet text into a "base model" that is a document autocomplete, not an assistant. Common Crawl's raw HTML is filtered by URL, language, and PII before tokenization (fine-web ends up ~44 terabytes, 15 trillion tokens for ~100,000 vocabulary symbols via byte-pair encoding); training GPT-2-scale (1.6B parameters, 100B tokens) cost about $40,000 in 2019 but he reproduced it for around $600 in a day, thanks to better data curation and much faster hardware and software.
- An assistant's personality comes entirely from supervised fine-tuning on human-written conversations, not from any deeper "understanding." Because this stage is algorithmically identical to pretraining except the dataset swaps to curated human-labeler conversations, a chat reply is best understood as a statistical simulation of a professional labeler following a company's instructions — which is also why unprompted "who are you" answers are hallucinated unless a system message or hardcoded self-description is added.
- Models must spread reasoning across many tokens because each token gets a fixed, small amount of computation. Demonstrated on "Emily buys 3 apples and 2 oranges": a solution that jumps straight to the boxed answer forces one token to silently contain all the arithmetic and fails on harder numbers, while one that writes out intermediate steps succeeds — the same fixed-compute-per-token limit explains why models miscount letters in "strawberry" and misjudge whether 9.11 exceeds 9.9, and why routing arithmetic or counting to a code interpreter is more reliable than trusting the model's "mental" math.
- Reinforcement learning on verifiable problems lets a model discover its own reasoning strategies, which plain imitation can't produce. DeepSeek-R1's training showed response length and backtracking language ("wait, let me reevaluate") emerging on its own as accuracy climbed, echoing AlphaGo's move 37 by finding solutions no human labeler would have written; RLHF, by contrast, trains against a learned human-preference simulator that reinforcement learning can eventually game (e.g., "the the the the" scoring near-perfectly), so it must be stopped after a few hundred updates rather than run indefinitely like true verifiable-domain RL.
Crosstalk SolutionsTurn Any PC Into a Secure Machine with Tails OS
- Tails boots any PC from USB and leaves no trace. Building it needs a USB drive of at least 8 GB (USB 3 recommended for speed): download the image from tails.net, verify it, and write it with balenaEtcher, choosing the target carefully because it is wiped. The host PC may need its BIOS boot order changed to boot from USB.
- Optional persistent storage is encrypted with a passphrase. The first-run wizard offers it, and you choose which data persists between boots, such as documents, bookmarks and Wi-Fi passwords.
- Tor masks location and activity but is not bulletproof. Tails offers connecting through Tor and bundles the Tor Browser, Pidgin for secure messaging, and network-monitoring and encryption tools; rebooting and removing the stick ends the session.
Scott HanselmanEPISODE 9 - Scott & Mark Learn To... Public Speaking
synopsis
Hanselman and Mark, each about to give a talk, arrive at the same conclusion: technical speaking is a set of learned techniques — knowing the audience, time-boxing the content, managing nerves and keeping momentum — and even after decades both still prepare, get nervous and stumble.
How much to prepare. Mark has given thousands of talks and preps little for incremental content, but their joint prepared demos take 10 to 15 hours. Hanselman, with six talks in a week, preps from 10 minutes to a couple of hours, more for high stakes: a 45-minute AI Tour slot in Johannesburg before 5,000 people, with backup machines, and an upcoming TEDx Portland talk.
Knowing the audience. Mark's first step is defining the target listener; for TEDx they sketch someone familiar with chatbots but not transformers. Hanselman starts from what people should leave with, and both play to a friendly face on each side of the room.
Structuring time. Mark splits his minutes across sections, writes clock times and checks them during the talk. Hanselman uses 5-minute intro and outro bumpers around seven 5-minute sections in three acts, keeps a notebook from every talk, and urges beginners to keep two or three talks ready for a no-show slot.
Nerves. Mark does a power pose and biofeedback to bring his heart rate under 100. Hanselman sees 105 to 110 and reframes it as his body preparing him to succeed.
Delivery. Mark primes himself to speak slowly and accept his first words, likening stumbling to watching your feet running downhill, and one host has swapped "um" for silence over about 10 years. Hanselman credits Billy Hollis for staging tricks, and recommends yes-and in two-person talks, and Toastmasters.
- Prep time scales with the stakes and how new the material is. Mark says after thousands of talks he preps little because his content is incremental, yet their joint prepared demos take 10 to 15 hours. Hanselman lists six talks in one week prepped from 10 minutes to a couple of hours, against a 45-minute AI Tour slot in Johannesburg before 5,000 people with backup machines, and a TEDx Portland talk of 15 to 18 minutes that is getting in his head.
- Design the talk for one defined target listener. Mark says to decide who the audience is and what they know, aiming at that person and briefly bringing others up with "for those of you not familiar"; not knowing the audience is when he has been flummoxed. Hanselman says to start from what people should leave with, such as something they can implement that day, and to remember the room wants you to win; both pick a friendly face on each side to play to.
- Time-box sections and keep the plan on paper. Mark divides his minutes across sections, rebalances them, writes clock times to check against, and memorizes slide order because unrehearsed transitions look amateur. Hanselman splits 45 minutes into 5-minute intro and outro bumpers plus seven 5-minute sections in three acts, keeps notebooks from every talk, and advises beginners to keep two or three talks ready, since in his experience one or two conferences in ten lose a speaker.
- Pre-stage nerves are physiology to manage, not a warning. Mark uses a power pose and biofeedback, watching his heart rate and breathing to get it under 100 before going on. Hanselman says his heart rate runs 105 to 110 before a talk and he reframes it as his body sending blood to his brain to help him succeed.
- Keep forward momentum instead of second-guessing words. Mark primes himself to speak slowly and accept the words he picks, likening stumbling to watching your feet while running downhill; one host says he has replaced "um" with silence over roughly the last 10 years, which they agree audiences do not notice. Hanselman credits Billy Hollis with staging tricks such as assigning client and server sides of the stage, and recommends Toastmasters as proof that speaking is learned technique.
Stock Tok$2 Small Caps That Caught My Eye 👀
- This post includes paid content that is not available publicly. See the original source.
Dr. Jon KrohnBAML is a New Language for AI Applications: This is Why We Need It
- BAML aims to do for prompts what React did for HTML strings. Gupta, CEO and co-founder of Boundary, says that as a Meta intern he saw how concatenated HTML strings let one unclosed div break the news feed, and that React's new syntax combining HTML and JavaScript gave instant error squiggles, state management and hot reloading; BAML, "basically a made-up language", applies that idea to LLM calls.
- Ever more prompts in codebases demand tooling that catches errors. Gupta predicts each year's codebase will hold more prompts than the last, and that models of all kinds will get easier for non-experts to use, which calls for developer tooling for consumers of models rather than their producers.
- Prompts need TypeScript-style type safety and static analysis. He argues that however good models get, their use is still code that must fit a codebase, and that BAML's algorithmic work reframes prompting the way TypeScript's type safety improved JavaScript.
GeoHussarDiscussion w/ Political Scientist Kari Oderman and Veteran Rebel Intel about US Foreign Policy
synopsis
The conversation ends with political scientist Kari Odermann holding that the US and Europe are in one of the most dangerous periods since the late 1980s: Western optimism about Russia was misplaced, Putin's war was planned rather than provoked, and Trump's territorial rhetoric weakens an international order Americans also depend on. Veteran Rebel Intel, leaning right, holds that US strategy is mostly continuous across administrations but that NATO expansion made the war foreseeable; the host's own view is that a cold-blooded US establishment profits from building and then strangling rivals.
Credentials and the information war. Odermann describes a PhD from the Free University of Berlin, Fulbright research on how Baltic states negotiate with superpowers, and 15 years in policy advocacy communication. She argues governments must take up space on platforms like TikTok, citing the AfD's reach among German schoolchildren, and that algorithms are underrated narrative-shapers. The host, who attended NATO's 75th summit as an invited creator, calls DeepSeek a piggyback on ChatGPT that rattled Nvidia but welcomes it as democratizing AI.
Texas. Rebel Intel joins and describes north Dallas suburbs growing at Sim City speed, including the PGA headquarters move to Frisco and a planned third airport, with infrastructure lagging. The host credits lax building rules for restraining Texas home prices, which nationally have doubled in 12 years; Rebel Intel agrees about 90%.
Trump and the public. Asked about Greenland, Canada and the Panama Canal, Odermann says trust rests on consistency and that an informed public sees the global consequences; the host doubts the public resists propaganda. Both stress community-building as the counter.
Thucydides and two-track diplomacy. Asked by Rebel Intel whether US aid to Russia and China's WTO entry created rivals, Odermann blames optimism about Kremlin behaviour and the habit of imposing democracy. The host lays out idealist and "dark side" diplomacy, citing the USSR, Japan and China, and claims China's deflation leaves it no currency escape.
Rightward drift. Rebel Intel worries about Putinism, Xi, Milei, Farage and Trumpism turning toward offensive realism; Odermann cites loss of institutional trust as a hallmark of sliding into authoritarianism. The host offers FDR and the "sick chicken" Supreme Court case as an earlier overreach that met a breaking point.
Federal cuts. On the Washington helicopter crash, Rebel Intel declines to speculate technically; the host guesses controller shortages and safety-committee cuts contributed.
Engineered war? Rebel Intel cites the 1975 Helsinki principles and Burns's cables to argue NATO expansion was a known red line and the war plausibly engineered; the host counters that NATO does not invade, but calls the war a cheap way to deplete Russia. Odermann redirects to the cost calculation: what losing part of Ukraine would mean for future security spending.
Russia and China in decline. The host cites 30% of Russian GDP on the military and refinery strikes; Odermann blames yes-men and false casualty reporting. Rebel Intel and the host predict Chinese decline, citing the one-child policy from 1980 to 2015 and a ratio of 1.3–1.4 men per woman. Odermann closes recommending Anne Applebaum's "Autocracy, Inc."
- Mainstream institutions lose by refusing to occupy social media. Kari Odermann, a political scientist and CEPA fellow, cites a German Green co-chair who refused TikTok while the far-right AfD reached schoolchildren there and did well in eastern Germany. She urges value-first, fact-based posting and warns algorithms will increasingly shape narratives, citing early concern over DeepSeek misleading by omission.
- US diplomacy runs two tracks, idealist and calculating, the host argues. The host's thesis is that a "dark side" of money and the defense establishment builds rivals up — the USSR, Japan, China — then strangles them. The host says China is now caught in deflation, unable to move its currency without hurting either imports or exports, and applies the same patient logic to Chinese lending in Africa.
- The panel splits on whether Ukraine's war was engineered. Rebel Intel, a right-of-centre veteran, cites William Burns's cables on NATO red lines since 2008 as grounds for a "reasonable inference" of engineering, while blaming Putin. The host replies NATO does not start wars but calls the war a cheap win for the defense industry; Odermann holds Putin planned it rather than fell into a trap.
- Russia's war economy is being hollowed out, the host argues. The host puts Russian military spending at 30% of GDP in 2025, against about 3% for the US versus roughly 10% in the 1950s, and says weekly Ukrainian drone strikes on refineries cut fuel for troops and exports. Odermann adds that a leader surrounded by yes-men, with casualty figures not reaching the Kremlin, decides badly.
- Trump's territorial talk erodes norms even if never acted on. Odermann argues that claims on Greenland, Canada and the Panama Canal by a Security Council member open an Overton window that Rwanda's Putin-style incursion into Congo already exploits, and distrust of institutions is an early sign of authoritarian drift. The host links air-traffic-controller shortages and FAA cuts to the Washington helicopter crash as a guess.