Frank Basso, vice president of infrastructure at Lightning AI, where the host is a fellow, walks through how a GPU cloud physically gets built and ends on a defence of the industry: in his account modern AI data centers neither consume water nor raise residential power bills, and their power demands are driving fuel cells, DC distribution and grid batteries. Lightning had over 35,000 GPUs at recording and expected about 50,000 by year-end.
Density changes everything. Five years ago, Basso says, an 18 kW rack was standard and Microsoft's 50 kW the extreme; 18 kW is now one server. Cabinets weigh about 4,200 lb against 800–900 lb before, ending raised floors. Nvidia's answer is to pack 72, 144 or 288 GPUs per liquid-cooled cabinet, heading toward half a megawatt. Lightning's GB300 superpod of about 10,000 chips in Chicago draws 20 MW in a 10,000 sq ft room that once would have been a whole 100,000–200,000 sq ft building's load.
Colocation at hyperscale. Builds start at 10 MW. Lightning asks partners how much grid power a site has, new build or brownfield, then designs to it: for GB300 a serviceable unit is 1,152 GPUs in 16 cabinets, and nine units come to about 10,300 GPUs, plus networking, storage and a growing CPU share. The host relays a rough figure of GPU-to-CPU ratios moving from 12 to 1 toward one to one as agentic workloads grow. Lightning derates nameplate power by 81%, which Basso calls more conservative than the 70–75% most use, contracting under 20 MW against a 22 MW plate rating. Build costs run 1 to 4 million per megawatt, recovered by the provider through a per-megawatt monthly fee over contracts usually 10 years long.
Designing for unknown tenants. Nvidia's reference architectures are a minimum, Basso says; Lightning adds no oversubscription anywhere in its network because, unlike xAI or OpenAI, a multi-tenant neocloud never knows what customers will run. Some push nearly a terabit through its private interconnects or about a gigabyte per GPU per second at storage, and inference is burstier than training. Lightning built metro networks and a nationwide backbone it likens to a telecom carrier's, and offers both bare metal and managed Kubernetes.
Four networks. East-west links every GPU to every other at full line rate, 400–800 Gb now and soon 1.6 Tb per GPU, over InfiniBand or RoCE, a split he calls a religious argument. Picosecond timing limits these links to about 50–500 m, so data halls group nodes physically; crossing halls through super-spines can break training without tuning. North-south carries internet, CPU and storage traffic; above it sit a management network and a true out-of-band "break-glass" network, which he says many skip but lights-out operation requires. Liquid-to-chip cooling has cut GPU failures, though Lightning staffs its sites 24/7.
Cooling, water and noise. Liquid cooling here is not immersion: filtered glycol on a sealed secondary loop hands heat to the building's loop, then to banks of car-radiator-style heat-rejection units. Basso says water-spraying towers are now nearly banned and a large site uses about a household's monthly water. Halls reach 105–110 decibels; staff wear earplugs plus passive earmuffs, since active cancelling produces equal sound pressure, and use intercoms and hand signals.
Power bills. Basso attributes residential rates to regulation, citing California prices from 10 to 58 cents per kWh. He says new sites fund grid upgrades worth millions to tens of millions, must feed backup power to the grid in emergencies, and increasingly use water-producing fuel cells, and he forecasts 800 V DC with grid batteries for 5–10% efficiency gains. The host concedes some see gas plants as a problem, cites The Economist that higher bills are almost certainly not due to data centers, and states his own techno-optimism.