What is the binding constraint on scaling AI compute?

Jonathan Ross · Feb 17, 2025

By avoiding HBM, Groq has effectively no manufacturing scale limit, whereas GPUs are constrained because memory is the hard-to-manufacture part

GPU logic uses the same silicon process as mobile phone chips, which are made first because they yield better; the only real difference and the only scale-limited component is the memory

29:26 20VC: NVIDIA vs Groq: The Future of Training vs Inference | Meta, Google, and Microsoft's Data Center Investments: Who Wins | Data, Compute, Models: The Core Bottlenecks in AI & Where Value Will Distribute with Jonathan Ross, Founder @ Groq

Jonathan Ross · Sep 29, 2025

There will be a compute supply constraint, driven not only by NVIDIA's buying power but by capital costs, conservative memory suppliers, and suppliers' incentive to keep HBM margins high

You must commit capital more than two years in advance and AI demand is hockey-sticking faster than even NVIDIA's cash flow can pre-commit to; and increasing HBM supply would compress the very high margins suppliers currently enjoy

18:07 20VC: OpenAI and Anthropic Will Build Their Own Chips | NVIDIA Will Be Worth $10TRN | How to Solve the Energy Required for AI... Nuclear | Why China is Behind the US in the Race for AGI with Jonathan Ross, Groq Founder

Andrew Feldman · May 26, 2026

Memory is the second biggest supply chain constraint after TSMC fab space, and it constrains all GPUs but not Cerebras, which doesn't use HBM

Only three companies — Samsung, Micron and Hynix — make the HBM that GPUs use, and they could not keep up with demand, sending prices through the roof

10:01 20VC: Cerebras CEO on the Future of Data Centres, Token Costs and Memory | We are Not in an Infra Bubble & Dario Got a Bad Deal with Elon for Compute | Should US Companies Sell to China & Why Most Layoffs are AI Washed with Andrew Feldman

Andrew Feldman · May 26, 2026

If demand stays high, memory shortages will persist for at least the next several years

Fab capacity is extremely lumpy — you can't add a little capacity; a new fab costs $40B and takes five years, so supply responds as a step function that lags demand explosions by years

Scope: conditional on demand staying high

10:54 20VC: Cerebras CEO on the Future of Data Centres, Token Costs and Memory | We are Not in an Infra Bubble & Dario Got a Bad Deal with Elon for Compute | Should US Companies Sell to China & Why Most Layoffs are AI Washed with Andrew Feldman

Ethan Mollick · Jul 31, 2024

If AGI arrives, demand for intelligence on demand will be effectively infinite, which makes compute the currency and energy the binding constraint

Anyone with an AGI would want it monitoring medical records, airspace, generating scientific ideas, and doing personal tasks simultaneously — intelligence on demand is power hungry and demand for it has no ceiling

Scope: Conditional on AGI actually being achieved and instantly useful

55:51 20VC: Is More Compute the Answer to Model Performance | Why OpenAI Abandons Products, The Biggest Opportunities They Have Not Taken & Analysing Their Race for AGI | What Companies, AI Labs and Startups Get Wrong About AI with Ethan Mollick

Hemant Taneja · Sep 22, 2025 · hedged

You cannot get AI right without getting energy right, which makes energy infrastructure the product gap GC most needs to solve

All of GC's transformation work depends on AI, AI depends on energy, and the new demand creates an opportunity to move toward sustainability profitably — though in the short term the only real option in the US is natural gas

Scope: GC is early in its thinking here; short-term solution is natural gas, not sustainable sources

65:16 20VC: General Catalyst CEO Hemant Taneja on The Future of Venture Capital: Chanel vs Walmart | Lessons Scaling GC to $40BN in AUM | Investing $5BN+ Into Stripe Over 14 Rounds | Investing Hundreds of Millions into Anthropic at $60BN Valuation

Andrew Feldman · May 26, 2026

Sam Altman, Elon Musk and others hold that the business is turning electricity into intelligence and therefore electricity is the ultimate limiting factor

25:12 20VC: Cerebras CEO on the Future of Data Centres, Token Costs and Memory | We are Not in an Infra Bubble & Dario Got a Bad Deal with Elon for Compute | Should US Companies Sell to China & Why Most Layoffs are AI Washed with Andrew Feldman

Alex Schultz · Sep 5, 2025

Data center buildout is badly lagging demand because of physical supply-chain bottlenecks — energy generation, generators, steel, fiber optics and switches — not because of will or capital

The global supply chain was never built to supply this volume: you can't spin up gigawatt-scale generation in one place, only so many large generators are ordered per year, and there's a global fiber optic shortage as everyone buys thousands of kilometers

Scope: humanity can build the supply chain out over a few years; after which a new bottleneck appears — TSMC, ASML, substrate

16:39 20Growth: Meta CMO Alex Schultz on How All Founders Have to Change Their Marketing Playbook in a World of AI | Is AI Plateauing, What it Means if China Wins the AI Race and Why Zuck is a Generational Leader

Aravind Srinivas · Jun 15, 2026

Physical build-out time is the permanent bottleneck on frontier AI capability, which is why vertically integrated infrastructure that converts GPUs, networking, power and cooling into frontier output tokens carries real value

Today's models were trained on Hopper; Blackwell-generation models and then Vera Rubin capacity next year will be far more powerful, and the lead time to build is what gates them

30:42 20VC: Micron Will Be More Valuable Than Meta | How Export Controls Helped Not Hurt China | Power is the Bottleneck to AI | Why Dario Has Done a Disservice to AI with his Labour Replacement Messaging with Aravind Srinivas, Founder @ Perplexity

KR Sridhar · Jun 29, 2026

Bloom can stand up power faster than customers can build their data centers, so power delivery is not the bottleneck — the real constraints are data center construction time, permits, and gas supply

Bloom's own delivery capacity outpaces data center build timelines; the remaining dependencies sit outside Bloom's product

Scope: as of right now

29:44 20VC: Leo Aschenbrenner's Largest Holding: Inside the $90BN Bloom Energy | Why Electricity, Not AI Models, Will Decide the Winners of the AI Race | Why We Are Not in an AI Capex Bubble | Energy Sovereignty and The Future of Power with KR Sridhar

Roman Chernin · Jun 8, 2026

Over a six-month horizon, additional capital cannot change an infrastructure company's position — you must deliver with the capacity you already have

Six months is too short a time for capital to convert into delivered capacity

Scope: six-month horizon

0:00 20VC: Nebius Co-Founder on AI Infrastructure Bubbles | The Real Impact of Open Source on OpenAI & Anthropic | How Price Elastic is Demand for Compute | Could Nebius Sell 10x More Compute If They Had It & more with Roman Chernin

Roman Chernin · Jun 8, 2026

The binding bottleneck on compute build-out depends entirely on time horizon: capital cannot help within six months, can accelerate somewhat over twelve, and clearly unlocks capacity at eighteen to twenty-four months

Build-out proceeds in phases — secure power and land, build data centers, then fill with GPUs — each requiring more capital, and Nebius builds a portfolio of capacity in parallel rather than one site

Scope: applies to a portfolio of projects, not a single data center

50:58 20VC: Nebius Co-Founder on AI Infrastructure Bubbles | The Real Impact of Open Source on OpenAI & Anthropic | How Price Elastic is Demand for Compute | Could Nebius Sell 10x More Compute If They Had It & more with Roman Chernin

Jonathan Ross · Sep 29, 2025

Both leading labs are compute-constrained in revenue-visible ways: Anthropic's biggest complaint is rate limits, and OpenAI throttles by running its chat service slower, which reduces engagement

With more compute they could produce more tokens and charge more money

12:30 20VC: OpenAI and Anthropic Will Build Their Own Chips | NVIDIA Will Be Worth $10TRN | How to Solve the Energy Required for AI... Nuclear | Why China is Behind the US in the Race for AGI with Jonathan Ross, Groq Founder

Andrew Ng · Nov 17, 2025

Many AI companies currently have excess demand — a rare problem — because there aren't enough semiconductors, data centers and electricity to supply the tokens people want

GenAI created very valuable workloads like AI-assisted coding, and users get rate-limited on products like Claude Code because supply can't meet demand

4:51 20VC: Andrew NG on The Biggest Bottlenecks in AI | How LLMs Can Be Used as a Geopolitical Weapon | Do Margins Matter in a World of AI? | Is Defensibility Dead in a World of AI? | Will AI Deliver Masa Son's Predictions of 5% GDP Growth?

Andrew Feldman · Oct 6, 2025

The first and most fundamental bottleneck in AI is human expertise, not chips or power

Universities aren't minting enough AI practitioners or data scientists who understand data pipelines, and US immigration restrictions cut off the J-1/H-1B pipeline that historically drew the best and brightest — which is also why top talent commands extraordinary compensation

33:07 Human expertise not chips or power is the fundamental bottleneck

David Cahn · Aug 5, 2024

Labour is the single biggest cost in building data centres, and the sheer physical mobilisation required is what most people are not paying attention to.

The build chain runs from hyperscaler to real-estate developer to general contractor to subcontractors who must recruit thousands of electricians into remote towns and house them.

18:29 Labor and physical workforce mobilization is the overlooked binding constraint on data center buildout

David Cahn · Aug 5, 2024

The binding constraint on AI buildout is the industrial supply chain: manufacturers of generators, steel and batteries don't believe the hyperscalers' demand forecasts and won't add factory capacity.

A generator manufacturer must build a new factory to expand, and if Microsoft stops buying, that capital sits dormant — so suppliers refuse to double capacity on a promise.

33:14 Supply chain manufacturers distrust hyperscaler demand forecasts and wont add capacity

KR Sridhar · Jun 29, 2026

Deploying tens of gigawatts quickly is achievable because the technology is a solid-state device built on electronics supply chains rather than traditional power supply chains

At tens of gigawatts the company would still consume only single-digit percentages of an existing electronics supply chain that already knows how to scale to millions of units, as it did for the smartphone; the manufacturing lines were designed from the outset for fast scaling

28:02 Power hardware scales on electronics supply chains not power supply chains

Jonathan Ross · Sep 29, 2025

The huge raises by OpenAI and Anthropic are not driven by building their own chips; they are driven by data centers, which cost more per year than the chips

Data centers are amortized over ten-plus years versus three to five for chips, so even at a third of annual cost the data center ends up costing more per year

19:22 Data center annual cost exceeds chip cost and drives fundraise scale

Anjney Midha · Apr 14, 2026

Financing compute infrastructure is not a fundraising problem but a systems-design problem — aligning equity, debt and mission-aligned balance sheets into a de-risked structure legible to large capital allocators.

Large long-term balance sheets want to help frontier scientists get compute but lack the OpEx to spend on it; once the structure is right the capital is available.

31:25 Financing structure not capital availability is the constraint

Andrew Feldman · May 26, 2026 · speculative

The case against energy being the binding constraint is that you bump into a different limit first — models stop getting smarter, which breaks the assumption that it keeps making sense to feed them more energy

The energy-bottleneck view has a built-in assumption that models keep improving enough to justify more energy input

25:34 Model improvement stalls before energy binds

Eiso Kant · Oct 7, 2024

Unlimited money cannot currently buy unlimited compute advantage, because interconnecting GPUs at scale faces real algorithmic and physical limits

Interconnecting more than 32,000 GPUs is extremely challenging today and 100,000 is only just becoming possible; million- or ten-million-GPU training clusters face both unsolved algorithmic problems and physical world limits — which is why a company with 10,000 GPUs can still exist

25:47 Cluster interconnect limits cap compute advantage regardless of capital

Eiso Kant · Oct 7, 2024

Cash does not straightforwardly convert into compute at frontier scale; the world still has far more demand for GPU compute than supply

Even as a frontier AI company they had to do enormous work understanding the market, building relationships and having plans A through Z; the demand-supply imbalance has not changed in the last six months

26:40 Demand for compute outstrips supply regardless of capital available

Eiso Kant · Oct 7, 2024 · hedged

A lot of knowledge does move around between AI labs as people jump between companies

38:27 Money buys compute directly but not talent data or proprietary research

Alex Schultz · Sep 5, 2025 · hedged

The lag before supply catches up with demand across most of these AI infrastructure bottlenecks is about eighteen months

18:56 Eighteen month lag before supply catches up with demand

Jonathan Ross · Feb 17, 2025

The industry will slightly overbuild power now, then pull back after it goes unused, and then face a hard power bottleneck in three to four years when real demand arrives

Around 20 gigawatts is being made available against roughly 15 gigawatts of existing global data center capacity, but chips double every 18-24 months, so 15 gigawatts becomes ~120 and then ~240, far beyond available power

31:25 Power overbuild then pullback precedes a hard bottleneck in three to four years