Will ASIC chips displace NVIDIA GPUs as the substrate for AI workloads?

Harry Stebbings · Apr 10, 2025

Nvidia and Jensen Huang will not passively cede the inference era despite their architecture being training-optimised

It is implausible that Jensen would simply enjoy the training era while it lasts and accept being displaced

71:09 20VC: How to Fix the UK Tech Ecosystem | Why We Need to Flood the UK with Venture Capital | What the UK Can Learn From Sequoia, Stripe and Norway | Why Now is the Time to be Bullish on China & Lessons from Jensen Huang with Tom Hulme & Stan Boland

Stan Boland · Apr 10, 2025

Nvidia is already making architectural changes to GPUs for inference and should be expected to keep moving toward inference efficiency

Jensen pays very close attention to the market and Nvidia has the cash and resource to move or build an organisation in any area it wants, organically or by acquisition

71:35 20VC: How to Fix the UK Tech Ecosystem | Why We Need to Flood the UK with Venture Capital | What the UK Can Learn From Sequoia, Stripe and Norway | Why Now is the Time to be Bullish on China & Lessons from Jensen Huang with Tom Hulme & Stan Boland

Jerry Murdock · Feb 28, 2026 · hedged

NVIDIA's acquisition of Groq is motivated by the need to make CUDA viable for the coming explosion of ASIC chips, not merely by handling different workloads and putting memory on the chip

Groq knows how to put memory directly on the chip, which is effectively an ASIC; NVIDIA knows what's coming and needs CUDA to support ASICs

Scope: his inference about an unstated motive

11:22 20VC: Why Cursor is Dead | An AI Tsunami is Coming & You Need to Prepare | Systems of Record Become Valueless Databases with Agents | Is This The End of Tech Private Equity with Jerry Murdock, Co-Founder of Insight Partners

Tomer Cohen · Dec 20, 2023

We will see verticalization of AI software onto dedicated chips, driven by closed, highly resourced optimized systems from companies like Apple and Microsoft

Chip design has always been about halving power, doubling capacity and lowering cost, and that same pressure now applies to AI workloads

Scope: most likely from the closed side of the open-vs-closed divide

16:09 20VC Roundtable: Spotify, Adobe & Linkedin CPOs on How AI Changes The Future of Product, Why AI is Now the Product, How TikTok Changed Product, Why Cost is the Biggest Barrier to LLM Usage & Why Incumbents Can Adopt AI Faster Than Any Prior Innovation Cyc

Jonathan Ross · Feb 17, 2025

Groq's inference cost is more than 5x lower than GPUs — the GPU's operating cost alone to produce the same tokens equals Groq's total CapEx plus OpEx

The memory in the latest GPUs alone costs more than Groq's fully loaded CapEx per deployed chip, and Groq uses about a third of the energy per token; over three years one third of Groq's cost is OpEx and two thirds CapEx

Scope: for inference workloads; over a three-year period

27:54 20VC: NVIDIA vs Groq: The Future of Training vs Inference | Meta, Google, and Microsoft's Data Center Investments: Who Wins | Data, Compute, Models: The Core Bottlenecks in AI & Where Value Will Distribute with Jonathan Ross, Founder @ Groq

Jerry Murdock · Aug 22, 2026

ASIC chips are ideal for model customization and specialization, where GPUs are unnecessarily expensive

The build-out is entering a model-specialization phase that doesn't require GPU-class hardware, which is why so many teams are now designing ASICs

Scope: specific to model specialization workloads

28:05 20VC: The AI Bubble Will Burst: Half the Neoclouds Will Die | China: Should We Ban Chip Exports & Be Fearful of Chinese Open-Source | Mag7: Who Dies and Who Thrives: Why Meta is Meh and Microsoft is Mega

Jonathan Ross · Feb 17, 2025

Training should be done on GPUs and NVIDIA will sell every GPU it makes for training, so cheap inference chips displacing NVIDIA's ~40% inference share would not reduce its unit sales

The more inference you have the more training you need and vice versa, so shifting inference to lower-cost chips increases training demand to absorb the same number of GPUs

22:55 20VC: NVIDIA vs Groq: The Future of Training vs Inference | Meta, Google, and Microsoft's Data Center Investments: Who Wins | Data, Compute, Models: The Core Bottlenecks in AI & Where Value Will Distribute with Jonathan Ross, Founder @ Groq

Jonathan Ross · Feb 17, 2025

NVIDIA is not a competitor to Groq: they sell a different product, since they offer neither fast nor low-cost tokens, while they have solved training so decisively that it isn't worth attacking

NVIDIA does training better than anyone by such a wide degree that it's a solved problem, and Groq shouldn't solve solved problems

Scope: distinguishes training from inference

26:49 20VC: NVIDIA vs Groq: The Future of Training vs Inference | Meta, Google, and Microsoft's Data Center Investments: Who Wins | Data, Compute, Models: The Core Bottlenecks in AI & Where Value Will Distribute with Jonathan Ross, Founder @ Groq

Jonathan Ross · Feb 17, 2025

Groq is one of the best things to happen to NVIDIA, because it lets NVIDIA sell every chip into high-margin training while offloading the low-margin, high-volume inference business

NVIDIA doesn't have to sully its margins with commoditized inference; Groq takes roughly 20% upfront margin on those deals

30:31 20VC: NVIDIA vs Groq: The Future of Training vs Inference | Meta, Google, and Microsoft's Data Center Investments: Who Wins | Data, Compute, Models: The Core Bottlenecks in AI & Where Value Will Distribute with Jonathan Ross, Founder @ Groq

David Luan · Jun 24, 2024

Displacing Nvidia in chips is incredibly hard but possible, and the economic returns are high enough that companies will do it

Google's TPU team was under 500 people on a shoestring budget and still taped out good chips every generation that trained Gemini and PaLM and are used by third parties — a counterexample to perpetual chip dominance

Scope: Nvidia is executing extremely well

26:25 20VC: Why Foundation Model Performance is Not Diminishing But Models Are Commoditising, Why Nvidia Will Enter the Model Space and Models Will Enter the Chip Space & The Right Business Model for AI Software with David Luan, Co-Founder @ Adept

Eric Vishria · Sep 25, 2024

Nvidia will not be the only game in town at the AI infrastructure layer, and the market's current setup wrongly assumes that if AI is real with real ROI, Nvidia simply keeps running at this level

The whole current pricing is built on that single assumption, which he doesn't believe

Scope: framed as a contrarian belief most around him disbelieve

52:53 20VC: Benchmark's Eric Vishria on Where is the Value in AI: Chips, Models or Apps | Why Nvidia Will Not Be The Only Game in Town | The Commoditisation of Foundation Models | Which AI Apps Have Sustaining Value vs Hype and Short Term Revenue

George Sivulka · Jan 22, 2025 · hedged

NVIDIA's stranglehold holds for training but not inference, so rival chipmakers may take meaningful inference share

CUDA is what ML scientists were trained on during their PhDs, but for inference it matters much less which stack you use

41:24 20VC: Why All AI Companies Are Under-Valued | The Future of Foundation Models: Scaling Laws, Generalised vs Specialised, Commoditised? | From Unable to Afford Rent to Raising $130M From Index and Peter Thiel with George Sivulka @ Hebbia

George Sivulka · Jan 22, 2025

The macro shift from training to inference will slightly destabilize NVIDIA's dominance, because inference can be deployed on AMD chips or custom architectures even when models are trained on NVIDIA

CUDA locks in researchers who train models, but the moment models are deployed they can run on cheaper infrastructure, and all major model providers are exploring custom silicon for inference

Scope: still bullish on NVIDIA overall; destabilization is 'slight'

50:05 20VC: Why All AI Companies Are Under-Valued | The Future of Foundation Models: Scaling Laws, Generalised vs Specialised, Commoditised? | From Unable to Afford Rent to Raising $130M From Index and Peter Thiel with George Sivulka @ Hebbia

Jonathan Ross · Sep 29, 2025

NVIDIA's position does not weaken as workloads shift to inference — it will sell every GPU it builds and can charge even higher margins, even if far more inference chips than GPUs ship

More inference creates more need to train models optimized for that inference, and more training creates demand for more inference deployment to amortize training cost — a virtuous cycle that raises GPU demand

Scope: holds even if Groq supplies 10x as many LPUs as GPUs

39:34 20VC: OpenAI and Anthropic Will Build Their Own Chips | NVIDIA Will Be Worth $10TRN | How to Solve the Energy Required for AI... Nuclear | Why China is Behind the US in the Race for AGI with Jonathan Ross, Groq Founder

Brendan Foody · Jun 1, 2026 · hedged

NVIDIA will not have the same monopoly in five years as we move to a multi-chip future, but even at 30-40% share of the largest market in the world it would still be the world's most valuable company

Cerebras is executing well, Etched is progressing, and most of the labs are building in-house chips — but the market is so large that a minority share still dominates

Scope: framed as a guess

51:21 20VC: Mercor CEO on Why Application Layer Companies Have No Defensibility, The Model is the Product | Token Spend Will Exceed Headcount Spend in 5 Years | The True Cost of Hiring AI Researchers in the Valley Today with Brendan Foody

Byron Deeter · Aug 25, 2025

It is no longer an NVIDIA-only world — chipsets from Amazon, Google and AMD are becoming quite capable, and new optimization paths will unlock leaps in training capability and inference cost

More hardware approaches and solutions are coming to market

30:33 Multiple hyperscaler chipsets are becoming competitive alternatives to nvidia

Jerry Murdock · Feb 28, 2026

Meta's refusal to buy from NVIDIA was a deliberate bet on ASIC chips, and was a gutsy correct call rather than being behind

They had the balls to say no to Jensen, which only makes sense as an ASIC bet

12:14 The asic bet was the gutsy correct call

Jerry Murdock · Feb 28, 2026

The choice between ASICs and NVIDIA chips will be decided by autonomous agents empirically testing options, not by developers reasoning from experience

Agents are probabilistic — they will spin up ten libraries in ten sandboxes, run the workload, and pick whichever performs better, whereas developers decide from prior experience

12:55 Agents will decide empirically not developers by experience

Steeve Morin · Feb 24, 2025 · hedged

Groq's claim that NVIDIA is not built for inference and will not win that market is technically right but practically wrong — NVIDIA will keep the inference market because its chips are actually available to buy today

availability matters enormously; you can buy an NVIDIA chip on a tab in your browser, and even after an H100 bubble bust those chips will be on the market and people will run inference on them

25:32 Nvidia keeps inference market via availability despite technical arguments for alternatives

Steeve Morin · Feb 24, 2025

Chips like Cerebras' and Grok's are so expensive because they rely on large amounts of on-chip SRAM, which consumes die surface, makes chips bigger and hurts yield

SRAM is extremely fast memory but takes up chip surface, making the chip larger and raising the probability of defects, so it is terribly expensive

27:14 Large on chip sram designs raise cost via die size and yield problems

Steeve Morin · Feb 24, 2025 · hedged

Unit cost on inference chips can be reduced by using a smaller process node and hooking to external memory rather than going all-SRAM

Some SRAM is needed, but a full SRAM design forces you to pay the price — there is no magic

28:54 Smaller process nodes plus external memory cut inference chip unit cost versus all sram

Steeve Morin · Feb 24, 2025

HBM remains necessary and NVIDIA's buying position in it is still meaningful; scaling SRAM instead is a dead end

Scaling SRAM means scaling chip surface, which creates compounding yield and design problems; both HBM and some SRAM will be needed, delivered by better dedicated architectures

31:11 Hbm remains necessary and scaling sram further is a dead end

David Frankel · Aug 8, 2026 · hedged

Photonic computing will be the technology that disrupts NVIDIA, or NVIDIA will acquire those companies

Everything in the data center that can be fiber already is; the chip is the last piece that hasn't been converted, and optical chips will be far more energy efficient — nothing stays the same

68:04 Photonics not asics is the eventual disruptor

Harry Stebbings · Feb 17, 2025

NVIDIA could cut its 70-80% margins and become radically more cost-competitive with Groq, destroying Groq's price advantage

Their margin gives them enormous room to drop price

30:20 Nvidia could cut margins to erase groqs cost advantage

Andrew Feldman · Mar 24, 2025

In five years NVIDIA will hold roughly 60% of the market, down from approximately all of it today — an outcome between the evenly-shared cloud market and Uber-style 90% dominance

38:03 Nvidia settles around sixty percent share down from near monopoly

Andrew Feldman · Mar 24, 2025

NVIDIA will keep a meaningful business in both training and inference — it won't cede inference — but the market's 100x growth leaves room for other very large companies

They are a world class company that has had one of the great decades in corporate history, going from around $10bn in 2014 to today's valuation, and they are exceptional at training; they won't roll over and play dead in inference

38:17 Market growth lets nvidia retain large share in both training and inference while others grow too

Andrew Feldman · Mar 24, 2025

Potential NVIDIA customer unhappiness over chip delays is a huge opening for competitors

When customers can't get their gear they may as well test somebody else's, and when a dominant player stumbles everyone piles on — as happened with Intel

44:46 Nvidia customer dissatisfaction over delays opens door for competitors