- AI chip memory costs have surged to 63% of total component spending, up from 52% in early 2024.
- AI chip memory costs — driven almost entirely by HBM — account for roughly $20 billion of new spending in 2025.
- Total component spend on AI chips nearly tripled from $22 billion in 2024 to $52 billion in 2025.
- Logic die costs stayed flat at 13–14%, while packaging and auxiliary components quietly shrank as a share.
- AI chip memory costs have surged to 63% of total component spending, up from 52% in early 2024.
- AI chip memory costs — driven almost entirely by HBM — account for roughly $20 billion of new spending in 2025.
- Total component spend on AI chips nearly tripled from $22 billion in 2024 to $52 billion in 2025.
- Logic die costs stayed flat at 13–14%, while packaging and auxiliary components quietly shrank as a share.
AI Chip Memory Costs Are Eating the Stack
If you want to understand where the money actually goes inside an AI chip, forget the processor. Forget the packaging. Forget the auxiliary components that keep everything humming. In 2025, AI chip memory costs are the story — and they’re getting bigger by the quarter. According to new data from Epoch AI, memory’s share of total AI chip component spending climbed from 52% in Q1 2024 to 63% by Q4 2025. That’s not a rounding error. That’s a structural shift in where value — and cost — is accumulating inside the most important hardware category in tech right now.
Epoch’s analysis covers chips designed by four of the most consequential players in AI infrastructure: Nvidia, AMD, Google, and Amazon. For each chip, the researchers estimated per-unit costs across four categories — HBM memory, logic dies, advanced packaging (specifically CoWoS), and auxiliary components — then multiplied those by estimated quarterly production volumes to model total spending. The result is one of the clearest pictures we have of where the AI hardware supply chain is actually straining.
What HBM Is and Why It Costs So Much
High Bandwidth Memory, or HBM, is the fast, stacked DRAM that sits directly alongside the compute die inside chips like Nvidia’s H100 and H200. Unlike conventional memory, HBM is manufactured through a complex 3D stacking process that bonds multiple DRAM layers together using through-silicon vias. It’s faster and more energy-efficient than GDDR memory, but it’s also significantly more expensive to produce — and there are only a handful of companies in the world that can make it at scale. SK Hynix, Samsung, and Micron are the three main suppliers, with SK Hynix widely understood to hold a commanding lead in HBM3E, the current-generation variant found in Nvidia’s Blackwell chips.
The economics here matter. When AI chip memory costs spike, it’s not because chip designers suddenly want to spend more on memory. It’s because the models they’re accelerating — large language models, diffusion models, recommendation engines — are increasingly memory-hungry. The trend toward larger context windows, more parameters, and faster inference all put pressure on memory bandwidth and capacity. Designers respond by stacking more HBM on each chip. More HBM means higher per-unit memory cost. And when you multiply that by the sheer production volumes Nvidia and others are now running, the numbers get very large very fast.
The Numbers Behind the Shift
Total component spending on AI chips grew from approximately $22 billion across all of 2024 to roughly $52 billion in 2025 — a 136% increase in a single year. That’s extraordinary on its own. But inside that number, HBM spending alone accounted for around $20 billion of the increase. Put another way, if you removed HBM from the equation, AI chip memory costs — and overall component spending — would have grown much more modestly. Memory isn’t just the largest cost category; it’s the primary engine of overall cost growth.
Meanwhile, the other three categories all shrank as a proportion of total spend. Advanced packaging — the CoWoS (Chip-on-Wafer-on-Substrate) process that TSMC uses to integrate HBM and compute dies together — fell from 19% to 15%. Auxiliary components dropped from 15% to 9%. Logic dies, which include the actual compute silicon, stayed roughly flat at 13–14%. That last figure is quietly striking. The brains of the chip — the part doing the actual matrix multiplication — now represents barely one-seventh of component costs. Memory has lapped it, twice over.
What This Means for Chip Designers
For Nvidia, this dynamic is something of a double-edged situation. On one hand, Nvidia designs some of the most HBM-dense chips on the market. The H200 packs 141GB of HBM3E. The B200 pushes even further. That gives Nvidia’s chips a genuine performance edge for AI workloads, but it also means their manufacturing costs are increasingly at the mercy of HBM pricing and availability — both of which are controlled by a small oligopoly of memory manufacturers. Rising AI chip memory costs therefore represent both a competitive moat and a structural vulnerability for Nvidia.
AMD finds itself in a similar position. Its MI300X accelerator was notable precisely for its enormous 192GB HBM3 capacity — more than Nvidia’s H100 at launch — which became a key selling point with hyperscale customers who care deeply about fitting large models into a single device. But that memory advantage comes with a cost structure that looks a lot like what Epoch’s data describes: memory-dominated, and exposed to the same supply constraints.
Google and Amazon, designing custom silicon through their TPU and Trainium lines respectively, have more control over how they balance compute and memory in their architectures. But they’re still buying from the same small pool of HBM suppliers. Nobody designing at the frontier has found a way around this particular bottleneck — at least not yet. For all four companies, AI chip memory costs remain a shared constraint that no architectural cleverness has fully resolved.
The Supply Chain Story Nobody’s Talking About Enough
There’s a broader industry tension embedded in these numbers. The conversation around AI hardware tends to fixate on compute — on FLOPS, on transistor counts, on the latest process node from TSMC. But AI chip memory costs tell a different story. The constraint isn’t always compute. Increasingly, it’s memory: how much you can fit on a chip, how fast data can move in and out of it, and how much it costs to manufacture.
HBM production is genuinely hard to scale. It requires advanced packaging capabilities that are concentrated in a tiny number of facilities globally. TSMC’s CoWoS capacity has been a known bottleneck for Nvidia’s supply chain over the past two years, and while TSMC has been aggressively expanding that capacity, demand has consistently outpaced supply. The fact that CoWoS packaging’s share of costs fell from 19% to 15% doesn’t mean it got cheaper in absolute terms — total spending rose sharply, so 15% of $52 billion is still a lot of money. It just means memory grew faster.
Samsung and SK Hynix have both announced major HBM capacity expansions. Micron is investing heavily in HBM3E production to close the gap with its Korean rivals. But building out semiconductor manufacturing capacity takes years, not months. The supply chain for HBM is tight, strategically sensitive, and increasingly central to which AI chip programs actually ship on schedule — and which ones don’t. Until that capacity catches up with demand, AI chip memory costs will remain elevated and supply will remain a binding constraint.
Where This Trajectory Points
If AI chip memory costs have gone from 52% to 63% of component spending in roughly six quarters, the obvious question is: where does it stop? The honest answer is that it probably depends on how AI architectures evolve. If the industry moves toward more memory-efficient model designs — smaller, more specialized models running at the edge, or new architectural approaches that reduce memory bandwidth pressure — the HBM intensity of each chip might plateau. There’s active research into processing-in-memory approaches that could change the equation more fundamentally.
But in the near term, the trajectory is clear. The next generation of frontier models will be larger, not smaller. Inference at scale — serving billions of queries — demands enormous memory capacity. And chipmakers are competing on memory specs as much as compute specs. Nvidia’s GB200 NVLink rack systems stack HBM to previously unimaginable levels. That’s not a direction that makes memory cheaper or less dominant in the cost stack.
For investors, hyperscalers, and anyone trying to model the economics of AI infrastructure, Epoch’s data is a useful corrective to compute-first thinking. The real cost story in AI hardware is increasingly being written in memory — and as AI chip memory costs continue their upward climb, SK Hynix, Samsung, and Micron have as much influence over that story as Nvidia, AMD, or any chip designer. That’s a power shift in the AI supply chain that hasn’t received nearly enough attention.
Source: https://epoch.ai/data-insights/ai-chip-component-cost-shares
