Does AI Really Need So Much HBM?
AI has created an enormous winner in the memory business.
SK hynix.
At the center of that success is HBM, or High Bandwidth Memory.
HBM has become one of the most important components in the AI boom.
But recently, Cathie Wood raised an interesting question.
She remains highly optimistic about AI. Yet she has been reluctant to own memory companies such as SK hynix and Micron.
That sounds strange.
If AI keeps growing, shouldn't it need more HBM?
And if it needs more HBM, shouldn't the companies making it keep benefiting?
The answer may be more complicated.
First, What Does HBM Actually Do?
Think of AI as a giant kitchen.
An NVIDIA GPU is an incredibly fast chef.
But even the fastest chef can't cook if the ingredients arrive too slowly.
So the chef needs a very fast supply of ingredients nearby.
That's roughly what HBM does.
GPU = the chef
HBM = the high-speed ingredient station next to the chef
As AI models process more data, that fast access to memory becomes increasingly valuable.
That's one reason demand for HBM has exploded.
And SK hynix is currently the leader.
According to market data cited by Reuters, SK hynix held roughly 58% of the global HBM market in the first quarter of 2026.
The company also plans to roughly double its overall wafer production capacity over the next five years as AI memory demand grows.
The basic story looks simple:
More AI
↓
More GPUs
↓
More HBM
↓
More demand for companies like SK hynix
But Cathie Wood is looking one step further.
What Happens When HBM Gets Too Expensive?
HBM is difficult to manufacture.
Demand is enormous.
Supply has been tight.
That can be very good for the companies selling it.
But for the companies buying it, expensive memory is a cost.
And technology has a habit of doing something interesting when an important component becomes too expensive.
Engineers start asking:
Do we really need this much of it?
That's essentially Wood's argument.
She has described memory as one of the more cyclical and commoditized parts of the semiconductor industry.
Her broader point is simple:
If HBM becomes expensive enough, the incentive to use less of it becomes stronger.
She has pointed to companies including Groq and Cerebras as examples of different approaches to AI computing.
The important part is that Wood isn't pessimistic about AI.
It's almost the opposite.
Her argument is that AI could become so large that cheaper ways of running it become increasingly valuable.
Groq Is Trying a Different Approach
Groq also builds chips designed to process AI workloads quickly.
But it doesn't solve the problem exactly the same way as a conventional GPU system.
Return to our kitchen.
A typical GPU system has an incredibly fast chef with a large, very fast ingredient station nearby.
Groq focuses more heavily on keeping the ingredients it needs extremely close to the chef and moving them in a highly predictable way.
The technical details get complicated.
The basic idea doesn't.
Moving data takes time, energy and money.
Reduce unnecessary data movement, and some AI workloads may become cheaper or faster.
This becomes particularly interesting in AI inference.
Inference is simply what happens after an AI has already been trained.
Ask an AI:
“Plan a three-day trip to Tokyo.”
The process of generating that answer is inference.
As AI spreads into search, shopping, businesses, cars, robots and everyday software, the world could generate an enormous amount of inference.
Then a different question becomes important.
Not just:
How powerful is the AI?
But:
How much does each answer cost?
That's the market Groq is trying to attack.
Cerebras Is Taking Another Route
Cerebras approaches the problem differently.
But it starts with a similar observation.
AI requires enormous amounts of data to move between memory and computing hardware.
So what if more of that data could stay closer to where the computation happens?
Cerebras designed an unusually large AI processor partly around this idea.
Again, the technical details aren't the important part.
The important part is that:
NVIDIA GPUs paired with HBM are not the only possible way to build AI computing systems.
And this isn't just a laboratory experiment.
In August 2026, Cerebras announced its new CS-4 system, targeting the rapidly growing AI inference market.
The company has also outlined plans to provide 600 megawatts of computing capacity by the end of 2027.
Money and engineering talent are already moving into alternative approaches.
Cathie Wood Isn't the Only One Asking the Question
This is where the story becomes more interesting.
If this were simply one famous person's opinion, it might not matter much.
But semiconductor companies are also trying to solve the same underlying problem.
Qualcomm, for example, has argued that as AI inference grows, the bottleneck increasingly isn't just:
How fast can we calculate?
It's also:
How efficiently can we move data?
The company is developing another approach called High Bandwidth Compute, or HBC, which aims to bring computing closer to memory.
You don't need to remember the name.
What matters is the pattern.
Groq is trying one approach.
Cerebras is trying another.
Qualcomm is exploring another.
Different technologies.
Similar question:
Can AI move less data and still do more work?
And the more expensive memory becomes, the more valuable a good answer to that question can become.
But That Doesn't Mean HBM Is Going Away
This is where the other side of the story matters.
There is a reason AI companies use HBM.
It's extremely good at moving huge amounts of data quickly.
And right now, demand remains very strong.
SK hynix leads the HBM market and is spending heavily to expand production.
The company has said it expects strong AI-driven memory demand to continue.
So this logic is far too simple:
Groq exists
↓
AI stops using HBM
↓
HBM companies lose
Reality could look very different.
In fact, something that sounds contradictory could happen.
AI systems could use less HBM per unit of computing while the total HBM market still becomes much larger.
Here's why.
A Very Simple Calculation
This isn't a forecast.
It's just a thought experiment.
Imagine there are 100 AI systems.
Each needs 10 units of HBM.
Total demand:
100 × 10 = 1,000
Now technology improves.
New architectures appear.
Existing GPUs become more efficient.
And each AI system eventually needs only 5 units of HBM instead of 10.
HBM use per system has fallen by half.
That sounds bad for HBM demand.
But suppose AI grows so quickly that the number of systems rises from:
100 → 500
Now calculate again:
500 × 5 = 2,500
Something surprising happened.
HBM use per AI system fell 50%.
But:
Total HBM demand increased from 1,000 to 2,500.
That's the part of this debate that's easy to miss.
SK hynix and Cathie Wood Could Both Be Right
At first, the two ideas seem contradictory.
The SK hynix story says:
AI grows → HBM grows.
Wood's argument says:
HBM gets expensive → engineers find ways to use less HBM.
But both could happen.
AI systems could become more memory-efficient.
At the same time, the total number of AI systems could grow much faster.
If that happens, the HBM market could continue expanding even as individual systems become more efficient.
But if new architectures dramatically reduce HBM requirements faster than the AI market grows, the economics could change.
So simply asking:
Will AI keep growing?
may not tell us enough.
There are two numbers worth watching.
How fast is total AI usage growing?
And:
How quickly is the amount of HBM required for each unit of AI falling?
The relationship between those two numbers may matter enormously.
Expensive Things Create Competitors
There is an old pattern in technology.
When something becomes expensive and scarce, two groups appear.
One says:
Let's make more of it.
SK hynix and other memory manufacturers are doing exactly that.
The other asks:
Can we use less of it?
Groq, Cerebras and other AI chip architectures are exploring versions of that question.
And companies such as Qualcomm are asking something slightly different:
Can we change the way computing and memory work together?
Nobody yet knows which approach will dominate.
Several could grow at the same time.
But the pattern itself matters.
Scarcity creates profits.
Profits attract competition.
High costs create reasons to redesign the system.
That's how technology markets evolve.
The More Interesting Question
HBM is one of the most valuable components of today's AI infrastructure.
SK hynix has become one of the biggest beneficiaries.
And current demand remains exceptionally strong.
But the most expensive and scarce component in a technology system often becomes the component engineers work hardest to optimize.
That doesn't mean HBM disappears.
It means the question changes.
Instead of asking:
Will HBM survive?
A more useful question might be:
How much HBM will it take to run the same amount of AI five years from now?
And immediately after that:
How much bigger will AI itself become during those five years?
If AI grows much faster than memory efficiency improves, HBM could remain one of the great infrastructure stories of the AI era.
If efficiency improves faster than expected, the economics could begin shifting toward different architectures.
Either way, something interesting is happening.
The AI memory race is no longer only about who can make more HBM.
It's increasingly also about:
Who can do more AI with less of it?
BEYOND THE OBVIOUS.
Sources
Reuters — SK hynix Plans to Double Wafer Capacity Over Five Years
Reuters — Cerebras Launches New Server and Chip System Designed to Speed AI Chatbots