Wait, isn't Flash supposed to be slow? How are SanDisk and SK Hynix using it to feed large models???
The latest publication from IEEE presents a rather counterintuitive new idea: The memory anxiety of large models might be alleviated by the same technology found in USB drives.
That's right, it's NAND Flash.
In many people's minds, Flash's biggest characteristics are being cheap, having large capacity, and retaining data without power. But when it comes to speed, especially for powering large model inference, it doesn't seem like a proper solution.
But now, SanDisk and SK Hynix are promoting a new thing: High Bandwidth Flash, abbreviated HBF.
Its approach is straightforward: apply the advanced packaging and chip stacking methods used in HBM (High Bandwidth Memory) to NAND Flash.
In other words, it's allowing the flash memory originally responsible for storage to also participate in the high-speed data supply for AI inference.
According to SanDisk's public information, HBF is expected to stack up to 16 NAND Flash chips, with a target of up to 512GB capacity per stack and target read bandwidth of up to 1.6TB/s.
In the subsequent second and third-generation roadmaps, read bandwidth will further increase to 2TB/s and 3.2TB/s.
While this number still can't catch up with top-tier HBM, it's no longer in the state of "slow storage" associated with traditional flash memory.
Behind this technological breakthrough might lie the global storage industry's urgent need to resolve supply shortage woes.
Just a few days ago (July 11, Beijing Time), SK Hynix's CEO Koo Joo-sun publicly stated during his debut on the US Nasdaq that the global storage industry is expected to face the most severe supply shortage in history by 2027. This remark quickly ignited capital markets and industry discussions.
Against this backdrop, SK Hynix partnering with SanDisk to launch HBF is perhaps the technical answer they are submitting to alleviate storage supply tension.
Let's take a look.
Isn't Flash Slow? How Can It Be Used for AI?
The question arises: isn't Flash notoriously slow for writes? How can it suddenly be used for AI?
Jim Handy, Director of semiconductor market research firm Objective Analysis, mentioned in an IEEE Spectrum interview, many people's first reaction is also this: how does this make sense? Flash is ridiculously slow.
But the key point is, Flash is slow, mainly in writes.
NAND Flash stores data via charges in floating-gate transistors, organized in blocks and pages. This mechanism allows it to retain data without power, unlike DRAM which requires constant refreshing, thus naturally suitable for long-term storage, combining high density and low cost.
However, the trade-off is that when writing data, charges need to be pushed into or removed from the insulated gate. This process is much slower than charging/discharging a capacitor.
So from a traditional computing perspective, it's hard to discuss Flash at the same table as high-speed memory like DRAM and HBM.
But if you only look at reading, things are different.
According to the latest ONFI 6.0 flash interface standard, a single die's theoretical bandwidth can reach up to 4.8GB/s.
Looking at this number alone, it's indeed not impressive.
But a single DDR5 DIMM can reach up to 70.4GB/s, and a single HBM4E stack can reach 3.6TB/s. Compared to a single NAND flash chip, that's roughly a 750x magnitude difference.
What HBF aims to do is significantly increase total read bandwidth by packaging and vertically stacking multiple NAND Flash dies together using 3D packaging.
Hoshik Kim, Senior Vice President of Memory System Research at SK Hynix, pointed out: "By applying advanced 3D packaging and vertical stacking technologies to NAND flash, HBF can achieve bandwidth far exceeding that of standard NVMe storage."
Just as HBM stacks DRAM chips, HBF builds chips with high storage density by stacking NAND flash dies.
What they want to do is not to suddenly make a single Flash chip faster, but to stack up read channels through stacking and packaging.
HBF Isn't Aiming to Replace HBM; It Has Found Its Niche
The truly interesting thing about HBF isn't that it aims to replace HBM.
It's that it has identified a scenario very suitable for Flash: AI inference.
Training and inference have different memory requirements.
When training a large model, the system constantly needs to read weights, calculate errors, and update weights through backpropagation;
This process requires massive reads and writes, which is very unfriendly to Flash;
Because Flash writes are slow, and heavy writing involves issues like endurance and erase management.
But the inference stage is different.
At this stage, the model is already trained, weights are mostly frozen. Most of the time, the system needs to quickly read out these massive model weights, not frequently rewrite them.
This falls exactly on Flash's relatively strong side.
Kim, SK Hynix's Senior VP of Memory System Research, stated: "In an inference environment, massive, read-intensive data such as static tens-of-billion parameter model weights or precomputed KV cache can be securely stored in the HBF tier."
This way, the expensive and capacity-limited HBM can be freed up to specifically act as a high-speed scratchpad.
This architecture is somewhat like adding a "massive capacity read-only memory pool" for AI.
HBM continues to handle the most urgent, frequent high-speed computing data exchange.
HBF is responsible for holding those massive, rarely changed model parameters.
For current large model inference, this idea is very meaningful.
Because the challenges faced by current large model inference are no longer limited to computing power shortage. Multiple pressures such as insufficient memory, insufficient bandwidth, high energy consumption, and expensive deployment costs are intertwining.
Especially against the backdrop of continuously expanding model sizes, insufficient single-card VRAM capacity and the high cost of HBM are becoming significant constraints on its development.
Often, it's not that the GPU can't compute, but that the model can't fit, or data can't be fed fast enough.
And if HBF can find a balance between capacity, bandwidth, and cost, it may allow larger models to run at a lower cost.
512GB per Stack, AI Servers Might Use Fewer Cards
SanDisk's target parameters for the first-generation HBF are up to 512GB capacity and 1.6TB/s read bandwidth.
This combination is crucial – HBM's advantage is extremely fast speed, but it's expensive and has limited capacity; HBF, while slightly slower, can provide larger capacity and lower unit cost.
This means that when deploying large model inference in the future, AI servers might not necessarily need to stuff all weights into HBM.
Some colder, more static, read-heavy/write-light data can reside in HBF.
△
The potential benefits of doing this are several:
First, alleviate the HBM capacity bottleneck.
Second, reduce the need to stack more accelerator cards just to fit the model.
Third, lower the overall cost and energy consumption of the inference system.
Kim's assessment is: HBF and HBM are not competitors, but complementary tools.
He pointed out: "HBF can effectively alleviate the severe capacity bottleneck faced by HBM without sacrificing data supply speed, thereby potentially reducing the number of accelerators required to run large-scale models."
On this basis, he expects this solution will also improve energy efficiency and reduce costs, thereby helping data centers further expand their AI inference hardware scale.
This is also the most noteworthy industrial logic of HBF: It's not about turning Flash into HBM, but about letting different data in the AI system reside in positions more suitable for themselves.
Hot data stays in HBM; large-capacity, static, read-intensive data goes to HBF; lower-speed, cheaper data continues to reside in SSD or other storage.
If this hierarchy holds, the memory architecture of AI inference hardware might gain a new tier.
Large-Scale Adoption Still Far Off, but New Signals of Division of Labor in Large Model Memory Systems Emerge
Of course, HBF is not yet a mature product.
According to IEEE Spectrum, HBF will likely need at least another year before possible shipment, with large-scale deployment potentially still years away.
On February 25 this year, SanDisk and SK Hynix jointly held a launch event, announcing the promotion of HBF standardization work under the Open Compute Project (OCP).
As an important open industry organization in the data center hardware field, OCP has led the development of numerous server, rack, power supply, and accelerator specifications, such as the server specification DC-MHS and accelerator specification OAI-OAM.
But currently, the HBF standard is still in progress, with no official release date yet determined – this also means HBF won't immediately change the AI server market in the short term.
It's more like a signal emerging ahead of time: AI inference is forcing a re-division of labor in storage and memory systems.
In the past, the focus of large model hardware was on GPU, HBM, and interconnects.
But as models grow larger and inference requests increase, bottlenecks will expand from "can it compute fast enough" to "can it fit," "can it be read fast enough," and "can the electricity bill be sustained."
HBF is targeting precisely this gap.
The AI Memory War Might Not Belong to HBM Alone
Interestingly, SK Hynix itself is one of the biggest winners in HBM.
HBM is bringing it record revenues. Logically, launching a cheaper alternative technology doesn't seem to align with business intuition.
But from a system perspective, HBF doesn't necessarily take away HBM's market.
What future AI servers need is not a single type of memory, but a tiered memory system.
Let HBM be responsible for ultimate performance, HBF for large-capacity high-bandwidth reads, and SSD for lower-cost large-scale storage.
If this combination succeeds, it would actually make AI inference systems more scalable.
Simply put: Not every piece of model data deserves to live in HBM's "mansion." Some data just needs a location with enough capacity, fast enough reads, and lower cost.
That's the role HBF wants to play.
From NAND Flash in USB drives and SD cards to a high-bandwidth model weight pool in AI servers, the leap sounds large. But the problem with large models is clear: parameters are increasing, inference is becoming more expensive, and HBM supply is tightening.
And the solution might not only come from more expensive memory, but also from repackaging familiar old technology.
Reference links:
[1]https://spectrum.ieee.org/high-bandwidth-flash?itm_source=homepage&itm_medium=hero&itm_campaign=hero-2026-07-15&itm_content=hero6
This article is from the WeChat public account "QbitAI," author: Ge Wenting
