SanDisk and SK Hynix standardize High Bandwidth Flash (HBF) memory
SK Hynix and SanDisk plan to fix AI’s “memory wall” with High Bandwidth Flash (HBF) memory
SK Hynix and SanDisk have released their first technical specification for High Bandwidth Flash (HBF), a new memory type that aims to overcome the capacity limitations of High Bandwidth Memory (HBM). Using vertically stacked NAND flash, this memory type aims to deliver 8-16x more per-stack memory capacity than HBM while still providing strong memory bandwidth.
With HBF, AI accelerators and other processors can feature more memory. This will allow these accelerators to handle larger AI models without relying on off-device memory. With up to 512GB of memory in a 16-die stack of HBF, future AI accelerators could feature terabytes of available memory. While High Bandwidth flash has its benefits, it is designed to be used alongside HBM memory. Both memory types can coexist, with new HBM versions providing lower latencies while HBF provides larger capacities.
(HBF can work alongside HBM to deliver high bandwidth and huge capacities)
More capacity at a lower cost
SanDisk believes that HBF memory can deliver “8-16x the capacity of HBM at a similar cost”. This could deliver tremendous benefits for future AI accelerators and other components. In simulations, SanDisk HBF memory performed within a 2.2% delta with a “hypothetically unlimited-capacity HBM” on a Llama 3.1 405B parameter model for reading pretrained weights.
Google and Tenstorrent are onboard
Google and Tenstorrent have joined the HBF consortium following its formation, signalling their interest in this new memory type. Given its capacity advantages, HBF memory could become a major part of future AI accelerators. If it reduces the memory requirements of AI servers, it would help alleviate the DRAM capacity crisis. That said, it could be a while before we see HBF memory in shipping products.
Early HBF memory samples are due to be delivered to potential customers this year. The first AI inferencing devices using this technology are due to be sampled in early 2027.
You can join the discussion on Sandisk and SK Hynix standardising high-bandwidth flash on the OC3D Forums.
