If you haven’t noticed, DRAM memory has gotten a lot more expensive in recent weeks.
Amid a shortage broadly attributed to a surge in demand for AI infrastructure, Reuters reports that memory giant Samsung has hiked prices 60 percent since September, citing two persons familiar with the matter.
Among the memory types affected are DDR5 memory modules used in almost all kinds of computing devices: smartphones, laptops, desktops, workstations, and servers.
DDR5 isn’t normally associated with AI accelerators. Those typically employ GDDR or HBM, which trade capacity for the higher bandwidth required for those workloads. However, the systems into which these accelerators are deployed do use DRR5 for the system memory to feed the CPU and to store intermediary data, like the key value caches used to prolong a model’s short-term memory over extended periods.
The memory shortage has been a boon for Samsung, which trailed its rivals in the AI chip space in part due to its struggles validating its latest generation HBM3e modules for use in Nvidia GPUs like the B200 and B300.
But while the AI infrastructure boom is being blamed for the chip crunch, it may not be the only factor contributing to the shortage. Unlike many chips, memory is a commodity market.
Micron, SK Hynix, Samsung, and a slew of smaller Chinese vendors like CXMT and YMTC all design chips to a common spec defined by standards bodies like JEDEC. While some memory vendors may get to market with new technologies before others, they pretty much all make the same thing. As a result, the memory market is prone to periodic boom-bust cycles, in which prices rise and fall as inventories rise and fall.