At the AI Infrastructure Summit in Santa Clara, Samsung Electronics unveiled its roadmap for next-generation memory technologies. The company aims to increase the response speed of AI accelerators by 10 times using its zHBM memory, which was announced last month. This technology vertically stacks HBM directly onto the accelerator, offering up to 8 times the performance and 3 times the power efficiency of HBM5.
Samsung executives stated that the goal is to boost processing speeds from the current 100 tokens per second per user to 1,000 tokens per second. Additionally, the company announced plans to begin sampling its 3D memory solution, zNAND-O, starting in 2028. These developments are part of Samsung's strategy to maintain its leading position in the high-bandwidth memory market.
The summit also featured discussions on Compute Express Link (CXL) technology. While some industry experts suggest CXL could reduce costs by supplementing HBM, skeptics remain. OpenAI researcher Daniel Morris noted a lack of practical applications for CXL in running AI models, while Intel architect Vidya Tiagarajan emphasized that CXL cannot replace HBM due to significantly lower data transfer speeds between GPUs.
