Qualcomm has unveiled the next generation of its Hexagon NPU, designed for on-device AI tasks. The update includes a 50% increase in local memory capacity and the integration of a new Element Accelerator, specifically optimized for transformer models. This component works alongside existing processing units to improve efficiency.
The memory upgrade aims to reduce data transfer between the NPU and external LPDDR memory, which typically increases latency and power consumption. By keeping more data near the compute blocks, the chip handles long contexts and multiple AI agents more effectively. For INT4 models, Qualcomm reports up to 50% faster performance during the initial input processing phase.
The new architecture supports Mixture-of-Experts models, allowing for the activation of only a subset of parameters per step. For instance, a 30-billion parameter model can operate using only about 3 billion parameters per step. The NPU supports INT2, INT4, INT8, FP8, and FP16 formats, providing flexibility between memory usage and computational precision.
Specific technical details, such as absolute memory capacity, peak performance metrics, and power consumption figures, remain undisclosed. The reported 50% performance gain is based on internal company testing for INT4 models. Full specifications are expected at the upcoming Snapdragon Summit in Hawaii, scheduled for September 22–24.
