Quantization simplifies neural networks by reducing weight precision, allowing models to run on hardware with limited memory. While training typically requires high-precision formats like FP32, engineers use mixed-precision techniques to optimize performance and reduce memory bottlenecks during the training process.
Mixed-precision training involves maintaining a master copy of weights in FP32 while performing computations in lower-precision formats like FP16, BF16, or FP8. This approach accelerates calculations and reduces data transfer volumes between memory and compute cores, though it requires strategies like loss scaling to maintain stability.
The BF16 format, developed by Google, addresses FP16 limitations by providing a dynamic range similar to FP32, which prevents gradient overflow. Meanwhile, FP8 is divided into sub-formats: E4M3, which prioritizes precision for forward passes, and E5M2, which prioritizes range for calculating gradients during backpropagation.
Hardware architecture often relies on byte alignment to optimize data access, where memory addresses are multiples of the data size. Formats that do not align with powers of two, such as 6-bit representations, can cause performance overhead because the processor must perform multiple read cycles to extract data spanning across memory blocks.
Modern hardware accelerators are increasingly designed to support specific low-precision formats natively. For instance, the AMD Instinct MI350 accelerator includes hardware-level support for FP6, reflecting the industry's shift toward specialized formats to improve computational efficiency.
