Researchers at MIPT have developed the HARP algorithm, designed to improve the quality of highly compressed large language models (LLMs). The method utilizes structured orthogonal rotations to optimize weight values before quantization, allowing models to maintain higher accuracy during aggressive compression.
The algorithm functions as an add-on to existing quantization pipelines rather than a replacement. During tests on Llama 2-70B and Qwen3-8B models, HARP reduced errors by 12–15% under 2-bit compression. The method also demonstrated versatility by improving results when applied to the QTIP compression backend.
Models processed with HARP operate approximately 2.1 times faster than their full-sized counterparts while retaining 90% of the speed associated with standard compression techniques. The process requires additional calibration but eliminates the need for full model retraining.
The source code for HARP is available on GitHub, and pre-compressed models have been published on Hugging Face. Researchers intend to explore applying the algorithm to other inference components in future studies.
