The author developed NanoGEMM, a compact C library designed to optimize matrix multiplication for small arrays, specifically in the range of 16x16 to 64x64. The tool addresses performance lags in NumPy, where operations on small matrices can take 3–5 microseconds due to the overhead of libraries like OpenBLAS or Intel MKL.
The performance issues in standard libraries stem from a complex dispatch stack, including type checks, alignment flag verification, and heuristics for thread pool activation. For a 16x16 float32 matrix, which occupies only 1 KB and fits in the L1 cache, the overhead of these libraries often exceeds the actual computation time.
NanoGEMM utilizes AVX2 instructions and a microkernel with a 6x16 block layout. It uses 12 registers as accumulators, 2 for loading matrix rows, and 1–2 for broadcasting elements via mm256_set1_ps. This configuration ensures all 16 vector registers are used without spilling to the stack.
To achieve zero-copy performance, the developer implemented a native C-extension for CPython using the Buffer Protocol (Py_buffer) instead of ctypes, which had previously caused significant overhead. For larger matrices, the library employs L1/L2 cache blocking with tiles of Mc=64, Nc=128, and Kc=128.
The resulting nanogemm package is approximately 100 KB and is available via pip. While OpenBLAS outperforms it on matrices 128x128 and larger due to multithreading, NanoGEMM provides a noticeable speed increase for smaller dimensions. The project is open-source under the MIT license.
