Manticore Search has optimized its KNN search performance by changing how it reads columnar data. Previously, the system read vectors one by one into a single buffer, preventing batch processing. By implementing mmap, the system now maps files directly into memory, allowing vectors to maintain constant addresses.
This change enables the engine to group up to 256 candidates and calculate distances for them simultaneously, similar to row-based storage. Testing on the DBpedia dataset using an AMD Ryzen 9 5950X showed that KNN query performance increased by 2.57 to 3.13 times, reaching 85–92% of the speed of row-based storage.
The performance gap between columnar and row-based storage narrows as the value of k increases. At k=500, the performance difference is only about 8%, whereas previously, columnar storage was approximately three times slower than row-based storage for these operations.
Tests on a large 88 GB taxi dataset, which exceeded the 32 GB memory limit, confirmed that the optimization does not negatively impact standard search queries. Performance for non-KNN queries remained virtually unchanged, with differences ranging from 0.12% to 1.54% depending on cache state and memory availability.
