Cloudflare has optimized its global infrastructure, freeing up approximately 100 TB of RAM. Engineers achieved this by modifying the consistent hashing algorithm within the open-source Pingora framework, which manages web content caching across servers.
The company previously used the Ketama algorithm, assigning around 100,000 virtual points per server to balance traffic. Statistical analysis revealed that increasing points beyond 10,000 provided negligible improvements in load distribution, leading to excessive memory usage.
Engineers reduced the number of virtual points by a factor of ten. Additionally, they optimized data structures written in Rust, shrinking each hash-to-server table entry by two bytes. These changes, applied across billions of records, resulted in the total memory savings.
To ensure system stability, Cloudflare implemented the new algorithm alongside the existing version. This parallel deployment allowed for an immediate rollback if any performance issues were detected. This follows a previous optimization of the 1.1.1.1 DNS cache, which also saved 100 TB of RAM.
