Cloudflare's Big Pineapple platform, which powers the 1.1.1.1 service, manages 250 billion cache entries. To optimize memory usage, the engineering team implemented five changes to the cache storage mechanism, reducing the memory footprint per entry by more than half.
The optimization freed approximately 100 TB of RAM across the company's server fleet, equivalent to the memory capacity of 130 13th-generation servers. The changes also improved performance, increasing insertion throughput by 43% and reducing lookup latency by 19%.
Engineers replaced Vec and String types with Box to eliminate unused capacity fields and unnecessary heap allocations. Additionally, they optimized data structures by using 2-byte offsets instead of 8-byte pointers for DNS sections, and packed multiple boolean fields into a single bit flag to reduce padding.
The team also optimized DNS record owners by removing the owner field when it matches the query domain, reconstructing it during read operations. Furthermore, they moved large enum variants into Box to prevent small records like A and AAAA from inheriting the size of the largest variant, such as NAPTR.
While boxing large variants initially risked memory fragmentation and reduced data locality, the team mitigated these issues by leveraging the jemalloc allocator. These combined efforts ensured that memory savings did not compromise the speed of the DNS service.
