100 terabytes of RAM came back when Cloudflare shrank one DNS cache entry by 56%

Cloudflare’s 1.1.1.1 resolver just gave back about 100 terabytes of RAM. Nobody bought hardware to do it. Five changes to the way a single DNS cache entry sits in memory cut the per-entry footprint from 953 bytes to 420, and the cache came out faster than it went in.

The arithmetic is what makes this worth reading. Big Pineapple, the Rust and WebAssembly platform behind 1.1.1.1, holds over 250 billion DNS cache entries at any given moment, so one wasted byte per entry costs more than 250 gigabytes across the fleet. Sebastiaan Neuteboom’s August 27 writeup walks through each change with before and after numbers. Four of the five have nothing to do with DNS.

What 1.1.1.1 keeps in memory

Every item in the cache is a key and a value. The key is the question that was asked: a domain name, a record type, a DNSSEC flag and a tag. The value is the response, split into answer, authority and additional record sections, plus metadata like a timestamp, a hit counter and the Time-to-Live.

That sounds small until you count the copies. When EDNS Client Subnet is in play, authoritative servers hand back different answers depending on which network the client sits on, so Cloudflare caches several versions of the same question. Data centers serving that traffic carry both more entries and fatter ones. It adds up quickly. Cache size varies per location, and once an instance hits its entry ceiling it evicts older or less popular items to make room.

The old layout used the obvious Rust types. Vec<Record> for each of the three sections, String and Vec<u8> for the variable-length bits, a RecordData enum with one variant per DNS record type. Every one of those choices is correct for code you’re still writing. Each one also carries machinery that a write-once cache entry never uses.

Five changes, 533 bytes per entry

The first change is the cheapest to copy. A Vec stores three fields: a heap pointer, a length and a capacity. Capacity exists so the vector can grow, and a cached DNS response never grows again after it’s stored. Swapping Vec<T> for Box<[T]> and String for Box<str> drops the capacity field and the slack the allocator had reserved for future pushes. Cloudflare counted 8 such fields per entry, 8 bytes each, 64 bytes per entry and over 15 terabytes fleet-wide.

Second, the three record sections collapsed into one list with two u16 offsets marking where the authority and additional sections begin. Two separate lists, each carrying an 8-byte pointer plus an 8-byte length, became two 2-byte offsets. That’s 28 bytes per entry, and removing small fields also let several booleans pack into a single bitflag, which shook out extra alignment padding.

Third, most records were storing a domain name they didn’t need. A record’s owner is usually identical to the domain that was queried, and the query is already in the cache key. So the owner field became Option<Box<Name>>, None for the common case, with the name reconstructed at read time. Records behind a CNAME still store the real owner. The DNS wire format solves the same duplication with name compression, a 2-byte back-pointer defined in RFC 1035, but following those pointers on every lookup is expensive, so the cache trades memory for speed and keeps names expanded.

Then comes the interesting one. The fourth change is the one most likely to be sitting in your own code right now, because a Rust enum is always as large as its biggest variant:

pub enum RecordData {
    A(Ipv4Addr),      //   4 bytes of payload
    Aaaa(Ipv6Addr),   //  16 bytes of payload
    Txt(Txt),
    Naptr(Naptr),     // 136 bytes: the largest variant
    Svcb(Svcb),
    // ...
}

NAPTR at 136 bytes pushed the whole enum, tag and padding included, to 144 bytes. A and AAAA records are over 80% of 1.1.1.1’s traffic and need 4 and 16 bytes. Most records were therefore burning more than 120 bytes on padding for a variant they’d never hold. Boxing the large variants shrank the enum to 24 bytes and saved 120 bytes on every A and AAAA record. The biggest variant now pays for a pointer plus allocation overhead, but NAPTR records are rare in practice, so that trade is easy.

Boxing brought its own bill, though. Every boxed variant is a separate heap allocation, and jemalloc rounds each one up to a size class. A TXT record asking for 32 bytes lands exactly in the 32-byte bin. An MX record asking for 40 rounds up to 48 and wastes 8. Worse, the boxed payloads scatter across the heap, so reading one costs a pointer chase and often a fresh cache line.

Change five ate both costs. Instead of a list of parsed enum values, record data now lives as one Box<[u8]> holding each record as a 2-byte length prefix followed by its raw wire bytes. One allocation, packed contiguously, no per-variant tag. The tradeoff is that records can no longer be indexed randomly, so features like round-robin rotation of A records have to walk the buffer. With one to four records per entry, that walk costs nothing measurable.

The payoff shows up on the read path. Building a response used to mean serializing every parsed field back into wire format. Now A, AAAA, TXT and all DNSSEC records get memcpy’d straight out of the buffer into the outgoing message. Only records containing domain names, CNAME, NS, MX and SOA, still need parsing so name compression can be applied. Cloudflare measured a 5% drop in lookup latency from that alone, and a further 13% gain in insert throughput from writing into a reusable scratchspace buffer before the single final copy.

What the rollout graph shows

Benchmarks are one number. Production is another, and Cloudflare published both.

Across production instances, p99 resident memory fell from 9.3 GB to 5.3 GB, a 43% cut. At p90 it went from 6.5 GB to 3.8 GB. Both are smaller than the 56% benchmark figure because resident memory covers the whole process, not only the cache. The sawtooth is restarts: a fresh instance boots with an empty cache and climbs as it fills, so read the plateaus, not the dips.

Cloudflare puts the fleet-wide saving at roughly 100 terabytes, or the RAM in 130 Gen 13 servers. Those machines ship with 768 GB of DDR5-6400 behind a 192-core AMD EPYC 9965, so the arithmetic lands at 99.8 TB. The measurement method is worth stealing too: a custom allocator wrapping Rust’s System allocator recorded the count and size of every allocation per cache entry, run against a synthetic workload matched to production shape (56% A records, 25% AAAA, 19% TXT).

Which of these you can steal

Strip the DNS vocabulary out and a short list of general rules is left. Most cost an afternoon.

• Immutable after write? Stop paying for growth. Any Vec or String you never push to again is carrying a dead capacity field plus reserved slack. Box<[T]> and Box<str> in Rust, shrink_to_fit plus a fixed array elsewhere.
• Your tagged union is as big as its worst member. Print size_of on every enum in a hot struct. If the common variant is 4 bytes and the rare one is 136, box the rare one.
• Derivable fields don’t need storing. If a value is already reachable from the key, store a null and rebuild it on read. Your records stop being self-contained, which only stings if the key isn’t at hand during lookup.
• Many small allocations lose to one big buffer. Contiguous bytes beat pointer chasing, and one allocation dodges the allocator’s rounding on each of the small ones.
• Profile allocations, not just struct sizes. A 40-byte struct in a 48-byte bin is really 48 bytes, and size_of will never tell you that.

One of Cloudflare’s five changes doesn’t generalize cleanly: storing records as raw wire bytes. It works here because DNS already has a compact serialization the cache can hand back almost verbatim. If your read path has to reparse a blob every time, you’ve traded memory for CPU rather than winning both.

Why this is landing now

The post hit Hacker News on August 27 and pulled 921 points and 284 comments, most of them arguing about whether any of this should have been necessary. “Using obviously better data structures the first time isn’t premature optimization,” went one line. Cloudflare’s John Graham-Cumming answered from the other side, writing that “if your goal is correctness and shipping fast and you’re not memory constrained then spending time using the least amount of memory is a waste of time specifically because you want to ship fast.”

The framing that actually travels came from a commenter going by edflsafoiewq: “A programming language’s native in-memory object format is typically optimized for random access, uniformity, and mutability. Serialization formats for network or disk tend to be designed explicitly to be more compact. But you can design your own in-memory representation too, with the properties you need.”

Cloudflare isn’t banking the 100 terabytes. It says the freed memory goes back into cache capacity, which raises hit rates and cuts the queries it pushes to upstream authoritative servers. That’s the claim to watch, because a bigger cache is testable from outside: resolve the same cold domains against 1.1.1.1 and 8.8.8.8 and compare. Until those numbers move, this is a very good memory story and an unproven latency one.