Comment by mannyv
12 hours ago
One question the article doesn't answer is: why are they cacheing at all? If your cache is that big it isn't a cache. How much bigger is the dataset in question? There are 250 billion entries. Assuming 80/20, that implies 1.25 trillion records?
What's the speed of service/response time relative to the data source?
At that point it might be enough to replace your multiple caches with fewer in-RAM databases?
It's an interesting problem.
Maybe I'm misunderstanding, but this powers 1.1.1.1, it doesn't front an internal dataset. A cache miss hits a nameserver. Which is to say, the dataset is "every DNS record in the world"
I think the question is probably more along the lines of - why not do a database with 100 TB of storage/records instead of a cache? tomato / tomato.. especially with smart caching in front of database. 100TB of flash is a good bit cheaper than 100TB of memory
Because it would be slower and have different scaling requirements than the ones they want.
It's not 100TB of data. It's probably 50 GB of data on each of 2000 servers. Because it's a cache. What is the point of a central cache if it's as slow to access as the original data?
1 reply →
I'm no expert but presumably all of throughout, latency, and churn. DNS is approximately a giant KV store where the typical record has a TTL of ~5 minutes.
This is smart, task-specific caching in front of database.
You have to cache, cloudflare doesn't know all the records ahead of time, they have to do recursive lookups to the authoritative servers that own the records and that is only good for the period of the TTL of the record. There is no "global" DNS record database or something like that.
>that is only good for the period of the TTL of the record.
Not really, TTLs are often short, but IPs might not change for years.
You can probably generate your own TTL, at scale, and avoid many DNS requests.
Why would anyone want to use a DNS resolver that tampered with records on a large scale? The TTL is intentionally set by the originator of the record.
Or alternatively, if you don't tamper why would I want to use a service that serves stale data?
3 replies →
In DNS, the owner of each record has full control over its TTL. Intermediary DNS servers are required to honor them and are not permitted to replace TTLs with their own.
11 replies →
then they would be breaking DNS at scale.
It's a recursive resolver. The global DNS dataset is not something you could collect to serve directly vs caching from observations.
The data source is authoritative name servers operated by third parties, some of which are slow on their own, some of which are behind slow or lossy networks. Origin response times vary between probably 1 ms and 2 seconds +/- origins that never respond.
> If your cache is that big it isn't a cache.
This is an incorrect statement. Caches do not have a requirement of being smaller than their source data set. CDN is an example of a cache that generally matches the size of the source data.
They’re adding the cache consumed across all of their servers. It’s not one giant deep cache.
The simple answer is that if you didn't cache, DNS traffic would skyrocket, and the load would pile up on the authoritative servers, which were intended to be small, and during the early days of the Internet, were frequently on bandwidth-constrained links.
DNS is designed to distribute query load to the edge as much as possible, and that's enabled by caching. It just so happens that "the edge" is now becoming concentrated among a small set of providers because they wanted to make a business out of it.[1] They knew that this would be expensive going in, though.
[1] Nobody has to use 8.8.8.8 or 1.1.1.1. Most people can use their ISP's cache or a local cache instead without any noticeable difference in behavior.
The problem is there is a noticable difference in behavior because the ISP cache is overloaded so queries take longer. Sure, that's not everyone's experience, but there's a reason people chose to use alternate servers.