Comment by calvinmorrison
8 hours ago
cache all of them? no... but you could cache some of them, likely the newer are accessed more frequently.
8 hours ago
cache all of them? no... but you could cache some of them, likely the newer are accessed more frequently.
Per the article, the vast majority of traffic is scrapers that are generating effectively random, permuted URLs. Even if they're already caching their popular-with-real-humans most recent data, that efficiency gain is a tiny drop in a very large bucket.