Comment by calvinmorrison
10 hours ago
I love to see the 'leet kernel hackers and maintainers' struggling with basic volume. Each page load should cost you near nil. Us lowly PHP developers have been caching shit for close to twenty years.
Learn how to cache your application and your cpu usage should be almost zero. In fact basically any read should cost nothing in comparison to writes.
The amount of memory required to cache all possible diffs (defined as an ordered pair of commits) would likely be in the exobytes. At current ram prices that's easily a trillion dollars of ram to run that cache lol. Git focuses on making diff calculations efficient largely because the space of possible diffs is very expensive to enumerate.
The following from Claude: """ A diff between two randomly chosen commits usually spans years of history, so it's not a few KB — the tree itself is ~1.5 GB of text, and a multi-year span rewrites a large slice of it. Call it 100–200 MB per pair on average: 8.5×10¹¹ pairs × ~2×10⁸ bytes ≈ 10²⁰ bytes, or ~150 exabytes """
I'm not sure if the website allows for diffs against arbitrary tree states, but if it does than the diff space is completely unbounded, and the ram demand is theoretically infinite.
cache all of them? no... but you could cache some of them, likely the newer are accessed more frequently.
Per the article, the vast majority of traffic is scrapers that are generating effectively random, permuted URLs. Even if they're already caching their popular-with-real-humans most recent data, that efficiency gain is a tiny drop in a very large bucket.
It's running a diff between two arbitrary blobs of text. Do you actually have a solution or are you just saying to remove the feature from the site entirely?
> Us PHP developers
I can tell.
it's not an insult. yes. cache heavily. shitty php stacks serving trillions of dollars of ecommerce sales have managed to do it for a long time.
Can you explain how a cache lets you avoid serving a request for the first time?
Those ecommerce stacks serve a large fixed number of pages. cgit does not.
Imagine if WooCommerce had a route "/product/<sku1>/compare/<sku2>" which displayed an auto-generated comparison between any two product pages.
Now imagine running a million-SKU WooCommerce site, where each product page was 100kb of text.
Now imagine scrapers are permuting those URLs.
How would you cache that?
That's what cgit/kernel.org and many other Git forges are dealing with. These aren't static websites, even if the underlying Git repo is largely static; they're rendering arbitrary diffs and other generated-on-the-fly views into Git history. The ability to do that is a large part of the value of a Git UI.