I work with another team at work that uses lots of message passing, C++ virtual methods in deeply nested class hierarchies, large DOMs and off-the-shelf C++ STL containers... on a STM32MP1, which has an in-order Cortex-A7 and a pittance of cache.
On the other hand, in my team I wrote my custom C++ STL containers that allocate inline, SAX-style processing libraries and an object pool with links that use index+generations like 2000s video games... on a STM32H5 with less than a megabyte of RAM.
Mechanical sympathy feels like a lost art to people I demonstrate it to... and I had to demonstrate to the first team with benchmark numbers that I would not be the bottleneck between us by orders of magnitudes.
I’ve had code review pushback against reorganizing data structures to be more cache-friendly on the grounds that it was “premature optimization” even though I pre-emotively justified the decision with benchmark results indicating that the performance difference was large and business relevant, and addressed a real performance bottleneck that I observed during testing.
This was on a team and at a time where Clean Code was very popular. I somewhat wonder if the problem is that, in the 2010s, many people became convinced that certain aesthetic standards were both objectively, provably correct, and more important than anything else.
A lot of my poor opinion on developer opinions dates back to a couple projects where Product and Marketing were begging us for substantial performance improvements and the teams pushed back saying this was as fast as you could expect. I got 2x in the first six months on both projects, and delivered 25% per quarter on one for almost 2 years (that maths to around 8x) before I started to scrape the bottom of my barrel.
It’s predominantly a way to shut down thinking, which is why it’s so often misquoted. Knuth’s version is more nuanced. Theirs is not at all.
As a compromise I got really good at doing optimizations that also improved readability. Things like hoisting and better variable naming combined together can be mistaken for just code cleanup.
I also discovered that keyhole optimizations are more palatable. Which is to say, move the slow bit yo a separate function with a single responsibility. Then do the optimization in a separate commit. Then the other devs don’t have to look at your changes and if they turn out to have a bug, we can always revert the change later. They almost never do, but having the option seems to grease the wheels.
That’s just pattern matching. Most people don’t do this level of prep so they assume you didn’t either. There’s usually no ill intent, and people are maybe a little embarrassed when you point it out, but if you are tactful it’s fine.
There's not enough context for me (or other commenters) to really evaluate this, but every change has risk, and some projects aren't in a position to be accepting that risk.
A "real performance bottleneck that I observed during testing" is a little bit unclear whether it occurred outside of test conditions, or whether speeding up that particular bit improves anything anyone cares about.
I recently spent three weeks paring down a piece of software until it could run entirely in the L1 cache of the processor it was running on. The performance was - to put it mildly - impressive. And this was 1970's technology running on a modern CPU. This stuff can do miracles if you use it right.
I work in the games industry with exceptionally skilled system programmers that can do that too. They're getting on in age, and there's fewer programmers in the younger generations that are interested in any of it. Technology is sadly increasingly homogenising around Unreal Engine 5
> Technology is sadly increasingly homogenising around Unreal Engine 5
I kinda get the commodification of games engines and Unreal. I am sure a lot of us saw the gaming revolution unfold, watching gaming companies one-up each other in graphics and interactivity. But once you get to the point of UE5, Unity, or Godot there is little need or value to make a game engine from scratch anymore.
Don't get me wrong, if you want to write an engine, go for it. But if you just want to make a game then an existing engine is good enough.
Not unique to gamedev; during the boom at the turn of the millennium the bar to entry to programming was dropped on the floor to fill demand.
The vast majority of programmers today don't even understand the basics of how-computer-work - not just microarchitecural considerations but even how to interface with the kernel efficiently / understand the performance considerations of their target platform.
Ask your average programmer how to say, simply perform I/O efficiently as possible on NT given some workload characteristics. One needs to first understand this before they can begin making concessions in favour of other demands (abstraction, flexibility, ect) and very few programmers can even get beyond the starting line.
If it has a pittance of cache, then mechanical sympathy might not mean cache-friendliness. Linked lists were mechanically sympathetic on uncached systems - it cost the same to access any byte of memory, so you might as well go directly to the one you wanted.
Looking at the specs: 32 KiB of L1, 256 KiB of L2, in-order and partial dual-issue.
That's the least forgiving kind of chip when it comes to performance: fast enough to outpace main memory by two orders of magnitude, yet the slightest speed bump (branch misprediction, cache miss) stalls it without any ability to reorder around it.
I didn't get the sense that this was the end-all, just a relatively simple way to show how much impact cache-friendly data structures can have for somewhat little effort over the naive solution and give readers a sense for how to think through the problem.
I work with another team at work that uses lots of message passing, C++ virtual methods in deeply nested class hierarchies, large DOMs and off-the-shelf C++ STL containers... on a STM32MP1, which has an in-order Cortex-A7 and a pittance of cache.
On the other hand, in my team I wrote my custom C++ STL containers that allocate inline, SAX-style processing libraries and an object pool with links that use index+generations like 2000s video games... on a STM32H5 with less than a megabyte of RAM.
Mechanical sympathy feels like a lost art to people I demonstrate it to... and I had to demonstrate to the first team with benchmark numbers that I would not be the bottleneck between us by orders of magnitudes.
I’ve had code review pushback against reorganizing data structures to be more cache-friendly on the grounds that it was “premature optimization” even though I pre-emotively justified the decision with benchmark results indicating that the performance difference was large and business relevant, and addressed a real performance bottleneck that I observed during testing.
This was on a team and at a time where Clean Code was very popular. I somewhat wonder if the problem is that, in the 2010s, many people became convinced that certain aesthetic standards were both objectively, provably correct, and more important than anything else.
A lot of my poor opinion on developer opinions dates back to a couple projects where Product and Marketing were begging us for substantial performance improvements and the teams pushed back saying this was as fast as you could expect. I got 2x in the first six months on both projects, and delivered 25% per quarter on one for almost 2 years (that maths to around 8x) before I started to scrape the bottom of my barrel.
It’s predominantly a way to shut down thinking, which is why it’s so often misquoted. Knuth’s version is more nuanced. Theirs is not at all.
As a compromise I got really good at doing optimizations that also improved readability. Things like hoisting and better variable naming combined together can be mistaken for just code cleanup.
I also discovered that keyhole optimizations are more palatable. Which is to say, move the slow bit yo a separate function with a single responsibility. Then do the optimization in a separate commit. Then the other devs don’t have to look at your changes and if they turn out to have a bug, we can always revert the change later. They almost never do, but having the option seems to grease the wheels.
That’s just pattern matching. Most people don’t do this level of prep so they assume you didn’t either. There’s usually no ill intent, and people are maybe a little embarrassed when you point it out, but if you are tactful it’s fine.
> objectively
I see what you did there
Processors I mostly write firmware for have little cache at all.
I think cache friendly code also tends to be reason about friendly and debugger friendly. None of that is remotely premature optimization.
3 replies →
Before AI, nobody has been responsible for more psychic damage against software developers than Bob Martin.
6 replies →
There's not enough context for me (or other commenters) to really evaluate this, but every change has risk, and some projects aren't in a position to be accepting that risk.
A "real performance bottleneck that I observed during testing" is a little bit unclear whether it occurred outside of test conditions, or whether speeding up that particular bit improves anything anyone cares about.
1 reply →
I recently spent three weeks paring down a piece of software until it could run entirely in the L1 cache of the processor it was running on. The performance was - to put it mildly - impressive. And this was 1970's technology running on a modern CPU. This stuff can do miracles if you use it right.
I work in the games industry with exceptionally skilled system programmers that can do that too. They're getting on in age, and there's fewer programmers in the younger generations that are interested in any of it. Technology is sadly increasingly homogenising around Unreal Engine 5
> Technology is sadly increasingly homogenising around Unreal Engine 5
I kinda get the commodification of games engines and Unreal. I am sure a lot of us saw the gaming revolution unfold, watching gaming companies one-up each other in graphics and interactivity. But once you get to the point of UE5, Unity, or Godot there is little need or value to make a game engine from scratch anymore.
Don't get me wrong, if you want to write an engine, go for it. But if you just want to make a game then an existing engine is good enough.
9 replies →
Not unique to gamedev; during the boom at the turn of the millennium the bar to entry to programming was dropped on the floor to fill demand.
The vast majority of programmers today don't even understand the basics of how-computer-work - not just microarchitecural considerations but even how to interface with the kernel efficiently / understand the performance considerations of their target platform.
Ask your average programmer how to say, simply perform I/O efficiently as possible on NT given some workload characteristics. One needs to first understand this before they can begin making concessions in favour of other demands (abstraction, flexibility, ect) and very few programmers can even get beyond the starting line.
If it has a pittance of cache, then mechanical sympathy might not mean cache-friendliness. Linked lists were mechanically sympathetic on uncached systems - it cost the same to access any byte of memory, so you might as well go directly to the one you wanted.
Looking at the specs: 32 KiB of L1, 256 KiB of L2, in-order and partial dual-issue.
That's the least forgiving kind of chip when it comes to performance: fast enough to outpace main memory by two orders of magnitude, yet the slightest speed bump (branch misprediction, cache miss) stalls it without any ability to reorder around it.
What is a cache friendly version of linked lists?
5 replies →
https://people.freebsd.org/~lstewart/articles/cpumemory.pdf
Has anyone rewritten this for 2026?
I feel like the solution presented is just a clustered hashmap or “locality sensitive” if you’d like it.
Still not orders of magnitude better.
Uh, isn't this just radix-partitioned aggregation? Hierarchical (fixed two level) aggregation is old. What's novel here?
Microcosm of Pivot itself I guess. I just don't get why this project exists when DuckDB and Clickhouse both exist.
I didn't get the sense that this was the end-all, just a relatively simple way to show how much impact cache-friendly data structures can have for somewhat little effort over the naive solution and give readers a sense for how to think through the problem.
From the article: "To do this, we'll use a method called radix partitioning."
I don't think they were claiming it was novel. Seemed just like an article explaining a performance optimization
[flagged]
Shout out to the author, it seems coherent.
I like flamegraphs
I know folks use perf to help generate them
But in the future, now that Llm is here, get and agent to check assembly, PMUs and also see what you’re doing in assembly.
My 2c and happy coding to everyone :)