Comment by bee_rider
5 hours ago
> Reasoning about l1/l2/l3 cache hit ratios and their implications basically throwing darts at the wall, in the wrong room. If you give it a benchmark feedback loop then it might get there eventually but still massive alpha for low level systems engineers who instinctively know how this stuff works and can now automate 99% of the grind.
I suspect a lot of the training set for this sort of thing is people online speculating about cache performance incorrectly.
Speaking as somebody who is a performance geek, my knowledge came from relentless experimenting over the years (starting in the 8 bit era). Beyond the basics, I haven’t seen much on high-performance engineering online. To learn, you need to do the hard yards and I think performance tweaking becomes almost instinctive rather than something driven by a hard set of rules.
At a low enough level, every performance tweak becomes unique and bespoke.
Of course, you could find people online talking about how to write high-performance code, but beyond a few basic techniques, their advice may not work for you — nobody can write a generalist article about performance engineering that will definitely solve the problem you have right now.
Arguably, there are fewer patterns for an LLM to infer as highly optimised code tends to become more and more opaque in the search for a nanosecond here or there.
I don't disagree, but IMO, a lot of code doesn't get to the point where those very low level techniques drive performance. Like, yes, if you are doing some heavy floating point math then that's where you end up needing it. However, in a lot of code finding hot paths and often simply switching out a O(n^2) for an O(n log n) or faster.
Getting and using tools to find hotpaths is generally the most important performance tweaking skill.
I mean, sure, but it really does depend on what you're doing. If you're working on a library with collection-types and you want to make each iteration as fast as possible, then roll up your sleeves. If you're writing a compiler and you want your language's source-code to finish compiling this week, roll up the sleeves. If you're working on a game-engine and you want to draw more than everyone else, roll up the sleeves...
There are plenty of real-world reasons why you'd want to get knee deep in this stuff. I wasn't suggesting not using tools (I've literally spent the day buried in JetBrains' memory and tracing tools!), but those tools can only tell you what is happening now, not what to do to improve it.
Profiling is, of course, essential. But performance tweaking can be quite a laborious process: if you're judging things by big-O notation, then that's a different level above the real low-level tweaking (imho of course). Picking the correct data-structures is all in the 101 of performance engineering. That's in the literature. But it's all too basic and simplistic. Most performance minded engineers wouldn't need a profiling tool to know which data-structure to use.
At the smallest level there's a lot of mental theory building and experimentation as you try out different approaches, which is where the instinct and intuition starts to build. I never see any of that in discussions about performance engineering.
4 replies →
I would think it's that the kinds of places which value this kind of knowledge often have major disincentive to share it. I'm thinking of HFT firms as one example.
Around six years ago, I worked for a switch manufacturer and we had insanely optimized networking code that will never see the public light of day.