Comment by hombre_fatal
7 hours ago
If it can be measured, then LLMs can optimize it.
Once I had repo commands that could dump `sample` results and a cpu profiler/trace and then a benchmark tool that let me A/A + ABBA/BAAB-test the current modified git workspace against HEAD or any commit, the LLMs could just do their thing.
And that's how my homemade terminal uses much less memory than ghostty/kitty/iterm yet has more throughput.
AI is going to increasingly unmask people and companies who don't care about correct and performant software now that it's become so trivial to guarantee both. It used to at least be expensive and time-consuming and expertise-demanding to do those things.
Agree it's amazing how much low-hanging performance fruit AI can trivially find. On the other hand though, once you get through the obvious no-brainer stuff, there's a lot of non-trivial tradeoffs in performance and I think that still demands a good amount of expertise to guide the AI in the right direction. Obviously AI will continue working it's way up the value chain, but I think there's a glass ceiling for AI where the right macro tradeoffs and perspectives on how software should work will bump into the hard and often articulated reality that different stakeholders want different things and often have either magical thinking or even self-deception about how those desires can co-exist with what everyone else wants.
This isn't a new problem by any means, but now that code is cheap, it means instead of getting frustrated with engineering and their pesky unimportant details, people will get frustrated with the AI and it's pesky unimportant details.
Yeah, the biggest example is performance optimizations that sacrifice your data model to the point that you'd never accept them.
I think it's one reason why ADRs are an important of a software project, especially with LLMs. You need a place were you can document invariants, why you have them + the rejected ideas and acceptable risks.
It helps smart agents like Fable help you decide on trade-offs and it's kind of incredible to witness that happening.
I think they can be useful for quickly iterating through benchmarks and trying lots of ideas, but they won't come up with them on their own. Also, I'm not sure why, maybe some mean reversion thing, but they will never, ever suggest writing a tool to make their own life easier, get more accurate information, or anything. Once I point it at a tool, it can be ok at using it (I say ok because they seem to skim the help docs, which is truly ironic, considering I seem to read it more thoroughly even though I'm 100x slower at it. I assume this is some token saving system prompt), but they won't suggest it for you.
This is why I'm not worried about being replaced for now or the forseeable future. For all of the improvements they've made, this part just never seems to change. They could slap another heuristic prompt for the edge case, but eventually it'll revert to the mean again.
I think there is a way to use LLMs to help with programming, but not when I'm not the driver in the seat writing the tests and deciding the architecture. Also I would never ship code written by them as the final product for anything I care about. Since I, like most people, find reading code to be arduous. The more fun thing to do is to force yourself to rewrite it all, treating the LLM's work as a rough draft.
> but [LLMs] won't come up with [ideas] on their own.
They can, in fact, generate plausible performance optimization ideas on their own.
> they will never, ever suggest writing a tool to make their own life easier, get more accurate information, or anything
Make sure you process doesn't depend on anyone reading your mind.
When I run into things like this, it becomes a one-liner in my instructions/harness or in the canned prompt/skill I use that sets off a process.
In this case, I instruct agents to proactively build/improve diagnostic tooling if it would help them with their task + if it meets a bar of generalization/reusability (else it should be an ephemeral probe that gets abandoned at the end of the solution).
> If it can be measured, then LLMs can optimize it
Then they can start attempting to optimize it. They can also spin round and round making the numbers worse because they don't actually know what to do.
I had this experience at work trying to optimize a little high level Pytorch. It can't really get better than it already was, but LLMs were quite willing to pretend they will. The real solution is I need to open a PR for one of Pytorch's tracking issues.
Yeah, but that's just the scientific process of hypothesis -> evidence -> conclusion.
You need a measurement that can falsify hypotheses and reject branches that won't work.
Also, if all you have left in your project are performance issues that are hard to identify without flailing around (even with Fable/Astra) despite sampler/profiler reports, then you're doing really well and I wouldn't assume you're going to fare much better than the sota models in terms of stabs in the dark.
The point of this post is that this is explicitly not the case. If the metric is measured, the agent finds a way eventually (around 5 total tries typically unless it gets stuck), and learns from iterations where changes caused a regression after a revert.
In one case I used a made-up metric (since I didn't know the exact name or if it existed) and it somehow optimized that too.
> They can also spin round and round making the numbers worse
"Claude, if this idea doesn't measure as an improvement (use X benchmark and a T-test), discard it and try the next idea."