Comment by rtpg
12 hours ago
I think it's an interesting thought experiment: could an LLM call be a "cheaper" grep?
Especially for deterministic activities it just feels impossible to imagine general LLM tech handling the problem better, despite everything being said.
But hey, tech is filled with "smashing the generalist hammer works better than the specialized tooling". Would be odd though!
One thing that's easy to miss about performance is that it depends on framing.
An example I like to give: optimizing a data processing program's runtime by 5x is obviously 5x speedup for everyone. But if, for some reason[0], this means it crashes and restarts more often, it stops looking like this to end users. If every restart means it needs to start from scratch, and it restarts 10 times on average now where it didn't restart before, the 5x speedup suddenly looks like 2x slowdown to end user.
In this sense, LLMs are already much more efficient than most CLI tools, by a combination of:
- User not having to remember the exact invocation, or even the name of the CLI tools
- LLM being able to run the CLI tools and chain them on its own
- LLM being able to self-correct in case it got things wrong, or when actual output show that user's idea was right, but the instructions were wrong
Prompting "okay, list those processes sorted by runtime and match them against these output files" is both faster to type than the actual commands, it also end-to-end completes much faster than doing it by hand.
--
[0] - And I don't mean a bug. Say it's some batch processing run on a cluster with aggressive resource usage management; 5x speedup means it runs much hotter than before, which may put it on the top of "kill list" for when the cluster managing code needs to free up some resources.