Comment by doubled112
4 hours ago
> mixed-technique approach
I think my biggest annoyance with the way we rolled out AI is that nobody seemed to want to use it to augment already working solutions.
Just throw everything out and have an LLM do it instead.
I've been frustrated with Copilot in this regard.
I work on a large C++ codebase, with large files. Human developers jump around between files with the Visual Studio fuzzy search, set breakpoints to trace execution in the Debugger, use the IDE's refactoring tools.
Microsoft's answer to this was to just ... expose none of this to their Agent Mode!? Replace the working semantic autocomplete with fucking lies!?
Maybe it's changed, I haven't been paying that much attention after bouncing off of this. I've gotten mild acceleration from using gptel-mode in emacs, manually adding references to context, and having models do various mechanical transformations on code. And I've even had some limited success writing tools for it to do LSP lookups.
It frustrates me too, it really feels like the next breakthrough will be when someone gets agents working "natively" with LSP on large code-bases.
Anthropic added LSP support to claude-code, but the current implementation is worse than useless, because any changes aren't reflected fast enough, so it's constantly working on outdated views / compilation caches, and it gets in a right muddle between its "internal" state / understanding in context, the real-world file, and the LSP.
If it could just leverage LSP to apply refactorings it would be amazing, but it feels like the LSP can't keep up, and I don't know if that's an LSP problem or a claude problem.
So we binned the LSP plugin and we're back to watching a machine find/replace, because while waiting on that is slower than LSP, it's a "Action => Wait" which the tooling understands, while LSP is "Possibly Wait for LSP to catch up => Action" which it doesn't understand nearly as well.
I suspect the LSP plugins also need better skills that pair with them so it reaches for them more often.
It hurts my soul to see it reach for find/replace to rename a class, complete with mistakes made in complex solutions where you might have name clashes in different namespaces. Something the LSP handles without problem, but can trip up an LLM.
I wonder, is the problem here that LSP is updating too slow all the time? Or just that there’s a chance it will update very slow, and you never really know if you’ll hit that chance, so your model always has to do the “long time wait” just in case? It seems like it ought to be possible for LSP to report that it is still processing, in the latter case, somehow…
1 reply →
I work in Unity and I got frustrated with Claude constantly doing gross bash/grep/awk/sed/grep nested loops that took forever that I finally described (and had Claude implement and install) a tool that could, in a single pass, gather all this info from a Unity forest of scenes at once and answer all the questions Claude ever wanted to ask about a Unity project in a single pass that takes 50ms instead of 10 30 second iterations. It still took a lot of coaching to get it to actually use this tool, but it seems like I’ve convinced it.
if it helps, I've found that using context (Claude.md etc) is way less effective for this type of pattern compared to using PreToolHook to capture "bad patterns" and either transparently rewriting them to "do the right thing" if that is possible statically, or if not then rejecting the tool use with a message that tells the agent "how" to use the intended tooling itself.
tool_call is just a fancy wrapper to a black box that executes console commands. Said commands are now the actual backbone of all agentic AI, It feels like the linux people are incredibly vindicated in the single responsibility principle
I recently saw a Claude skill that used Claude, with no tools, as a spell checker.
I wanted to hurl my laptop out to the window.
This type of laziness isn't novel.
Check out left pad or the two dozen other "utility" packages that could be done in a single line of code.
Isn't this pretty much why language models were invented?
Pasting something directly into the chat interface seems weird, but if you could somehow just see where P(token | context) falls off a cliff, that's a pretty good hint that your writing has problem.
What would be a better way to incorporate AI as a spell checker?
In comparison to non-AI traditional tools, AI has the advantage of "understanding" the text, reducing the number of "stupid" mis-corrections. And its spelling correctness is usually already impeccable, so what is there to gain by interfacing it with traditional solutions, and how can it be achieved?
AI can’t really spell check without risking changing the meaning of sentences. Spell checking was a solved problem before this.
15 replies →
AI certainly is the shiny new hammer, and it is tempting to see the world as nails.
Traditional methods might not be perfect, but they also easily fit in the memory of even low power devices. Perhaps it isn't a problem worth burning a dollar of tokens for every spelling mistake.
The fact that it produces correctly spelled words says nothing about it’s ability to find spelling mistakes or to correct them without errors like completely changing the word.
1 reply →
> What would be a better way to incorporate AI as a spell checker?
Don't do a stupid thing like that in the first place.
> In comparison to non-AI traditional tools, AI has the advantage of "understanding" the text, reducing the number of "stupid" mis-corrections.
I doubt it, but if that's true, run a normal spell checker, and then give the output to your LLM to filter.
> what is there to gain by interfacing it with traditional solutions,
About a billionfold improvement in compute efficiency, and a lower error rate.
> and how can it be achieved?
10 seconds of actual thought.
I am skeptical that AI brings any benefit to spell checking at all.
I swear that so many AI usecases I see are: "I did not have the skill or realize that you can write a program for this obvious logic".
I guess that works if you aren't a programmer or don't want to hire somebody, but then wtf would I pay for your service or product?
Way too much engineering effort to make something that might get leapfrogged by the next gen LLM.
It's a tantalizing thing, but far too treacherous to actually go for it, most of the time.
There are many domains where a hybrid of numeric and AI approaches would make sense. For example in those domains where there's already a rich practice of numeric tools such as with IC layout.
Something something bitter lesson blah blah
I think the bitter lesson is severely misapplied in the current situation: If progress from "just add more resources" is very slow, and a huge amount of money is at stake, continous work on hand-engineering can give a continuous and very valuable competitive advantage.
The labs all seem to be going for AGI through bigger LLMs, and I am reasonably sure that it's not going to happen like that.
> The labs all seem to be going for AGI through bigger LLMs
I don't know if this is still the case. Labs like anthropic and openai are spending a huge amount of their time on custom model wrappers. Something which they used to leave to their customers.
A few days ago someone on HN commented that a teammate uses Claude to search for text in files on their own computer. Buddy... There's Command-line Tools Can Be 235x Faster Than Your Hadoop Cluster and then there's Command-line Tools Can Be ∞ Faster Than Your AI.
As snark, I've been using the phrase "ask GPT about it" for things that clearly do not need an LLM to be involved. The other day, I was on a zoom call and said it, only to see the present actually doing it. I hope my unmuted laugh wasn't too distracting.
> nobody [wants to use AI] to augment already working solutions
Plenty of people do, but that only produces a blog post that will get you to the front page of HN. If you want VCs to drop $40M on your head, you need to pretend to reinvent the world.
Then, to further appease the rain gods, you need to sue the bloggers on the front page of HN who are challenging your world-changing narrative. Which will, heh, drop you on the front page of HN.
Our community is, literally, eating itself at this point. There was a time when we actually took "make something people want" literally. Now it's just part of the fiction.