Comment by smithkl42
1 day ago
This is roughly my experience with LLMs. I've had a lot of friends that have had good experience vibe coding very small new apps. And occasionally I've had AI speed things up for me when adding a specific feature to our main app. But at roughly 2 million lines of code, and with 10 years of accumulated tribal knowledge, LLMs really seem to struggle with our current codebase.
The last task I tried to get an LLM to do was a fairly straightforward refactor of some of our C# web controllers - just adding a CancellationToken to the controller method signature whenever the underlying services could accept one. It struggled so badly with that task that I eventually gave up and just did it by hand.
The widely cited study that shows LLMs slow things down by 20% or so very much coheres with my experience, which is generally: fight with the LLM, give up, do it by hand.
My experience is that sometimes they give you a 10x speedup but then you hit a wall and take 30 times longer to do a simple thing and a lot of people just keep hammering because of the first feeling. Outside of boilerplate, I haven't seen it be this magical tool people keep claiming it is.
That's the definition of an advanced scaffolding tool. And yes, I subscribe to that. From time to time I use Gemini CLI for little tools I have no time to read all the docs of thinkgs I'm not used to, but in the end I need to make flow changes and be forced to understand the generated code. x10 faster bootstrap, x30 slower manual changes, 100% my codebase problem.