Comment by cyanydeez

I'm assuming the devs are seeing 10x code generation and equating that to the improvement.

It's when they practically ignore the rabbit holes where it's suspect. I'm definitely seeing speed ups. I troubleshot a linux system yesterday with minimal effort using a local llm. It likely would have taken me a few hours to locate all the docs & testing procedures. the llm did it with only a few prompts. To ensure it did it correctly, I had to interrogate it a few times before letting it proceed.

Humans make really bad scientists, and it takes a lot of effort to properly catalog and provide statistics for these things.

There is an improvement, but I doubt any random dev can give a real estimate since before LLMs they couldnt really give you a real estimate anyway. I do know when I encounter a bug now, debugging is almost immediately possible.