← Back to context

Comment by overgard

3 months ago

I just watched copilot today turn a 8 line fix into 500 lines, so, yeah, verbosity is a big side effect

21 comments

overgard

Reply
verdverm  3 months ago

It occurs to me this pattern might be the average code we humans have produced. We all have made those quick fixes, copy-pastas, and dirty hacks... they learned it somewhere! I also assume that some of the behavior is an artifact of their training regime.

  • TheRoque  3 months ago

    So with LLM outputting average code, and people using LLM more and more, I guess the average code will become worse over time ?

    • verdverm  3 months ago

      There is a belief that everyone is just taking whatever the LLM (really agents now) outputs. This is not the case anywhere I work. We use human oversight to have it iteratively improve the code. The average quality is going up.

      2 replies →

    • nielsbot  3 months ago

      Not advocating for AI code slop--but if AI coded software works correctly, maybe it doesn't matter? Except sometimes when a specialist will have to get involved. Not a perfect analogy, but most people don't write assembly these days--they have a compiler do that. Assembly still has a place, but it's a specialist task.

      8 replies →

  • overgard  3 months ago

    In my case, where I see it most often is when the LLM has to rework something multiple times, and the feedback loop is vague (especially when all I have to give it is "no error messages, but it's still broken"). It seems like after the third or fourth try it just kinda goes off the rails. I find that the one-shot quality tends to be a little better, if the slot machine happened to work correctly that time.

    • verdverm  3 months ago

      You shouldn't be using an LLM directly (web chat style). A proper harness allows an agent to see the errors itself and correct as needed. You can the correct it at higher, more meaningful levels.

      2 replies →