Comment by jama211

25 days ago

Reading this article seems outdated and therefore quaint in some areas now, the “we’ve all felt that moment of staring at a small bit of simple code that can’t possibly be failing and yet it does” - I so rarely experience this anymore as I’d have an LLM take a look and they tend to find these sort of “stupid” bugs very quickly. But my earlier days were full of these issues so it’s almost nostalgic for me now. Bugs nowadays are far more insidious when they crop up.

14 comments

jama211

Reply
alecbz  25 days ago

Trying to get LLMs to understand bugs that I myself am stuck on has had an approximately 0% success rate for me.

They're energetic "interns" that can churn out a lot of stuff fast but seem to struggle a lot with critical thinking.

  • jama211  24 days ago

    I’m not going to take bitter advice from someone who either hasn’t used them in a long time, or is terribly bad at using them. Especially as it seems like you hate them so much.

    I don’t particularly like them or dislike them, they’re just tools. But saying they never work for bug fixing is just ridiculous. Feels more like you just wanted an excuse to get on your soapbox.

    • alecbz  24 days ago

      It's not that they can't fix bugs at all, but I find that if I've already attempted to debug something and hit a wall, they're rarely able to help further.

      2 replies →

  • chrisjj  25 days ago

    > seem to struggle a lot with critical thinking.

    It is an illusion arising from anthropomorphisation. They aren't thinking at all. They are just parotting the output of thinking that has long gone.

    • alecbz  24 days ago

      This feels too strong IMO.

      Just focusing on the outputs we can observe, LLMs clearly seem to be able to "think" correctly on some small problems that feel generalized from examples its been trained on (as opposed to pure regurgitation).

      Objecting to this on some kind of philosophical grounds of "being able to generalize from existing patterns isn't the same as thinking" feels like a distinction without a difference. If LLMs were better at solving complex problems I would absolutely describe what they're doing as "thinking". They just aren't, in practice.

      5 replies →