← Back to context

Comment by dvfjsdhgfv

8 days ago

> If you aren't doing this level of work by now, you will be automated soon.

It's harder and harder to detect sarcasm these days but in case you're being serious, I've tested a similar setup and I noticed Claude produces perfectly plausible code that has very subtle bugs that get harder and harder to notice. In the end, the initial speedup was gone and I decided to rewrite everything by hand. I'm working on a product where we need to understand the code base very well.

8 comments

dvfjsdhgfv

Reply
rckclmbr  8 days ago

I keep hearing “Claude creates subtle bugs”, but how is that different than people engineers? I’ve never worked in a bug free codebase

  • jplusequalt  8 days ago

    When you write the code yourself you are slowly building up a mental model of how said thing should work. If you end up introducing a subtle bug during that process, at least you already have a good understanding of the code, so it shouldn't be much of an issue to work backwards to find out what assumptions turned out to be incorrect.

    But now with Claude, the mental model of how your code works is not in your head, but resides behind a chain of reasoning from Claude Code that you are not privy too. When something breaks, you either have to spend much longer trying to piece together what your agent has made, or to continue throwing Claude at and hope it doesn't spiral into more subtle bugs.

  • eeperson  8 days ago

    Everybody produces bugs, but Claude is good a producing code that looks like it solves the problem but doesn't. Developers worth working with, grow out of this in a new project. Claude doesn't.

    An example I have of this is when I asked Claude to copy a some functionality from a front-end application to a back-end application. It got all of the function signatures right but then hallucinated the contents of the functions. Part of this functionality included a look up map for some values. The new version had entirely hallucinated keys and values, but the values sounded correct if you didn't compare with the original. A human would have literally copied the original lookup map.

    • tehjoker  8 days ago

      I asked claude to help me figure out some statistical calculation in Apple Numbers. It helpfully provided the results of the calculation. I ignored it and implemented it in the spreadsheet and got completely different (correct) results. Claude did help me figure out how to do it correctly though!

    • NewsaHackO  8 days ago

      > Developers worth working with, grow out of this in a new project. Claude doesn't.

      There is no way this is true. People make fewer bugs with time and guidance, but no human makes zero bugs. Also, bugs are not planned; it's always easy to in hindsight say "A human would have literally copied the original lookup map," but every bug has some sort of mistake that is made that is off the status quo. That's why it's a bug.

      2 replies →