Comment by jrockway

8 days ago

Yup. Working on a 10 year old codebase, I always wondered whether a test failing was "a long-standing bug was accidentally fixed" or "this behavior was added on purpose and customers rely on it". It can be about 50/50 but you're always surprised.

Change detector tests add to the noise here. No, this wasn't a feature customers care about, some AI added a test to make sure foo.go line 42 contained less than 80 characters.

7 comments

jrockway

Reply
baq  8 days ago

I like calling out behavioral vs normative tests. The difference is optics, mostly, but the mere fact that somebody took the time to add a line of comment to ten or hundred lines of mostly boilerplate tests is usually more than enough to raise an eyebrow and I honestly don’t need more than just a pinch of surprise to make the developer pause.

groestl  8 days ago

> a long-standing bug was accidentally fixed

In some cases (e.g. in our case) long standing bugs become part of the API that customers rely on.

  • strbean  8 days ago

    It's nearly guaranteed, even if it is just because customers had to work around the bug in such a way that their flow now breaks when the bug is gone.

    Obligatory: https://xkcd.com/1172/

    • thaumasiotes  7 days ago

      That comic doesn't show someone working around a bug in such a way that their flow breaks when the bug is gone. It shows them incorporating a bug into their workflow for purposes of causing the bug to occur.

      It isn't actually possible for fixing a bug to break a workaround. The point of a workaround is that you're not doing the thing that's broken; when that thing is fixed, your flow won't be affected, because you weren't doing it anyway.

      2 replies →