Comment by noduerme
3 years ago
Things that either work or don't, don't really need tests. Things that have a ton of edge cases... you might spend ten times as long writing tests as writing the code. In a few cases it might be worth it. Like, I have a piece of hotel software that lets rates be set by the night plus peak extra rates over overlapping time periods, and has to delineate the nightly totals for a customer stay that crosses over lots of those. Lots of edge cases. I had that in production for 5 years before someone noticed that although the totals were right, if the very last night was at the default rate after coming off a peak rate, the line item wasn't printed on the bill. It was a simple case of greedily including one more segment in what was printed. Now... would a unit test have found that? You have to know exactly what you'd be looking for first. The test would have to be smarter than the person writing the code.
No, the test would not help to find it, but now you can write a test for that bug. Later, when you change something, related to this code or something unrelated, you can be sure it did not affect this part.
Tests are useful when you change code that is unrelated, but still somehow connected. You would never notice the issue as you don't manually try the unrelated code, but the test would catch it.
If you had unit/integration tests for all potential edgecases you came up with, you don't have to manually test that over and over.
yeah but because the code is segmented into very discrete functions... and it's quite easy to see their call chains...
Well, I get it. You're right in principle. The truth is, though, I have had to rewrite software from scratch in new languages every 10 years, and these sorts of bugs only appear a few times per decade. Once a piece of central business logic works, it usually works forever; a change to that central logic will of course require changes and tests across the whole system, from user inputs to annual reports, but that can't be helped. I suspect you'd then have to rewrite the unit tests, too.
It's probably true that when I die, most of my software will slowly wind down and eventually be abandoned. But also that's probably true even if I were to write a lot of tests.
> Now... would a unit test have found that? You have to know exactly what you'd be looking for first. The test would have to be smarter than the person writing the code.
This is a textbook example of something that property based testing can catch. Yes, the tests are smarter than the person that wrote the code. At least, smarter in a different way or maybe it’s that the people who wrote the property based testing framework are super smart.
Look up QuickCheck for the OG or property based testing + your language.
I don’t exactly understand your bug, but the way it would work is something like you’d make some generators that create night stays and rates and put those together into a bill. Then you test the property that the number of line items on the bill is the number of nights + number of discounts (or whatever is right). It will search to break this property and if it breaks it, find a minimal case. Seriously check it out, it’s perfect for this sort of thing.