Comment by cedws

6 days ago

>if software developers screw up

Well, yes, they did screw up, but the fallout was amplified 100x by bad management.

"The Horizon IT system contained "hundreds" of bugs[0]."

If your accounting software has hundreds of bugs then you are really in the deep shit.

[0]https://en.wikipedia.org/wiki/British_Post_Office_scandal#:~...

  • Every system has bugs, even deployed, high visibility accounting systems. Debian stable, which I personally view as the gold standard for a robust general purpose OS, has hundreds of bugs.

    That is not to say that bugs are good. They are bad and should be squashed. But the Horizon failure, IMO, is with the management, that pretended that the system was bug free and, faced with the evidence to the contrary, put the blame on postmasters. My 2c.

  • If any large system wasn’t constantly logging errors I’d immediately assume there was something wrong with the error logging system. Only trivial software is bug free.

  • I'd be shocked if any piece of software large enough to qualify as an "accounting system" didn't contain at least hundreds of bugs. We're just not that good at building software. Especially if you consider that the system encompasses all of the dependencies, so you should count bugs in the OS, CPU, any relevant firmware, etc.

  • So long as the jury understands this, it's all fine.

    If you're on trial for doing X and your jury is told by a prosecution witness "mrkramer did X" and under cross they admit that's based on computer records which are often bogus, inconsistent, total nonsense, it doesn't take the world's best defence lawyer to secure an "innocent" verdict. That's not a fun experience, but it probably won't drive you to suicide.

    One of the many interlocking failures here is that the Post Office, historically a government function, was allowed to prosecute people.

    Suppose I work not for the Post Office (by this point a private company which is just owned in full by the government) but for say, an Asda, next door. I'm the most senior member of staff on weekends, so I have keys, I accept deliveries, all that stuff. Asda's crap computer system says I accepted £25000 of Amazon Gift Cards which it says came on a truck from the depot on Saturday. I never saw them, I deny it, there are no Gift Cards in stock at our store.

    Asda can't prosecute me. They could try to sue, but more likely they'd call the police. If the police think I stole these Amazon cards, they give the file to a Crown Prosecutor, who works for the government to prosecute criminals. They don't work for Asda and they're looking at a bunch of "tests" which decide whether it makes sense to prosecute people.

    https://www.cps.gov.uk/about-cps/how-we-make-our-decisions

    But because the Sub-postmasters worked under contract to the Post Office, it could and did in many cases just prosecute them, it was empowered to do that. That's an obvious mistake, in many of these cases if you show a copper, let alone a CPS lawyer your laughable "case" that although this buggy garbage is often wrong you think there's signs of theft, they'll tell you that you can't imprison people on this basis, piss off.

    A worse failure is that Post Office people were allowed to lie to a court about how reliable this information was, and indeed they repeatedly lied in later cases where it's directly about the earlier lying. That's the point where it undoubtedly goes from "Why were supposedly incompetent morons given this important job?" where maybe they're morons or maybe they're liars, to "Lying to a court is wrong, send them to jail".

    • > Asda can't prosecute me.

      They can, actually. Anyone in the UK can launch a private prosecution. It's rare because it's expensive and the CPS can (and often do) take over any private prosecution then drop it.

      Nevertheless, the power exists and has been intentionally protected by parliament. I think most would agree it needs reform, however.

      2 replies →

    • > If you're on trial for doing X and your jury is told by a prosecution witness "mrkramer did X" and under cross they admit that's based on computer records which are often bogus, inconsistent, total nonsense, it doesn't take the world's best defence lawyer to secure an "innocent" verdict. That's not a fun experience, but it probably won't drive you to suicide.

      I imagine digital records are involved in nearly every trial at this point. Good luck getting this point admitted by the justice system.

      1 reply →

  • Well not really, no one should be committing suicide due to a buggy system. If you know the details of the case it was widespread but the post office decided to gaslight everyone and put people in debt and prison. That’s what caused this, the bugs were just a catalyst for shitty humans to do shitty things

    • Yea management failed but wouldn't the most logical thing be to call in computer forensics experts and quality test the software, reverse engineering it and try to catch the bugs. This wasn't the classic case of financial fraud, this was all about faulty software.

      11 replies →

  • But it was the decision to gaslight and charge the postmasters with crimes that caused the suicides, not the bugs in the code. If they had just admitted that the accounting issues were due to bugs in the system then I really doubt anyone would have committed suicide.

  • [flagged]

    • Where are you getting the idea that anyone suggested this?

      At any rate, it was the persecuted postal workers who committed suicide, not the software developers.

    • I meant top management is in deep shit if their finical departments run low quality buggy accounting software not the staff. Or in this case post office branches run the buggy software. All in all, decentralized nature of post office system was the thing that drove everything to this madness.

Indeed. This is not about Horizon's bugs. It is about management that was incurious and perhaps politically and financially motivated to ignore Horizon's shortcomings, enough so to knowingly destroy lives. Charges of murder should be laid.

But we hold engineers to much higher ethical standards than management. One does not expect management to blow the whistle - or even understand whats what when dealing with complex issues in distributed systems. If the engineers start lying - its game over.

I cried when I was reading the book. So much suffering. Bought a copy for all the it architects in my company and asked all of them to read it. Should be part of curriculum for aspiring software engineers.