← Back to context

Comment by gregw2

1 year ago

Pardon me an old story... I never built a double entry accounting system but decades ago I did build a billing system for a internet/telcom startup that grew to a modest 8 figures revenue.

By accident and not knowing any better as a young dev, I ended up building the billing logic from day one, and for better and worse building it in two places in the system (on a consumer-facing billing webpage, and on a separate backed process that generated invoices and charged credit cards.)

It turned out to be remarkably hard to keep them in sync. We were constantly iterating trying to get traction as we burned down our capital, releasing new products and services, new ways of discounting and pricing (per use, per month, first X free, etc), features like masterpayer/subaccounts for corporate accounts, user-assignable cost centers, tax allocation to those cost centers with penny allocation, etc such that new wrinkles and corner cases would keep popping up causing the numbers on my two screens/methods not to match.

Being personally responsible for the billing, I would go over all the invoices by hand for a couple days each month to insure they matched before we charged the credit cards and mailed out printed invoices as a final check to prevent mistakes. There was always/often some new problem I'd find affecting one or a small handful of customers which I would then fix the code before we billed. I never felt good letting go and not doublechecking everything by hand.

I thought about refactoring the billing logic to occur in one place to eliminate these mismatches and my manual crosschecking, but after a lot of thought I realized I wasn't comfortable with a single codebase and liked having two codebases as it helped me catch my own errors. I then just made it easier and easier to run automate and crosschecks between the two. The billing code was a little too gnarly to be proud of, but I was very proud of the outcome in how accurate our billing was, the lack of complaints and many near misses we avoided for many years. I do feel twinges of guilt for the complexity I left my successors but I still don't really regret it.

After that experience, the motivation for double entry bookkeeping has always made a lot of sense to me. I had sort of reinvented it in my own hacky way with double logic billing code to prevent my mistakes from causing problems for my customers...

Billing, or anything else involving money, is so easy to get wrong.

The data team I ended up leading at a previous company, had an unfortunate habit of "losing" money - it wasn't real money being lost in transit elsewhere, but records of something we should charge a customer.

Or if the team wasn't losing revenue, it was double charging, etc. etc.

Took us 3 years of hard work to regain the trust of the business leaders.

No offense but this sounds like a nightmare. It also sounds like you did a fantastic job achieving accuracy despite the complexity of the system. That’s something to be proud of.

  • This is The Nightmare. Devs building systems they barely understand, complexity leaking all over the place, and someone inheriting that awful job of keeping it running without having made any bad decisions themselves.

    Software is full of these systems.