Comment by OtherShrezzing
3 days ago
While I think this is good advice in general, I don’t think your statement that “there is no process to create scalable software” holds true.
The uk gov development service reliably implements huge systems over and over again, and those systems go out to tens of millions from day 1. As a rule of thumb, the parts of the uk govt digital suite that suck are the parts the development service haven’t been assigned to yet.
The Swift banking org launches reliable features to hundreds of millions of users.
There’s honestly loads of instances of organisations reliably implementing robust and scalable software without starting with tens of users.
The uk government development service as you call it is not a service. It’s more of a declaration of process that is adhered to across diverse departments and organisations that make up the government. It’s usually small teams that are responsible for exploring what a service is or needs and then implementing it. They are able to deliver decent services because they start small, design and user test iteratively and only when there is a really good understanding of what’s being delivered do they scale out. The technology is the easy bit.
The UK Gov has many service and process docs [1]. It started out that way but has grown rapidly and changed. Including a library for authentication, frontend templates and libraries, custom docker images.
[1]: https://github.com/alphagov
UK GDS is great, but the point there is that they're a crack team of project managers.
People complain about junior developers who pass a hiring screen and then can't write a single line of code. The equivalent exists for both project management and management in general, except it's much harder to spot in advance. Plus there's simply a lot of bad doctrine and "vibes management" going on.
("Vibes management": you give a prompt to your employees vaguely describing a desired outcome and then keep trying to correct it into what you actually wanted)
> and those systems go out to tens of millions from day 1
I like GDS (I even interviewed with them once and saw their dev process etc) but this isn't a great example. Technically GDS services have millions of users across decades, but people e.g. aren't constantly applying for new passports every day.
A much better example I think is Facebook's rollout of Messenger, which scaled to billions of actual users on day 1 with no issues. They did it by shipping the code early in the Facebook app, and getting it to send test messages to other apps until the infra held, and then they released Messenger after that. Great test strategy.
GDS's budget is about £90 million a year or something. There are many contracts that are still spent on digital, for example PA consulting for £60 million (over a few years) which do a lot of the gov.uk home-office stuff, and their fresh grads they hire cost more to the government than GDS's most senior staff...
SWIFT? Hold my beer. SWIFT did not launch anything substantial since its startup days in early 70-ies.
Moreover, their core tech did not evolve that far from that era, and the 70-ies tech bros are still there through their progeniture.
Here's an anecdote: The first messaging system built by SWIFT was text-based, somewhat similar to ASN.1.
The next one used XML, as it was the fad of the day. Unfortunately, neither SWIFT nor the banks could handle 2-3 orders of magnitude increase in payload size in their ancient systems. Yes, as engineers, you would think compressing XML would solve the problem and you would by right. Moreover, XML Infoset already existed, and it defined compression as a function of the XML Schema, so it was somewhat more deterministic even though not more efficient than LZMA.
But the suits decided differently. At one of the SIBOS conferences they abbreviate XML tags, and did it literally on paper and without thinking about back-and-forth translation, dupes, etc.
And this is how we landed with ISO20022 abberviations that we all know and love: Ccy for Currency, Pmt for Payment, Dt for Date, etc.
Harder to audit when every payload needs to be decompressed to be inspected
Is it? No auditor will read binary, so you already need a preprocessing step to get it to a readable format. And if you're already preprocessing then adding a decompression step is like 2 lines tops.