> Haskell gives you tools to encode these incantations in types so they cannot be forgotten. This is, for my money, the single most valuable thing the language offers a production engineering organization.
Haskell is admittedly, probably the most powerful widely (or even somewhat widely) used language for doing this, but this general pattern works really well in Rust and TypeScript too and is one of my very favorite tools for writing better code.
I also really like doing things like User -> LoggedInUser -> AccessControlledLoggedInUser to prevent the kind of really obvious AuthZ bugs people make in web applications time and time again.
I've found this pattern to be massively underutilized in industry.
This isn't specific to Rust or Typescript. You can do this in basically any language.
Imagine you have to distinguish between unescaped and escaped strings for security purposes. Even with a dynamically typed language, you can keep escaped strings as an Escaped class, with escape(str)->Escaped and dangerouslyAssumeEscaped(str)->Escaped functions (or static methods). There's a performance cost to this, so that's a tradeoff you have to weigh, but it is possible.
Another way of doing this is Application Hungarian[1], though that relies on the programmer more than it does on the compiler.
That part is (de facto) required for dynamically typed languages, but not for statically typed ones where the newtype constructor/deconstructor can be elided at compile time. Rust and C++ especially both do the latter by having true value types available for wrappers that evaporate into zero extra machine code.
But then just this moment I wondered: do any major runtimes using models with no static type info manage to do full newtype elision in the JIT and only box on the deopt path? What about for models with some static type info but no value types, like Java? (Java's model would imply trickiness around mutability, but it might be possible to detect the easy cases still.) I don't remember any, but it could've shown up when I wasn't looking.
And of course Rust and TypeScript were heavily influenced by Haskell... they just don't mention it and call things differently, to avoid the "monads are scary, I need to write a tutorial" effect. Though it's less about monads and more about things like type classes.
I loved working in Haskell for a few years. I wasn't actively looking it, but the opportunity just sort of landed in my lap. It was exciting and mentally stimulating. But the unfortunate fact is, I am easily twice as productive in Rust as I am Haskell, even after 3 years of nothing but Haskell. There are more pitfalls in Haskell that you have to just know how to avoid. It can be very difficult to digest as the language can be borderline write-only at times, depending on the author of the code. The tooling is often married to Nix, which is it's own complex beast. And it feels like language extensions are all over the place. Cabal files are not my favorite. And the compiler errors take some time to get get used to.
Pretty surprising -- I had much the opposite experience.
On our last product, we decided to start switching from Typescript to Rust on the backend because we got tired of crashes. I consider that to be one of the greatest technical mistakes I've made ever, as our productivity slowed massively. I'll just share two time-draining issues that only occur in Rust: (1) Writing higher-order functions (e.g.: a function to open a database connection, do something, and then close it -- yes, I know you can use RAII for this particular example), which is trivial in Haskell and TypeScript and JavaScript and C++ and PHP, turned out to be so impossible in Rust [even after asking Rust-expert friends for help], that I learned to just give up and never try, though it sometimes worked to write a macro instead. (2) It's happened many times that I would attempt a refactoring, spend all day fixing type errors, finally get to the top-level file, get a type error that's actually caused somewhere else by basic parts of the design, and conclude the entire refactoring I had attempted is impossible and need to revert everything.
On top of that, Rust is the only modern language I can name where using a value by its interface instead of its concrete type lies somewhere between advanced and impossible, depending on what exactly you're doing.
I came away concluding that application code (as opposed to systems or library code) should, to a first approximation, never be written in Rust.
That's pretty interesting. I was thinking about starting a new pet project and was considering doing it in Rust to learn as I never tried anything with it and after some small pocs I had the feeling it was too verbose to my taste, but wasn't sure it was just me and/or my lack of experience with Rust.
Still, wonder if it's still worth it to give a shot considering other positive elements of the language.
I appreciate Rust for making affine types mainstream, and having at least the C++ community start caring about security, even if half hearted.
However I share your conclusion, outside scenarios where having automated resource management as the main approach is either technically impossible, or a waste of time trying to change pervasive culture, I don't see much need for Rust.
In fact those that write comments about wanting a Rust but without borrow checker, the answer already exists.
(1) Higher order functions are pretty much the same as all the other languages you mentioned, using closure syntax? What was the problem you ran into?
(2) In such situations the compiler (type system or borrow checker) is telling you that what you wanted to do has hidden bugs, and therefore refuses to compile. Usually that's a good thing.
Maybe it depends on the application, but web servers are effortless with something like axum. Libraries can do a lot of heavy lifting to expose straightforward coding patterns. Never had any problems like you desribed with database connections and such. In rust with db pools things just work and get closed on drop etc. I would never even consider making a higher order function for that.
Only other language that I think gets close to rust ergonomics is Kotlin, but it suffers from having too many possibilities for abstractions.
That is a very unusual Rust experience. I find "application code" very pleasant to write in Rust. Of course there are things that aren't as ergonomic in Rust as in other languages (e.g. callbacks) but that's true of pretty much any language.
I think perhaps contrary to popular belief, Mercury choosing Haskell and their early leadership having such a storied experience in it probably played some non-insignificant role in their success.
As a customer of Mercury, it's truly one of the critical companies my toolkit, and I just can't help but feel that their choosing of Haskell made their progress, development and overall journey that much better. I realize that you can make this argument with most languages, and it's not to say that a FP lang like Haskell is a recipe for success, but this intentional decision particularly pre "vibe coding" and the LLM era seems particularly prescient, of course combined with their engineering culture that was detailed in the post.
I'd also wager that hiring generalists with no prior experience in the language actually helped them, because they got to instill their culture and style from the ground up with their new hires. Pre vibe-coding, most of those people would'nt have wanted to just jump in and hack away with zero instruction.
> [To lib authors] Nobody is obviously in charge in the way a fast-moving production team would mean "in charge," and that creates understandable hesitation around making breaking changes, even when experience has taught us better ways to design these systems.
> This is not a complaint about volunteer maintainers. It is simply one of the ambient risks of building serious systems on a smaller ecosystem.
And so instead of paying the lib authors who already have domain expertise and know their codebase, they chose to rewrite it from scratch/fork without contributing back. So classic.
I don't believe I'm the target market (I'm plenty happy with my small CU), and seeing their billboards makes me want to never use them BUT: seeing this post and their culture and that they use Haskell is kinda changing my mind.
It’s hard to imagine what two millions lines of Haskell could possibly be doing. I mean that’s a lot of code and I have the impression that Haskell is “tight” meaning a little code can do a lot. Maybe they have a lot of libraries to do things like json serializing/deserializing, rest api frameworks, logging etc?
> The problem is that we cannot trust code we cannot instrument. If a third-party binding makes HTTP calls through concrete functions, we have no way to add tracing, no way to inject timeouts tuned to our SLOs, no way to simulate partner outages in testing, and no way to explain the 400ms gap in a trace except by squinting at it and developing theories. So we write our own. More work upfront, but the clients we write are observable by construction, because we built them that way from the start.
Nit: the quality of a language that you call "tight" is usually called "expressive." You can use few characters to express a relatively very abstract idea.
Some people call this "high-level," too.
I will say, though, that 2 million lines of code is much less code than it sounds like at first glance, especially for a company in a highly-regulated space like finance, plus a few years of progress.
Absolutely not an objective metric, but I have found that Haskell just has a different "aspect ratio". Line count may be somewhat lower, but the word count is essentially largely the same as more imperative OOP languages.
I obviously don't know what the codebase looks like, but
a) Haskell's reputation for terseness partially comes from its overrepresentation in academic / category-theoretic circles, where it's typically fine to say things like `St M -> C T`. But for real software it's a lot more useful to say things like `TransactionState Debit -> Verified Transaction` etc etc.
b) The other part of Haskell's terse reputation is cultural, something extending back to LISP: people being way too clever about saving lines with inscrutable tricks or macros. I imagine that stuff is discouraged at a finance company like Mercury in favor of clarity and readability: e.g. perhaps the linter makes you split monadic stuff into pedantic multiline do expressions even if you can do it in a one-liner with >> and >>=.
My bestie works at this company and looking from the outside they have a good engineering culture. I do think Haskell is the right tool for the job, and they are playing to it's strengths, but part of me wonders if a lot of their success is attributable to the place just being well run in general.
That would not run counter to the popular (whether true or not) idea that by using functional programming languages you filter for a higher quality labor pool / applicant pool.
The version I've always heard is just well designed but less popular languages, but the ones I can think of were all functional (Haskell/F#/OCaml/Clojure/Elm/Erlang)
I think you have to get a Haskell job early in career and stick to Haskell jobs. Breaking in is really hard as you come without experience there will be plenty of others with Haskell experience to compete. And because the jobs are rare if it doesnt work out (company becomes bad to work for or layoff) you can be unstuck (or I guess you would switch to Rust, Scala or F#)
As somebody who has helped hire many Haskell devs, I can say that lots of Haskell experience isn't always a positive. We have to filter carefully to make sure that we end up with developers who want to build real things, not developers who just want to get paid for noodling around with Haskell. As far as I'm concerned, I'd much rather hire somebody with lots of experience building things who ended up coming to Haskell later because they viscerally understand the benefits and risks. Somebody with lots and lots of Haskell experience who never delivered much is a big risk.
haha i've abused this recruiting mindset for a decade
it's so easy to scout when a company has this haskell philosophy. either by the interviewers themselves or by the bloggers they hired to guide their team.
the trick? i just..lie. "oh yeah i'm super pragmatic. i'm not hardline about haskell. i don't think you should be fancy." see how easy it is? i am suddenly hired and got a fat raise. and if the company moves off haskell? i quit immediately, get another haskell job, and talk to my former coworkers on the way out to embolden them to do the same.
it helps that i have the "real world" stuff on my resume.
i rode the 2010s job hopping ride as a haskeller doing this. each time a 20-30% raise. and i get to still write haskell. and i am always a top percentile haskeller at the company so i can code however tf i want lolol. suddenly - singletons, Generics, HKD!
so here's to earning another million bucks "noodling around with Haskell" :cheers:
My fear with something like Haskell particularly and with hiring people who really love Haskell is that you risk ending up with a certain kind of personality who fetishizes the tool over the problem.
I've been this person, and I've worked with this kind of person, and been the victim of this kind of person. They love language X, or framework Y, and are convinced that so many problems in front of them are shaped in a way that would be solved through the application of it.
They now have a hammer and they go searching for nails to hit with it.
I've been in shops that used Haskell, and it was... fine? It's I guess nice for people who enjoy writing in it -- I prefer other FP languages personally. I like nerdy things like that and used to hang out on Lambda the Ultimate or whatever. But I don't think there's any real secret powers in Haskell or most other tools. I've been burned too many times by that kind of approach.
I use Haskell a lot, but I notice that it's very hard to cross-compile it.
If only cross-compilation became easy so that I can develop on my chip Macs and deploy on x64/AMD Linux servers.
>statically linking Haskell binaries is quite a challenge
>build requirements really slow down the process. I have to use dockers to help cache dependencies and avoid recompiling things that have not changed, but it is still slow and puts out large binaries.
Also, the Docker-based deployment takes a lot of time as it needs to recompile each module. While you can cache some part of it, it's still slow.
Meanwhile with Go it's painless. And i am not the only one having this issue:
This is a great read. The "observable by construction" principle especially building traceability in from the start rather than bolting it on later is something more teams should internalize.
Hell yeah, I made a hyperbolic PDE solver in a bizarre constrained space in haskell! Unboxed vectors and performant algebriac systems out of functional programming is a blast.
I'm not particularly into Haskell or functional languages, but I came here to say it warms my heart to hear that people in finance actually embrace it (also J/APL). It seems like a good choice for banking.
The Mercury site also looks way better than most other banks I have ever used (load speed is also very good.) On the danger of seeming like a shill (I'm not), I'm tempted to try them out.
Mercury has been awesome, I've been using them for my business account for years and recently started using them for personal as well. I didn't know they used Haskell until well after I started using them, but it definitely tracks. The quality of their exposed software surface is at least a couple stddev above median.
Just wanted to say that rarely has a text make me want to work for a company as much as this one. This is the way engineering should be done. I have never worked in such an organisation.
not speaking in any official capacity, but: we great internal training material courtesy of some very thoughtful folks, and ultimately one hopes that most of the code is going to be pretty straightforward wherever possible.
There are two countervailing effects when you choose a more theoretically advanced programming language. On the one hand, your hiring pool shrinks. On the other hand, the quality of the remaining hiring pool goes way up, which acts as an excellent recruiting filter (for both employer and employee). Jane Street made a similar play with OCaml.
I really believe in FP and Haskell but I want to examine this objectively. Empirically speaking is what Mercury done successful truly because of Haskell? Do they have metrics that demonstrates clear superiority along some vectored trait like complexity, bug count, etc?
>A couple million lines of Haskell, maintained by people who learned the language on the job, at a company that moves huge amounts of money? The conventional wisdom says this should be a disaster, but surprisingly, it isn't. The system we've built has worked well for years, through hypergrowth, through the SVB crisis that sent $2 billion in new deposits our way in five days,1 through regulatory examinations, through all the ordinary and extraordinary things that happen to a financial system at scale.
This one is quite telling. Do people have counter examples?
Without having run the whole company twice in parallel, once using Haskell and again in some other language, and without having measured both runs exactly the same way, I don't think metrics like you're interested in could possibly have sufficient context to mean anything reliable.
Obviously Mercury is successful, and obviously Haskell is how they did it. So it's essential to their success. Would it be instrumental to anyone else's anywhere else doing anything else? Can't possibly know, I don't think.
The irony of these fancy FP languages that were designed to develop compilers or to get PL academics off is that they're actually also really good at the most mundane code imaginable.
Being able to minimize boilerplate and have strong refactoring and bug resistant types is a huge edge.
The only problem is their ecosystems are limited so you might spend more time than you like implementing an API or binding a system library.
It's probably more likely it comes down to: if your programmers are capable of learning and working in Haskell, they're likely a cut above in terms of keenosity and nerdiness and motivation around programming, and you're likely to early-cull the people who just got into CS because mom and dad told them it paid well. That's likely to help produce better overall code health.
I know this is not the point of the article, but I find the anecdote in the beginning about null pointer errors somewhat ironic. Haskell's solution to null pointers are option types (`Maybe x` in Haskell), but these are known to be suboptimal.
In languages with option types, if you want to weaken the type requirement for a function parameter, or strengthen the guarantee for a return type, you have to change the code at every call site. E.g, if you have a function which you can improve by changing
- a parameter Foo to Option<Foo> or
- a return value Option<Bar> to Bar
you would have to change the code at all call sites. Which could be anything between annoying and practically impossible.
In languages that solve null pointer errors instead with untagged union types (like TypeScript or Scala 3), this problem doesn't occur. So you can change
- a parameter Foo to Foo | Null or
- a return value Bar | Null to Bar
and all call sites of the function can remain unchanged, since the type system knows that weakening the type requirement for a parameter, or strengthening the promise for a return type, is a safe change than can't cause a type error.
So yes, option types do avoid null pointer exceptions, but they solve the issue in a very suboptimal way.
>you would have to change the code at all call sites.
Actually I think you can just change concrete argument `Foo` to type constraint in Haskell as well using a type class. So the function would be something like `foo :: ToMaybeFoo a => a -> .. ->`. And you would implment `ToMaybeFoo` instance for `Foo` and `Maybe Foo`.
Agree that this is more involved than typescript, but you get to keep `null` away from your code...
Admittedly I don't really understand your construction. But this solution, if it works, doesn't look practical enough that it could be routinely used in practice like Foo|Null could be. By the way, some languages even shorten "Foo|Null" to "Foo?" as syntax sugar.
> but you get to keep `null` away from your code...
I don't think this would be desirable once we have eliminated null pointer exceptions with untagged unions.
Mostly though if you do anything with the returned value at the call site you need to change that code anyways? If it is not just passing it on, and even then you might need to adapt its signatures. E.g. if you change from String | Null to String you remove the null handling. If you add Null you need to add Null handling?
If you were calling a function which might return null (String | Null), you will already have null handling at the call site, but if you now change that function such that it never returns null (String), you still have the (now unnecessary) null handling, but this doesn't hurt and you don't have to change anything at the call site.
Likewise, if you were passing a String to a function that doesn't accept null (String), the call site already made sure that the parameter isn't null, and if you change the function so that it does now accept null (String | Null), again nothing needs to be changed at the call site.
I just wanted to leave a glowing comment about Mercury itself, since this is one of the few times I’ll be able to.
I’ve been using Mercury for 5 years. In that time, I’ve been able to wire transfer money without having to worry it might disappear (functionally impossible at certain other banks), created hundreds of virtual debit cards each with their own limit and pulling from different accounts, created dozens of accounts (a “place to put money”) named by function (each of my household utilities gets its own account, with an automatic rule to pull in money whenever it gets paid out), and… well, I think that covers everything.
This has given me unprecedented insight into my financial life. I know exactly how much I spend on groceries, on each utility, and on entertainment. I can project ahead and get a burn rate for my household. And my ex wife uses it too, on the same login, which is as easy as “make an account named with her first name” and a corresponding virtual debit card.
I’m convinced the only reason people don’t use Mercury is that they don’t know what they’re missing.
You have to pay for personal banking (a couple hundred a year iirc), but the business banking is free. If you want to try them out, you can start an LLC for a few dollars (at least in Missouri) and get overnight access to Mercury. All that’s required is your EIN.
They’ve been one of the single best products I’ve ever used. The sole wrinkle was when they canceled all their existing virtual cards due to reasons, which threw my recurring billing into chaos. But every great company is allowed at least one mega annoyance, and that one was a blip.
If you’re wondering whether to try them out, the answer is yes, and I’m excited for you to discover how cool it is. https://www.mercury.com
I badly want to try out Mercury, in fact I have an account already, but they don't yet support Zelle. I really hope their national bank charter application gets approved. (They applied for one last December.) Once they support Zelle, I'll drop U.S. Bank, as their app sucks and they don't have an API.
> I’m convinced the only reason people don’t use Mercury is that they don’t know what they’re missing.
Very well could be true because I had no idea who or what they are.
Do they have strong low level automation support for the customer programmatically even for personal accounts? I use ledger for plaintext accounting for both personal and business and sync of data is slightly annoying, perhaps Mercury’s products solve that trivially?
I run a small business books using mercury and beancount. The API supports enough operations in the free tier to do so with ease, though mostly I’m just fetching transactions. I do pay the ~35USD / mo for the extended API to get invoicing, though that’s not something a personal user would need
Feel free to use it as it stores data on your browser's local storage only. For syncing between devices, you would be able to use Google firebase's free tier and export your accounts (after compressing and encrypting) there and import from another device. Let me know if you want to try it..
> Haskell gives you tools to encode these incantations in types so they cannot be forgotten. This is, for my money, the single most valuable thing the language offers a production engineering organization.
Haskell is admittedly, probably the most powerful widely (or even somewhat widely) used language for doing this, but this general pattern works really well in Rust and TypeScript too and is one of my very favorite tools for writing better code.
I also really like doing things like User -> LoggedInUser -> AccessControlledLoggedInUser to prevent the kind of really obvious AuthZ bugs people make in web applications time and time again.
I've found this pattern to be massively underutilized in industry.
This isn't specific to Rust or Typescript. You can do this in basically any language.
Imagine you have to distinguish between unescaped and escaped strings for security purposes. Even with a dynamically typed language, you can keep escaped strings as an Escaped class, with escape(str)->Escaped and dangerouslyAssumeEscaped(str)->Escaped functions (or static methods). There's a performance cost to this, so that's a tradeoff you have to weigh, but it is possible.
Another way of doing this is Application Hungarian[1], though that relies on the programmer more than it does on the compiler.
[1] https://www.joelonsoftware.com/2005/05/11/making-wrong-code-...
> There's a performance cost to this
That part is (de facto) required for dynamically typed languages, but not for statically typed ones where the newtype constructor/deconstructor can be elided at compile time. Rust and C++ especially both do the latter by having true value types available for wrappers that evaporate into zero extra machine code.
But then just this moment I wondered: do any major runtimes using models with no static type info manage to do full newtype elision in the JIT and only box on the deopt path? What about for models with some static type info but no value types, like Java? (Java's model would imply trickiness around mutability, but it might be possible to detect the easy cases still.) I don't remember any, but it could've shown up when I wasn't looking.
2 replies →
> You can do this in basically any language.
You can do it in Assembly. That doesn't mean it's cost effective.
2 replies →
> works really well in Rust and TypeScript too
And of course Rust and TypeScript were heavily influenced by Haskell... they just don't mention it and call things differently, to avoid the "monads are scary, I need to write a tutorial" effect. Though it's less about monads and more about things like type classes.
Imitation is the sincerest form of flattery.
Are type classes scary? PHP has had them since 2012.
2 replies →
I loved working in Haskell for a few years. I wasn't actively looking it, but the opportunity just sort of landed in my lap. It was exciting and mentally stimulating. But the unfortunate fact is, I am easily twice as productive in Rust as I am Haskell, even after 3 years of nothing but Haskell. There are more pitfalls in Haskell that you have to just know how to avoid. It can be very difficult to digest as the language can be borderline write-only at times, depending on the author of the code. The tooling is often married to Nix, which is it's own complex beast. And it feels like language extensions are all over the place. Cabal files are not my favorite. And the compiler errors take some time to get get used to.
Pretty surprising -- I had much the opposite experience.
On our last product, we decided to start switching from Typescript to Rust on the backend because we got tired of crashes. I consider that to be one of the greatest technical mistakes I've made ever, as our productivity slowed massively. I'll just share two time-draining issues that only occur in Rust: (1) Writing higher-order functions (e.g.: a function to open a database connection, do something, and then close it -- yes, I know you can use RAII for this particular example), which is trivial in Haskell and TypeScript and JavaScript and C++ and PHP, turned out to be so impossible in Rust [even after asking Rust-expert friends for help], that I learned to just give up and never try, though it sometimes worked to write a macro instead. (2) It's happened many times that I would attempt a refactoring, spend all day fixing type errors, finally get to the top-level file, get a type error that's actually caused somewhere else by basic parts of the design, and conclude the entire refactoring I had attempted is impossible and need to revert everything.
On top of that, Rust is the only modern language I can name where using a value by its interface instead of its concrete type lies somewhere between advanced and impossible, depending on what exactly you're doing.
I came away concluding that application code (as opposed to systems or library code) should, to a first approximation, never be written in Rust.
That's pretty interesting. I was thinking about starting a new pet project and was considering doing it in Rust to learn as I never tried anything with it and after some small pocs I had the feeling it was too verbose to my taste, but wasn't sure it was just me and/or my lack of experience with Rust. Still, wonder if it's still worth it to give a shot considering other positive elements of the language.
2 replies →
I appreciate Rust for making affine types mainstream, and having at least the C++ community start caring about security, even if half hearted.
However I share your conclusion, outside scenarios where having automated resource management as the main approach is either technically impossible, or a waste of time trying to change pervasive culture, I don't see much need for Rust.
In fact those that write comments about wanting a Rust but without borrow checker, the answer already exists.
1 reply →
(1) Higher order functions are pretty much the same as all the other languages you mentioned, using closure syntax? What was the problem you ran into?
(2) In such situations the compiler (type system or borrow checker) is telling you that what you wanted to do has hidden bugs, and therefore refuses to compile. Usually that's a good thing.
(3) &dyn Trait
3 replies →
Maybe it depends on the application, but web servers are effortless with something like axum. Libraries can do a lot of heavy lifting to expose straightforward coding patterns. Never had any problems like you desribed with database connections and such. In rust with db pools things just work and get closed on drop etc. I would never even consider making a higher order function for that.
Only other language that I think gets close to rust ergonomics is Kotlin, but it suffers from having too many possibilities for abstractions.
1 reply →
That is a very unusual Rust experience. I find "application code" very pleasant to write in Rust. Of course there are things that aren't as ergonomic in Rust as in other languages (e.g. callbacks) but that's true of pretty much any language.
5 replies →
Is the productivity 2x all across the board, or are there some parts that are less productive with Rust? Also, what do you mean by write-only?
>what do you mean by write-only?
I think they meant that in Haskell it is very easy to write externally unreadable code..
I think perhaps contrary to popular belief, Mercury choosing Haskell and their early leadership having such a storied experience in it probably played some non-insignificant role in their success.
As a customer of Mercury, it's truly one of the critical companies my toolkit, and I just can't help but feel that their choosing of Haskell made their progress, development and overall journey that much better. I realize that you can make this argument with most languages, and it's not to say that a FP lang like Haskell is a recipe for success, but this intentional decision particularly pre "vibe coding" and the LLM era seems particularly prescient, of course combined with their engineering culture that was detailed in the post.
I'd also wager that hiring generalists with no prior experience in the language actually helped them, because they got to instill their culture and style from the ground up with their new hires. Pre vibe-coding, most of those people would'nt have wanted to just jump in and hack away with zero instruction.
I have noticed that everything in their app Just Works. It's very satisfying coming from other services!
I feel the same way. I only started using Mercury about 6 months ago and I’m continually impressed that it just makes sense.
> [To lib authors] Nobody is obviously in charge in the way a fast-moving production team would mean "in charge," and that creates understandable hesitation around making breaking changes, even when experience has taught us better ways to design these systems.
> This is not a complaint about volunteer maintainers. It is simply one of the ambient risks of building serious systems on a smaller ecosystem.
And so instead of paying the lib authors who already have domain expertise and know their codebase, they chose to rewrite it from scratch/fork without contributing back. So classic.
Now you can develop the lib in the direction that you need and you have people on payroll that do it, this seems like good risk management.
I don't believe I'm the target market (I'm plenty happy with my small CU), and seeing their billboards makes me want to never use them BUT: seeing this post and their culture and that they use Haskell is kinda changing my mind.
It’s hard to imagine what two millions lines of Haskell could possibly be doing. I mean that’s a lot of code and I have the impression that Haskell is “tight” meaning a little code can do a lot. Maybe they have a lot of libraries to do things like json serializing/deserializing, rest api frameworks, logging etc?
From TFA:
> The problem is that we cannot trust code we cannot instrument. If a third-party binding makes HTTP calls through concrete functions, we have no way to add tracing, no way to inject timeouts tuned to our SLOs, no way to simulate partner outages in testing, and no way to explain the 400ms gap in a trace except by squinting at it and developing theories. So we write our own. More work upfront, but the clients we write are observable by construction, because we built them that way from the start.
Nit: the quality of a language that you call "tight" is usually called "expressive." You can use few characters to express a relatively very abstract idea.
Some people call this "high-level," too.
I will say, though, that 2 million lines of code is much less code than it sounds like at first glance, especially for a company in a highly-regulated space like finance, plus a few years of progress.
> Haskell is “tight”
Absolutely not an objective metric, but I have found that Haskell just has a different "aspect ratio". Line count may be somewhat lower, but the word count is essentially largely the same as more imperative OOP languages.
I obviously don't know what the codebase looks like, but
a) Haskell's reputation for terseness partially comes from its overrepresentation in academic / category-theoretic circles, where it's typically fine to say things like `St M -> C T`. But for real software it's a lot more useful to say things like `TransactionState Debit -> Verified Transaction` etc etc.
b) The other part of Haskell's terse reputation is cultural, something extending back to LISP: people being way too clever about saving lines with inscrutable tricks or macros. I imagine that stuff is discouraged at a finance company like Mercury in favor of clarity and readability: e.g. perhaps the linter makes you split monadic stuff into pedantic multiline do expressions even if you can do it in a one-liner with >> and >>=.
My bestie works at this company and looking from the outside they have a good engineering culture. I do think Haskell is the right tool for the job, and they are playing to it's strengths, but part of me wonders if a lot of their success is attributable to the place just being well run in general.
That would not run counter to the popular (whether true or not) idea that by using functional programming languages you filter for a higher quality labor pool / applicant pool.
The version I've always heard is just well designed but less popular languages, but the ones I can think of were all functional (Haskell/F#/OCaml/Clojure/Elm/Erlang)
Will AI agents turn Haskell into a commercially viable language?
A similar Haskell success story (from Bellroy) is the subject of an upcoming Melbourne Compose meeting: https://luma.com/uhdgct1v
I think you have to get a Haskell job early in career and stick to Haskell jobs. Breaking in is really hard as you come without experience there will be plenty of others with Haskell experience to compete. And because the jobs are rare if it doesnt work out (company becomes bad to work for or layoff) you can be unstuck (or I guess you would switch to Rust, Scala or F#)
As somebody who has helped hire many Haskell devs, I can say that lots of Haskell experience isn't always a positive. We have to filter carefully to make sure that we end up with developers who want to build real things, not developers who just want to get paid for noodling around with Haskell. As far as I'm concerned, I'd much rather hire somebody with lots of experience building things who ended up coming to Haskell later because they viscerally understand the benefits and risks. Somebody with lots and lots of Haskell experience who never delivered much is a big risk.
haha i've abused this recruiting mindset for a decade
it's so easy to scout when a company has this haskell philosophy. either by the interviewers themselves or by the bloggers they hired to guide their team.
the trick? i just..lie. "oh yeah i'm super pragmatic. i'm not hardline about haskell. i don't think you should be fancy." see how easy it is? i am suddenly hired and got a fat raise. and if the company moves off haskell? i quit immediately, get another haskell job, and talk to my former coworkers on the way out to embolden them to do the same.
it helps that i have the "real world" stuff on my resume.
i rode the 2010s job hopping ride as a haskeller doing this. each time a 20-30% raise. and i get to still write haskell. and i am always a top percentile haskeller at the company so i can code however tf i want lolol. suddenly - singletons, Generics, HKD!
so here's to earning another million bucks "noodling around with Haskell" :cheers:
Interesting, I guess it then depends on the company (or recruiter) then.
This happened to me.
I've made all my money over a decade in Haskell. Millions. Paid for all my stuff.
It all started with a recruiter on LinkedIn
My fear with something like Haskell particularly and with hiring people who really love Haskell is that you risk ending up with a certain kind of personality who fetishizes the tool over the problem.
I've been this person, and I've worked with this kind of person, and been the victim of this kind of person. They love language X, or framework Y, and are convinced that so many problems in front of them are shaped in a way that would be solved through the application of it.
They now have a hammer and they go searching for nails to hit with it.
I've been in shops that used Haskell, and it was... fine? It's I guess nice for people who enjoy writing in it -- I prefer other FP languages personally. I like nerdy things like that and used to hang out on Lambda the Ultimate or whatever. But I don't think there's any real secret powers in Haskell or most other tools. I've been burned too many times by that kind of approach.
I use Haskell a lot, but I notice that it's very hard to cross-compile it.
If only cross-compilation became easy so that I can develop on my chip Macs and deploy on x64/AMD Linux servers.
>statically linking Haskell binaries is quite a challenge
>build requirements really slow down the process. I have to use dockers to help cache dependencies and avoid recompiling things that have not changed, but it is still slow and puts out large binaries.
Also, the Docker-based deployment takes a lot of time as it needs to recompile each module. While you can cache some part of it, it's still slow.
Meanwhile with Go it's painless. And i am not the only one having this issue:
https://news.ycombinator.com/item?id=47957624#47972671
Such a shame Haskell is beautiful and performant language still build is slow.
This is a great read. The "observable by construction" principle especially building traceability in from the start rather than bolting it on later is something more teams should internalize.
Impressive scale! Would love to hear more about their testing strategy and how they handle dependency management at that size.
Hell yeah, I made a hyperbolic PDE solver in a bizarre constrained space in haskell! Unboxed vectors and performant algebriac systems out of functional programming is a blast.
I'm not particularly into Haskell or functional languages, but I came here to say it warms my heart to hear that people in finance actually embrace it (also J/APL). It seems like a good choice for banking.
The Mercury site also looks way better than most other banks I have ever used (load speed is also very good.) On the danger of seeming like a shill (I'm not), I'm tempted to try them out.
Do it. You won’t be disappointed. I’ve been using them for about 5 years now.
> written using the mosaic theory of information and a range of journalistic tools
what does that mean?
Mercury has been awesome, I've been using them for my business account for years and recently started using them for personal as well. I didn't know they used Haskell until well after I started using them, but it definitely tracks. The quality of their exposed software surface is at least a couple stddev above median.
Just wanted to say that rarely has a text make me want to work for a company as much as this one. This is the way engineering should be done. I have never worked in such an organisation.
Haskell is like an instrument I can never quite master, yet I find it utterly fascinating and keep trying to learn it without success.
risky move, what is the talent pool for Haskell devs these days?
not speaking in any official capacity, but: we great internal training material courtesy of some very thoughtful folks, and ultimately one hopes that most of the code is going to be pretty straightforward wherever possible.
There are two countervailing effects when you choose a more theoretically advanced programming language. On the one hand, your hiring pool shrinks. On the other hand, the quality of the remaining hiring pool goes way up, which acts as an excellent recruiting filter (for both employer and employee). Jane Street made a similar play with OCaml.
I really believe in FP and Haskell but I want to examine this objectively. Empirically speaking is what Mercury done successful truly because of Haskell? Do they have metrics that demonstrates clear superiority along some vectored trait like complexity, bug count, etc?
>A couple million lines of Haskell, maintained by people who learned the language on the job, at a company that moves huge amounts of money? The conventional wisdom says this should be a disaster, but surprisingly, it isn't. The system we've built has worked well for years, through hypergrowth, through the SVB crisis that sent $2 billion in new deposits our way in five days,1 through regulatory examinations, through all the ordinary and extraordinary things that happen to a financial system at scale.
This one is quite telling. Do people have counter examples?
Without having run the whole company twice in parallel, once using Haskell and again in some other language, and without having measured both runs exactly the same way, I don't think metrics like you're interested in could possibly have sufficient context to mean anything reliable.
Obviously Mercury is successful, and obviously Haskell is how they did it. So it's essential to their success. Would it be instrumental to anyone else's anywhere else doing anything else? Can't possibly know, I don't think.
I’m asking for solutions and answers. Yeah. I’m aware of how hard it is to get metrics.
You can still compare lines of code and bug rate over the same period of time.
The irony of these fancy FP languages that were designed to develop compilers or to get PL academics off is that they're actually also really good at the most mundane code imaginable.
Being able to minimize boilerplate and have strong refactoring and bug resistant types is a huge edge.
The only problem is their ecosystems are limited so you might spend more time than you like implementing an API or binding a system library.
It's probably more likely it comes down to: if your programmers are capable of learning and working in Haskell, they're likely a cut above in terms of keenosity and nerdiness and motivation around programming, and you're likely to early-cull the people who just got into CS because mom and dad told them it paid well. That's likely to help produce better overall code health.
A couple million lines of code?
Try a better programming language next time, dagnabbit!!!
(There will be downvotes I suppose. More lines of code the better?)
I know this is not the point of the article, but I find the anecdote in the beginning about null pointer errors somewhat ironic. Haskell's solution to null pointers are option types (`Maybe x` in Haskell), but these are known to be suboptimal.
In languages with option types, if you want to weaken the type requirement for a function parameter, or strengthen the guarantee for a return type, you have to change the code at every call site. E.g, if you have a function which you can improve by changing
- a parameter Foo to Option<Foo> or
- a return value Option<Bar> to Bar
you would have to change the code at all call sites. Which could be anything between annoying and practically impossible.
In languages that solve null pointer errors instead with untagged union types (like TypeScript or Scala 3), this problem doesn't occur. So you can change
- a parameter Foo to Foo | Null or
- a return value Bar | Null to Bar
and all call sites of the function can remain unchanged, since the type system knows that weakening the type requirement for a parameter, or strengthening the promise for a return type, is a safe change than can't cause a type error.
So yes, option types do avoid null pointer exceptions, but they solve the issue in a very suboptimal way.
>you would have to change the code at all call sites.
Actually I think you can just change concrete argument `Foo` to type constraint in Haskell as well using a type class. So the function would be something like `foo :: ToMaybeFoo a => a -> .. ->`. And you would implment `ToMaybeFoo` instance for `Foo` and `Maybe Foo`.
Agree that this is more involved than typescript, but you get to keep `null` away from your code...
Admittedly I don't really understand your construction. But this solution, if it works, doesn't look practical enough that it could be routinely used in practice like Foo|Null could be. By the way, some languages even shorten "Foo|Null" to "Foo?" as syntax sugar.
> but you get to keep `null` away from your code...
I don't think this would be desirable once we have eliminated null pointer exceptions with untagged unions.
1 reply →
Mostly though if you do anything with the returned value at the call site you need to change that code anyways? If it is not just passing it on, and even then you might need to adapt its signatures. E.g. if you change from String | Null to String you remove the null handling. If you add Null you need to add Null handling?
No that's not right.
If you were calling a function which might return null (String | Null), you will already have null handling at the call site, but if you now change that function such that it never returns null (String), you still have the (now unnecessary) null handling, but this doesn't hurt and you don't have to change anything at the call site.
Likewise, if you were passing a String to a function that doesn't accept null (String), the call site already made sure that the parameter isn't null, and if you change the function so that it does now accept null (String | Null), again nothing needs to be changed at the call site.
4 replies →
I just wanted to leave a glowing comment about Mercury itself, since this is one of the few times I’ll be able to.
I’ve been using Mercury for 5 years. In that time, I’ve been able to wire transfer money without having to worry it might disappear (functionally impossible at certain other banks), created hundreds of virtual debit cards each with their own limit and pulling from different accounts, created dozens of accounts (a “place to put money”) named by function (each of my household utilities gets its own account, with an automatic rule to pull in money whenever it gets paid out), and… well, I think that covers everything.
This has given me unprecedented insight into my financial life. I know exactly how much I spend on groceries, on each utility, and on entertainment. I can project ahead and get a burn rate for my household. And my ex wife uses it too, on the same login, which is as easy as “make an account named with her first name” and a corresponding virtual debit card.
I’m convinced the only reason people don’t use Mercury is that they don’t know what they’re missing.
You have to pay for personal banking (a couple hundred a year iirc), but the business banking is free. If you want to try them out, you can start an LLC for a few dollars (at least in Missouri) and get overnight access to Mercury. All that’s required is your EIN.
They’ve been one of the single best products I’ve ever used. The sole wrinkle was when they canceled all their existing virtual cards due to reasons, which threw my recurring billing into chaos. But every great company is allowed at least one mega annoyance, and that one was a blip.
If you’re wondering whether to try them out, the answer is yes, and I’m excited for you to discover how cool it is. https://www.mercury.com
> I’ve been able to wire transfer money without having to worry it might disappear
I thought that was the whole point of banks - does money randomly disappear when people do wire transfers?
Can confirm. I've been using Mercury for three years now for my LLC. Everything is just so smooth.
I badly want to try out Mercury, in fact I have an account already, but they don't yet support Zelle. I really hope their national bank charter application gets approved. (They applied for one last December.) Once they support Zelle, I'll drop U.S. Bank, as their app sucks and they don't have an API.
> I’m convinced the only reason people don’t use Mercury is that they don’t know what they’re missing.
Very well could be true because I had no idea who or what they are.
Do they have strong low level automation support for the customer programmatically even for personal accounts? I use ledger for plaintext accounting for both personal and business and sync of data is slightly annoying, perhaps Mercury’s products solve that trivially?
They sure do!
API: https://docs.mercury.com/docs/welcome
MCP: https://docs.mercury.com/docs/what-is-mercury-mcp
I run a small business books using mercury and beancount. The API supports enough operations in the free tier to do so with ease, though mostly I’m just fetching transactions. I do pay the ~35USD / mo for the extended API to get invoicing, though that’s not something a personal user would need
> I use ledger for plaintext accounting for both personal and business and sync of data is slightly annoying..
I made this to solve it https://sras.me/accounts/
Feel free to use it as it stores data on your browser's local storage only. For syncing between devices, you would be able to use Google firebase's free tier and export your accounts (after compressing and encrypting) there and import from another device. Let me know if you want to try it..
[dead]
[dead]