← Back to context

Comment by hansvm

15 hours ago

Producing that general-purpose API from "battle proven 3rd party dependencies" comes at a cost. IME:

1. The API isn't quite what you need, so you add enough extra logic to integrate it that you might as well have reinvented the wheel instead and had a proper solution, plus you have all the bugs and performance problems associated with bolting on extra code to a domain mismatch issue.

2. The API covers cases you don't care about, opening you up to bugs and performance problems from the project's complexity while not actually saving much time since the cases you care about can be handled more simply.

3. Both (1) and (2) are amplified as you change the project going forward. If you wrote the data structure, you could easily tack on the modifications you need. If not, in the presence of desired modifications you needed to re-implement it anyway (or bolt on even more hacks), and the 3rd party isn't a good starting point because of (2).

For something like cryptography, with all the ways constant-time execution and whatnot can bite me in the ass, I'm happy to use a 3rd-party dependency. For almost everything else I'll home-cook it. Networking starts at io_uring. CPU float-intensive software starts with (depending on the domain) a Tensor type capable of lazy, fused operations [0].

That's just a heuristic I use, and from time to time I'll definitely take a shortcut. More often than not though, I'll be replacing the shortcut in less than a year, and the value from having the feature sooner is only sometimes worth the lost productivity. The driving factor is just that as I learn more it's easier and easier to home-cook those sorts of things, whereas the costs induced by 3rd parties haven't really diminished.

[0] I'm curious what a language designed around cache obliviousness and reducing data dependencies might look like. There are some patterns I've used which compose nicely, but they only solve part of the problem. Even custom vector languages like ISPC require a fair bit of work from the programmer.