Comment by rootnod3
3 months ago
Fully agree. That is why I vendor all my dependencies. On the common lisp side a new tool emerged a while ago for that[1].
On top of that, I try to keep the dependencies to an absolute minimum. In my current project it's 15 dependencies, including the sub-dependencies.
I didn't vendor them, but I did do an eyeball scan of every package in the full tree for my project, primarily to gather their license requirements[1]. (This was surprisingly difficult for something that every project in theory must do to meet licensing requirements!) It amounted to approximately 50 dependencies pulled into the build, to create a single gstreamer plugin. Not a fan.
[1] https://github.com/ValveSoftware/Proton/commit/f21922d970888...
Vendoring is nice. Using the system version is nicer. If you can’t run on $current_debian, that’s very much a you problem. If postgres and nginx can do it, you can too.
The system package manager and the language package/dependency managers do a very different task.
The distro package manager delivers applications (like Firefox) and a coherent set of libraries needed to run those applications.
Most distro package managers (except Nix and its kin) don't allow you to install multiple versions of a library, have libs with different compile time options enabled (or they need separate packages for that). Once you need a different version of some library than, say, Firefox does, you're out of luck.
A language package manager by contrast delivers your dependency graph, pinned to certain versions you control, to build your application. It can install many different versions of a lib, possibly even link them in the same application.
But I don’t really want your version of the application, I want the one that is aligned to my system. If some feature is really critical to the application, you can detect them at runtime and bailout (in C at least). Most developers are too aggressive on version pinning.
> Most distro package managers (except Nix and its kin) don't allow you to install multiple versions of a library
They do, but most distro only supports one or two versions in the official repos.
1 reply →
> If you can’t run on $current_debian, that’s very much a you problem.
This is a reasonable position for most software, but definitely not all, especially when you fix a bug or add a feature in your dependent library and your Debian users (reasonably!) don't want to wait months or years for Debian to update their packages to get the benefits. This probably happens rarely for stable system software like postgres and nginx, but for less well-established usecases like running modern video games on Linux, it definitely comes up fairly often.
Something I have seen that recently have become much more common is the software upstream authors providing a Debian repository for the latest versions of their software, including backports for old Debian releases.
1 reply →
That is an impossible task in practice for most developers.
Many distros, and Debian in particular, apply extensive patches to upstream packages. Asking a developer to depend on every possible variation of such packages, across many distros, is a tall order. Postgres and Nginx might be able to do it, but those are established projects with large teams behind them and plenty of leverage. They might even be able to influence distro maintainers to their will, since no distro will want to miss out on carrying such popular packages.
So vendoring is in practice the only sane choice for smaller teams and projects.
Besides, distro package managers carrying libraries for all programming languages is an insane practice that is impossible to scale and maintain. It exists in this weird unspecified state that can technically be useful for end users, but is completely useless for developers. Are they supposed to develop on a specific distro for some reason? Should it carry sources or only binaries? Is the dependency resolution the same for all languages? Should language tooling support them? It's an entirely ridiculous practice that should be abandoned altogether.
Yes, it's also silly that every language has to reinvent the wheel for managing dependencies, and that it can introduce novel supply chain attack vectors, but the alternative is a far more ludicrous proposition.
> distro package managers carrying libraries for all programming languages is an insane practice that is impossible to scale and maintain.
That's not the idea. If a software is packaged for a distro, then the distro will have the libraries needed for that software.
If you're developing a new software and wants some new library not yet packaged, I believe you can figure how to get them on your system. The thread is about the user's system, not yours. When I want to run your code, you don't have to say:
8 replies →
You do not depends on a package, you depends on its API. Implementation details shouldn't matter if behavior stays the same. Why do you care if the distro reimplemented ffmpeg or libcurl, or use an alternative version built with musl? Either the library is there or it's not. Or the minimum version you want is there or it's not. You've already provided the code and the requirement list, it's up to the distro maintainer or the user to meet them. If the latter patch the code, why do you care that much?
And if a library have a feature flags, check them before using the part that is gated.
5 replies →
But that would lock me in to say whatever $debian provides. And some dependencies only exist as source because they are not packaged for $distribution.
Of course, if possible, just saying "hey, I need these dependencies from the system" is nicer, but also not error-free. If a system suddenly uses an older or newer version of a dependency, you might also run into trouble.
In either case, you run into either an a) trust problem or b) a maintenance problem. And in that scenario I tend to prefer option b), at least I know exactly whom to blame and who is in charge of fixing it: me.
Also comes down to the language I guess. Common Lisp has a tendency to use source packages anyway.
> If a system suddenly uses an older or newer version of a dependency, you might also run into trouble.
You won't. The user may. On his system.
1 reply →