Comment by ergonaught
1 year ago
While acknowledging one does not "have to" have so many dependencies, the prevalence of this npm-esque type of practice is one of the two things that destroyed all of my interest in Rust.
1 year ago
While acknowledging one does not "have to" have so many dependencies, the prevalence of this npm-esque type of practice is one of the two things that destroyed all of my interest in Rust.
Rust dependencies tend to be pretty high quality in my experience. Maintained by experts and offer new improvements over state-of-the-art.
But if you compare to C/C++ at least with Rust you _can_ but aren't required to use dependencies. In C/C++ if you want to, it's a _massive_ pain.
I care less about the quality of the dependencies than about the burden of protecting against supply chain attacks when there are a lot of dependencies.
In the past, I worked on a project for Luxembourg's CTIE (their IT administration). In most cases, they explicitly requested that we reimplemented features we needed instead of including more third party libraries. They just allowed essential libraries for the project, like Struts for the Web framework, or implementations of standard libraries like JPA, JTA etc. that came with WebSphere. Everything else, we had to reimplement. For them, it was just much easier to manage, given the amount of systems they have to manage. And the allowed libraries were only allowed in versions that they had reviewed before for security issues. In the end, reimplementing features/functions that we could have included with other libraries was never a reason for any problem : this practice requires some additional work, but has never been significant for the ability to deliver the project as expected.
1 reply →
Indeed, and that's a good reason to avoid third-party dependencies. But that's irrelevant to the choice of programming language; a language with a bad dependency manager might force you to build everything yourself, but you can always just do that, even in a language with a good dependency manager, you just choose to build everything yourself if you care.
Perplexingly, the original commenter seems to understand that this doesn't matter, and then handwaves away the correct conclusion.
16 replies →
Well if you look at the most recent open source supply chain attack on openssh, that used social engineering to add a backdoor to a project that openssh did not have a dependency on anywhere in it's SBOM. And with the xz example the backdoor had to be rushed out when it was deployed because the dynamic dependencey was being removed before the backdoor was completely in place. Doing a open source supply chain attack is not easy, fast or reliable for long.
It is not as simple as you say. Sometime it is better to know all of you dependencies are static linked at build time and specified when you are releasing your code. And the more sane you build system is the harder it is to add shellcode to your dependency's tarball and build scripts without turning peoples heads with random unsafe code.
3 replies →
Thank you.
As someone who works in cybersecurity and works closely with our developers, a lot of them tend to inherently trust third-party code with no auditing of the supply chain. I am always fighting that while yes, we don't need to reinvent the wheel and libraries/packages are important, our organzation and developers need to be aware of what we are bringing into our network and our codebase.
3 replies →
Nothing stops you from vendoring them into your repo and hand update each. But how would you do this in c++? Write everything from scratch? I mean rust doesn’t stopp you there
[edit] typos
Well, no one is forcing you to use these dependencies? Rust crates tend to be very minimal because how easy it is to use them.
The amount of code you have to review stays the same.
So ... then don't use them? No one forces anyone to use any dependencies in rust lol its just faster to use shit thats already made
One malicious dependency is enough. When you have 600 dependencies "tend to be pretty high quality" does not cut it.
it's completely stupid to measure "number of dependencies" in absolute numbers.
Lots of packages have a `-macros` or `-derive` transient dependency, meaning a single dependency can end up coutning as 3 additional dependencies.
Rust makes it simple to split packages into workspaces - for example, regex[1] consists of `regex-automata` and `regex-syntax` packages.
This composition and separation of concerns is a sign of good design, and not an npm-esque hellhole.
1. https://crates.io/crates/regex/1.11.1/dependencies
I suppose you could say that the audit burden scales linearly with the number of module publishers, with a small additional amount on every release point to confirm that the publisher is still who they purport to be and hasn't been compromised.
This is assuming that the audit consists of validating dependency authorship, and not the more labor-intensive approach of reviewing dependency code.
Hard no. Burden scales with number of lines. Lines being split into smaller chunks (crates) only speed up the process in long run.
6 replies →
Indeed. It's actually been quite handy on a few occasions to be able to just pull in a smaller crate as opposed to the whole project. (in constrast to, say, boost in C++, which is a big mess of a dependency even though it's one that goes to at least a little bit of effort to let you split it up, but through an ad-hoc process as opposed to a standard package management system).
(I would genuinely be interesting in an experiment which pushes this as far as possible: what if each function was a 'package'? There's already a decent understanding of how dependencies within a library work in the compiler, what if that extended to the package manager? You would know exactly what code you actually needed, and would only pull in exactly what was necessary)
that's kind of on rust for pushing crates front and center rather than groupings of crates that are developed / reviewed / released together as a single cohesive unit (typically a git repo).
e.g. go dependencies are counted on modules (roughly git repos), rather than packages (directories, compilation units). java is counted in packages rather than classes.
The vulnerability to supply chain attacks gives me pause. It's not unique to rust and it bothers me with npm or Python as well.
What are you comparing this to? Do you have positive examples? This seems to be a general dependancy management issue unrelated to rust—the reason C++ has this is that C++ also lacks any concept of dependencies, so people kind of just make do with modifying what packages are already integrated into the build process. This certainly doesn't imply you should trust boost (or the standard library, or whatever people use this decade, or xz, or whatever).
How many transitive dependencies is the right number for a database?
Honestly, current best practice puts that number right around zero, which you see for ambitious implementations.
A non-obvious issue is that database engines have peculiar requirements for how libraries are designed and implemented which almost no conventional library satisfies. To make matters worse, two different database implementations may have different requirements in this regard, so you can't even share libraries between databases. There are no black boxes in good database engines.
Compression libraries, OpenSSL, ICU, etc. are all common dependencies for databases.
Looking at the dependencies list (https://gist.github.com/tisonkun/06550d2dcd9cf6551887ee6305e...) I see plenty of reasonable things like:
* Base64/checksum/compression encoding libraries
* Encryption/hash libraries
* Platform-specific bindings (likely conditional dependencies)
* Bit hacking/casting/zero-copy libraries like bytemuck, zerocopy, zero-vec, etc.
* "Small"/stack allocated data structure libraries (smallvec, tinystr, etc.)
* Unicode libraries
There are certainly things that would add bloat too, but I think it's silly to pretend like everything here is something a database engine would need custom implementations of.
4 replies →
> current best practice puts that number right around zero
In the case where the answer is "zero", then that means that one does not actually need a package manager at all, in which case the features of the package manager are not relevant to the choice of language. This would imply that the parent commenter has no need to reject Rust.
Just tried to look at what some macro was generating using cargo-expand. It requires a LOT of dependencies. Took like 5 minutes to compile it all (run `cargo install cargo-expand` if you want to try). I almost aborted because the description of the crate says "Wrapper around rustc -Zunpretty=expanded." so I had expected the simplest possible crate to do that.
> Took like 5 minutes to compile it all
TBF this has nothing to do with dependency complexity and everything to do with semantic complexity. You could easily do this without using any dependencies at all.
unless you're downloading dependencies during the build or something like that, of course.
This take is utter nonsense to me - just don't use them...
imagine if one dependency is GPL lol
With over 600 dependencies, the probability goes up and up.