Comment by kibwen
2 days ago
> This is an effect of the language not having a proper ABI for compiling libraries as dynamically loadable modules
No, this is a design decision of Cargo to default to using project-local cached artifacts rather than caching them at the user or system level. You can configure Cargo to do so if you'd like. The reason it doesn't do this by default is because Cargo gives crates great latitude to configure themselves via compile-time flags, and any difference in flags means you get a different compiled artifact anyway. On top of that, there's the question of what `cargo clean` should do when you have a global cache rather than a local one.
> You can configure Cargo to do so if you'd like
How does one do this?
Why can't Cargo have a system like PyPI where library author uploads compiled binary (even with their specific flags) for each rust version/platform combination, and if said binary is missing for certain combination, fallback to local compile? Imagine `cargo publish` handle the compile+upload task, and crates.io be changed to also host binaries.
> Why can't Cargo have a system like PyPI where library author uploads compiled binary
Unless you have perfect reproducible builds, this is a security nightmare. Source code can be reviewed (and there are even projects to share databases of already reviewed Rust crates; IIRC, both Mozilla and Google have public repositories with their lists), but it's much harder to review a binary, unless you can reproducibly recreate it from the corresponding source code.
> Unless you have perfect reproducible builds
Or a trusted build server doing the builds. There is a build-bot building almost every Rust crate already for docs.rs.
1 reply →
I don’t think it’s that much of a security nightmare: the basic trust assumption that people make about the packaging ecosystem (that they trust their upstreams) remains the same whether they pull source or binaries.
I think the bigger issues are probably stability and size: no stable ABI combined with Rust’s current release cadence means that every package would essentially need to be rebuilt every six weeks. That’s a lot of churn and a lot of extra index space.
16 replies →
It runs counter to Cargos curreat model where the top-level workspace has complete control over compilation, including dependencies and compiler flags. I've been floating an idea of "opaque dependencies" that are like python depending on C libraries or a C++ library dependening on a dynamic library.
A trustworthy distributed cache would also work very well for this in practice. Cargo works with sccache. Using bazel + rbe can work even better.
That would work for debug builds (and that's something that I would appreciate) but not for release, as most of the time you want to compile for the exact CPU you're targeting not just for say “x86 Linux” to make sure your code is optimized properly using SIMD instructions.