Comment by baalimago
2 days ago
>what exactly is the problem with the current organization structure that would prevent any "grand" issues like these?
Well, it's summarized quite well here:
>"Performing large cross-cutting changes is also tricky because it will necessarily conflict with a lot of other changes being done to the compiler in the meantime. You can try to perform the modifications outside the main compiler tree, but that is almost doomed to fail, given how fast the compiler changes8. Alternatively, you try to land the changes in a single massive PR, which might lead to endless rebases (and might make it harder to find a reviewer). Or, you will need to do the migration incrementally, which might require maintaining two separate implementations of the same thing for a long time, which can be exhausting." - OP
A rigid organizational form (such as a company) can say: "Okay, we'll make an investment here and feature freeze until the refactors are done". I have a hard time seeing how the open source rust community who are doing this out of passion would get on board on such a journey. But maybe, who knows! The 'compilation people' would need to not only refactor to speed up the compilation times, they'd also need to encourage and/or perform refactors on all features being developed 'on main'. That, to me, sounds tedious and boring. Sort of like a job. Maybe something for the rust foundation to figure out.
I understand that quoted part as "it's tricky" rather than "It's impossible because no one wants to do it", just like many collaboration-efforts in FOSS. But you're right that it's probably for the foundation to figure out, a lone compiler-optimization geek isn't gonna be able to come up with a project-wide solution and force it through.
Haven't the Rust team already implemented "grand features" that took many years to get across the finish line? For example, GATs didn't look particularly fun, exciting or sexy, but somehow after being thought about and developed for like 5-6 years eventually landed in stable.
Edit: Also just remembered a lot of the work that "The Rust Async Working Group" has done, a lot which required a large collaborations between multiple groups within Rust. Seems to have worked out in the end too.
This is a concern for any fast-moving project, i.e. it's a good problem to have! You can work on your modifications on a side branch and then forward port them to the current state of main before proposing them for merge, it will probably be less work overall.