Complete rewrites have been done before. They're not impossible tasks. C to C++ (like with fish), C++ to Rust (fish again, Tor, etc), Java to C# (something I've seen), C# to C++ (evernote), Blender Game Engine to Godot (many small cases), Godot to Lumberyard (something I've seen), Unity to Godot (many small cases), etc. And there are all the myriad rewrites within the same language, or across major breaking changes (like Python 2 to Python 3).
I think "you should never do a full rewrite" is something of a reactionary response to the juvenile idea new devs and interns get every now and then. But sometimes, a rewrite really is a good idea.
But in this case, per the announcement ( https://blog.torproject.org/announcing-arti/ ), the reasons are pretty convincing. The major reasons are, unsurprisingly, all around memory safety.
> Godot to Lumberyard
Do you remember what project(s) this was? I'd be super curious on the motivations of the developers. I guess I could see it when it first went open source and Godot wasn't all it was today.
It was a project I worked on as an intern at Electric Boat. I can only speak to specifics I was explicitly given the okay to share though (i.e. this is all on my LinkedIn).
The project was a virtual training simulation for a submarine cockpit, first in Blender Game Engine. With support for BGE being deprecated, I was tasked to find an alternative engine and to port the work. (I'm very proud my research pointed to Godot- this was during the Godot 2 era).
There was a lot of interest in the project, but my internship was coming to an end and they decided to port it to Amazon Lumberyard for fear of Godot's longevity and support. They didn't want to be left "waiting for Godot". The final task I took on was to document the project, to make that transition easier for whoever came next.
The issue is that every other week there is a rewrite of something in Rust. I just do an eyeroll whenever I see that yet another thing is being rewritten in Rust.
I've tried compiling large projects in Rust in a VM (8GB) and I've run out of memory whereas I am sure a C/C++ large project of a similar size wouldn't run out of memory. A lot of this tooling I had to compile myself because it wasn't available for my Linux distro (Debian 12 at the time).
A lot of the tooling reminds me of NPM, and after spending a huge amount of my time fighting with NPM, I actually prefer the way C/C++/CMake handles stuff.
I also don't like the language. I do personal stuff in C++ and I found Rust really irritating when learning the language (the return rules are weird) and just gave up with it.
> whereas I am sure a C/C++ large project of a similar size wouldn't run out of memory
This is a common issue on large C++ projects for users with limited resources. You typically have to tinker with build settings and use a different linker etc.. to get the project building without OOMing.
> A lot of the tooling reminds me of NPM
My feeling is that you'd have the same criticism about a lot of other modern ecosystems because they allow you to pull dependencies easily. With things like vcpkg, that's also possible in C++ but even without it: nothing stops the same behavior of importing "random" code from the internet.
To be fair, you haven't explained why it's an issue to see projects being rewritten in Rust, other than it is a bit annoying to you?
For me, I had a very good experience rewriting a project in Rust (from Python). It was just an improvement in every regard (the project was easier to build and distribute, it was a good opportunity to rearchitect it, and the code ended up 20k times faster.) So, I have the opposite feeling when I see titles like these.
I also feel the opposite about the tooling. For me, cmake, npm, pip, maven, etc. all live in this realm where any invocation could become a time purgatory. The big thing that makes me like Rust is that I've never had that experience with cargo. (In fact, pip was the primary motivation to move away from Python to Rust. Everything else was just nice.)
I'm not saying this to convince you to feel otherwise, I just wanted to offer how I felt about the same topics. I grew up with C and Java, but Rust clicked after I read SICP and followed along in Racket. I could probably fill a page of small grievances about Rust's syntax if I had to.
> A lot of the tooling reminds me of NPM, and after spending a huge amount of my time fighting with NPM, I actually prefer the way C/C++/CMake handles stuff.
I guess you and me live different lives because I have spent far more time messing with ancient C/C++/CMake/Automake/Autoconf/configure/just have this particular .so file in this particular spot/copy some guys entire machine because this project only builds there/learn and entirely different language just to write build files (CMake sucks and 5 other alternatives are just lipstick on a pig) etc. etc.
I am of the opinion that half of Rusts success is based on that fact that C/C++'s tooling is annoying and ancient. People want to write code not mess with build envs.
Sure. That doesn't matter. Of course you can keep writing your C/C++ or using CMake, nobody is going to stop that. But other people's project are not going to stop adopt new tech stack because how you feel about it.
RE: memory, any self-respecting CI/CD system will allow you to compile any Rust project without out-of-memory halts.
RE: NPM, you have a right to a preference of course. I certainly don't miss the times 20 years ago when pulling in a library into a C++ project was a two-week project in itself. make and CMake work perfect right up until they don't and the last 2% cost you 98% of the time. Strategically speaking, using make/CMake is simply unjustified risk. But this is of course always written off as "skill issue!" which I gave up arguing against because the arguers apparently have never hit the dark corners. I have. I am better with Just and Cargo. And Elixir's mix / hex.
What you like as a language (or syntax) and what you roll your eyes at are obviously not technological or merit-based arguments. "Weird" I can't quite place as an argument either.
Use what you like. We all do the same. The original article lists good arguments in favor of Rust. Seems like a good case of "use the right tool for the job" to me.
Comelete rewrite are not always bad, they are _just_ very costly. You just need to do cost benefit analsys and compare it with the cost and benefit of other actions.
Well, an underrated aspect of the Rust rewrites, is that it's easy to publish and share official libraries from the projects that the community can use, something that is too hard in C land.
If anything, from a security standpoint, this is one of the bigger issues with a rewrite in Rust. Besides the "MIT EVERYTHING" mindset, Rust people are also similar to JS devs in having projects with huge supply chain attack surfaces, from their eagerness to rely on so many crates.
I agree this is problematic, sure, and is not unique to Rust or JS. Feel free to propose and work on something better, I'd be your enthusiastic supporter.
It's simply a way to be able to move at a reasonable speed is how I see it.
Nobody ever said rewrites are always a bad idea. You just made that up. It has been done successfully many times, with many more happening (e.g. TypeScript compiler rewrite).
What matters is how the project is planned and implemented.
Typically, complete rewrites that halt all feature development and fail to achieve feature parity is a recipe for disaster. On the other hand, balancing the work for feature/regular maintenance and rewrite and gradually achieving feature parity before rolling things out has worked well.
Complete rewrites have been done before. They're not impossible tasks. C to C++ (like with fish), C++ to Rust (fish again, Tor, etc), Java to C# (something I've seen), C# to C++ (evernote), Blender Game Engine to Godot (many small cases), Godot to Lumberyard (something I've seen), Unity to Godot (many small cases), etc. And there are all the myriad rewrites within the same language, or across major breaking changes (like Python 2 to Python 3).
I think "you should never do a full rewrite" is something of a reactionary response to the juvenile idea new devs and interns get every now and then. But sometimes, a rewrite really is a good idea.
But in this case, per the announcement ( https://blog.torproject.org/announcing-arti/ ), the reasons are pretty convincing. The major reasons are, unsurprisingly, all around memory safety.
> Godot to Lumberyard Do you remember what project(s) this was? I'd be super curious on the motivations of the developers. I guess I could see it when it first went open source and Godot wasn't all it was today.
It was a project I worked on as an intern at Electric Boat. I can only speak to specifics I was explicitly given the okay to share though (i.e. this is all on my LinkedIn).
The project was a virtual training simulation for a submarine cockpit, first in Blender Game Engine. With support for BGE being deprecated, I was tasked to find an alternative engine and to port the work. (I'm very proud my research pointed to Godot- this was during the Godot 2 era).
There was a lot of interest in the project, but my internship was coming to an end and they decided to port it to Amazon Lumberyard for fear of Godot's longevity and support. They didn't want to be left "waiting for Godot". The final task I took on was to document the project, to make that transition easier for whoever came next.
The issue is that every other week there is a rewrite of something in Rust. I just do an eyeroll whenever I see that yet another thing is being rewritten in Rust.
I've tried compiling large projects in Rust in a VM (8GB) and I've run out of memory whereas I am sure a C/C++ large project of a similar size wouldn't run out of memory. A lot of this tooling I had to compile myself because it wasn't available for my Linux distro (Debian 12 at the time).
A lot of the tooling reminds me of NPM, and after spending a huge amount of my time fighting with NPM, I actually prefer the way C/C++/CMake handles stuff.
I also don't like the language. I do personal stuff in C++ and I found Rust really irritating when learning the language (the return rules are weird) and just gave up with it.
> whereas I am sure a C/C++ large project of a similar size wouldn't run out of memory
This is a common issue on large C++ projects for users with limited resources. You typically have to tinker with build settings and use a different linker etc.. to get the project building without OOMing.
> A lot of the tooling reminds me of NPM
My feeling is that you'd have the same criticism about a lot of other modern ecosystems because they allow you to pull dependencies easily. With things like vcpkg, that's also possible in C++ but even without it: nothing stops the same behavior of importing "random" code from the internet.
1 reply →
To be fair, you haven't explained why it's an issue to see projects being rewritten in Rust, other than it is a bit annoying to you?
For me, I had a very good experience rewriting a project in Rust (from Python). It was just an improvement in every regard (the project was easier to build and distribute, it was a good opportunity to rearchitect it, and the code ended up 20k times faster.) So, I have the opposite feeling when I see titles like these.
I also feel the opposite about the tooling. For me, cmake, npm, pip, maven, etc. all live in this realm where any invocation could become a time purgatory. The big thing that makes me like Rust is that I've never had that experience with cargo. (In fact, pip was the primary motivation to move away from Python to Rust. Everything else was just nice.)
I'm not saying this to convince you to feel otherwise, I just wanted to offer how I felt about the same topics. I grew up with C and Java, but Rust clicked after I read SICP and followed along in Racket. I could probably fill a page of small grievances about Rust's syntax if I had to.
1 reply →
> A lot of the tooling reminds me of NPM, and after spending a huge amount of my time fighting with NPM, I actually prefer the way C/C++/CMake handles stuff.
I guess you and me live different lives because I have spent far more time messing with ancient C/C++/CMake/Automake/Autoconf/configure/just have this particular .so file in this particular spot/copy some guys entire machine because this project only builds there/learn and entirely different language just to write build files (CMake sucks and 5 other alternatives are just lipstick on a pig) etc. etc.
I am of the opinion that half of Rusts success is based on that fact that C/C++'s tooling is annoying and ancient. People want to write code not mess with build envs.
1 reply →
Sure. That doesn't matter. Of course you can keep writing your C/C++ or using CMake, nobody is going to stop that. But other people's project are not going to stop adopt new tech stack because how you feel about it.
1 reply →
RE: memory, any self-respecting CI/CD system will allow you to compile any Rust project without out-of-memory halts.
RE: NPM, you have a right to a preference of course. I certainly don't miss the times 20 years ago when pulling in a library into a C++ project was a two-week project in itself. make and CMake work perfect right up until they don't and the last 2% cost you 98% of the time. Strategically speaking, using make/CMake is simply unjustified risk. But this is of course always written off as "skill issue!" which I gave up arguing against because the arguers apparently have never hit the dark corners. I have. I am better with Just and Cargo. And Elixir's mix / hex.
What you like as a language (or syntax) and what you roll your eyes at are obviously not technological or merit-based arguments. "Weird" I can't quite place as an argument either.
Use what you like. We all do the same. The original article lists good arguments in favor of Rust. Seems like a good case of "use the right tool for the job" to me.
5 replies →
Comelete rewrite are not always bad, they are _just_ very costly. You just need to do cost benefit analsys and compare it with the cost and benefit of other actions.
Fish did a complete rewrite in Rust, but they did it in the right way.
1. Rewrite each C file in Rust as a 1:1 mapping.
2. Rewrite the Rust to make it idiomatic.
3. Start making structural changes.
Well, an underrated aspect of the Rust rewrites, is that it's easy to publish and share official libraries from the projects that the community can use, something that is too hard in C land.
If anything, from a security standpoint, this is one of the bigger issues with a rewrite in Rust. Besides the "MIT EVERYTHING" mindset, Rust people are also similar to JS devs in having projects with huge supply chain attack surfaces, from their eagerness to rely on so many crates.
On the flip side C/C++ devs like to let ancient libs hang around because updating them can be such a pain in the ass.
You can choose to write Rust with fewer external crates or vendor them like you would with c++ that is a dev choice.
Having more choices is better.
I agree this is problematic, sure, and is not unique to Rust or JS. Feel free to propose and work on something better, I'd be your enthusiastic supporter.
It's simply a way to be able to move at a reasonable speed is how I see it.
I worked at a startup where the original code base was outsourced to cheap developers and was riddled with security vulnerabilities.
A complete rewrite made sense.
Opinions with “always” in them are always a bad idea.
Nobody ever said rewrites are always a bad idea. You just made that up. It has been done successfully many times, with many more happening (e.g. TypeScript compiler rewrite).
What matters is how the project is planned and implemented.
Typically, complete rewrites that halt all feature development and fail to achieve feature parity is a recipe for disaster. On the other hand, balancing the work for feature/regular maintenance and rewrite and gradually achieving feature parity before rolling things out has worked well.