Comment by kibwen

6 months ago

Rather, they found one memory safety bug in their Rust codebase, and measured it against the legions of memory safety bugs they found in their C++ codebase. In neither case are they measuring against bugs not found, so no, it's not biased.

9 comments

kibwen

Reply
samdoesnothing  6 months ago

Except it's not an apples-to-apples comparison. The C++ code has been around a lot longer, and a lot of it was written with older versions of C++ which didn't have modern safety features. I'm sure there is a bunch of new/delete in their codebase still. And I'm sure they're actively looking for memory safety issues in C++, and probably not so hard (if at all) with Rust.

  • SkiFire13  6 months ago

    > The C++ code has been around a lot longer

    They made an earlier report where they found out that older C/C++ code has actually a lot less new vulnerabilities compared to new code, so I guess here they are comparing to new C/C++ code to get the higher ratio, meaning the comparison should actually be apples-to-apples.

  • Jweb_Guru  6 months ago

    Is it a usual thing you do that when you're given data about a literal thousandfold improvement, in a context where there are well-understood theoretical and practical reasons why you might have expected to see such an improvement, you make up reasons why it is actually not an improvement at all, without either investigating to see whether those reasons are actually true or demonstrating that even if they were true, they could possibly explain more than a tiny fraction of the improvement?

    • samdoesnothing  6 months ago

      I usually am skeptical about a literal thousandfold improvement, yes. And I'm not saying it's impossible, but rather that the data and the way it's presented has inherent biases. Its on the people making grandiose claims to prove them.

      1 reply →