Comment by jsnell
2 days ago
I don't understand what distinction you're trying to draw here. The very specific claim[0] in the Bun blog post that Kelley is calling a fabrication was:
> We fuzz Bun's runtime APIs 24/7 using Fuzzilli, the JavaScript engine fuzzer used by V8 & JavaScriptCore
It does not look to be a fabrication, and is very explicit just about what they meant by fuzzing.
[0] I mean, that sentence doesn't actually match Kelley's paraphrase, but it is literally the only claim in the post related to what fuzzing was done on the Zig-based bun codebase. So it has to be what Kelley was referring to, and his paraphrase is as sloppy as his fact-checking.
For me, using Fuzzilli for testing a Zig code is not fuzzing, it's integration testing. If you're running code externally (e.g. wrapping binary) you cannot guarantee that side effect isn't caused by IO. I consider fuzzing a low level activity with many external variables removed.
Depending on where you are and how you communicate semantics matter more or less. It's very similar to compiler/transpiler. E.g. TypeScript "Compiler" is called compiler but in fact it's transpiler (it emits other high-level language as a result).
My point is that Kelley did not argue that what Bun does isn't really fuzzing. He wrote that the post's claim is a fabrication. But that claim is really specific, and to evaluate whether it is true it doesn't matter what Kelley's unstated definition of fuzzing is.
So an argument about definitions doesn't seem super valuable here.
The Fuzzilli PR was merged on Nov 20. The acquisition was announced on Dec 3. A big holiday was in the middle.
The teams no longer interacted after the acquisition, and in prior interactions the Bun team would've been correct in saying they weren't fuzzing.
So Jarred isn't wrong, and Andrew also isn't wrong.
3 replies →
That misses the mark here. Every other Kelley's claim has also been unsourced and not cited. I was able to guess that the integration might've simply been something which happened after they'd once not had it.
Given the lack of due diligence here, it seems Kelley's intentions weren't to be objective potrayal of truth but whatever was most damaging.
> For me, using Fuzzilli for testing a Zig code is not fuzzing, it's integration testing. If you're running code externally (e.g. wrapping binary) you cannot guarantee that side effect isn't caused by IO. I consider fuzzing a low level activity with many external variables removed.
I've never heard anyone restrict the definition of "fuzzing" in this way. If I repeatedly generate inputs to a program and then run the program with those inputs, that's fuzzing. It doesn't matter if there's IO or not.
> Depending on where you are and how you communicate semantics matter more or less. It's very similar to compiler/transpiler. E.g. TypeScript "Compiler" is called compiler but in fact it's transpiler (it emits other high-level language as a result).
It's still a compiler. It translates code from one language to another. You can argue whether we need the term "transpiler," but a source-to-source compiler is a compiler.
> You can argue whether we need the term "transpiler," but a source-to-source compiler is a compiler.
That's true today, but compiling was historically was defined as getting source code (human readable) to bytecode (machine runnable without an interpreter).
Some people didn't like that definition, and consequently the waters have been murkied. Just like with eg crypto. Or real time.
2 replies →
The conversation amounts to “You should fuzz your code” “we’re already fuzzing the big external dependency, using their own fuzzing setup that they already use upstream”.
It’s not nothing, but clearly not what Andrew meant.