Comment by shawabawa3
2 days ago
So, the charge was that they submitted a build specifically tuned for clickbench
The defense was that it wasn't specifically tuned for clickbench, it was just an early incomplete build that happened to only support exactly and clickbench queries and nothing else...
Doesn't feel like much of a defense to me. At best it's benchmaxxing if not clear outright cheating
> The defense was that it wasn't specifically tuned for clickbench, it was just an early incomplete build that happened to only support exactly and clickbench queries and nothing else.
ClickBench tests a very narrow set of SQL: scans, aggregations, sorts, and filters. It ends up being a great starting point for building a query engine before layering in more functionality
> The defense was that it wasn't specifically tuned for clickbench, it was just an early incomplete build that happened to only support exactly and clickbench queries and nothing else...
According to both the ClickHouse founder who was levying the accusations and the pgrust creator, the submission worked for many other queries it just didn’t have full SQL support.
So what is the point of benchmarking it?
Whenever people benchmark something my default assumption is that they want to know how well it performs, but I’m not the author and can’t speak to their intentions.