Comment by remywang

So I spent some time poking at the released 0.4-beta code. Controversy aside, it is interesting to see an open source system that beats ClickHouse on ClickBench - 4 out of the 5 systems ahead of CH (Umbra, elosdb, Hyper, CedarDB) are all closed-source, while the only open one is pivotlake.

What makes pgrust fast? There's no one single reason, but a combination of techniques like dictionary encoding [1], zone maps [2], late materialization [3], vectorization [4], and a gazillion others. It's not unlike all the countless optimization passes in a compiler.

None of the optimizations are groundbreaking - in fact, I could find a counterpart of every optimization I checked in ClickHouse as well! So why is CH still slower?

I believe the difference is in when they decide to apply each optimization. Unlike compiler optimizations which are usually beneficial, DB optimizations are highly data-dependent, and the query planner needs to turn them on an off based on data statistics. For example, CH is more conservative in applying dictionary encoding, and it basically doesn't get used at all for ClickBench.

Turning off dictionary-related optimizations in pgrust immediately made it 5x slower (from 3.8s to 20.4s on my machine), while ClickHouse takes a total of 15.0s.

This points to one important nuance in benchmarking - when all you target is one single benchmark, you can be more aggressive during query planning, as plans that work well for one query probably also work well for others, due to characteristics shared among queries/datasets. But when you build a system that needs to support millions of users, you have to be conservative with optimizations.

[1]: https://motherduck.com/glossary/dictionary-encoding/

[2]: https://motherduck.com/glossary/zone-map/

[3]: https://arrow.apache.org/blog/2025/12/11/parquet-late-materi...

[4]: https://clickhouse.com/resources/engineering/vectorized-quer...

To turn off dictionary-related optimizations in pgrust, I used the following setting:

  SET pgrust.gravity_road_off = 'dict.entries,percode.table,percode.length,verdict.coderange,
    code.carry,code.render,code.front_part,code.front_dense,code.front_light,code.sparse,
    code.cut,code.heads,key.codeif,prefetch.fp,text.frame';