Comment by tudorg

2 days ago

> - Is ~300x faster than Postgres on analytical workloads. Right now it's 2x slower than Clickhouse on clickbench and I think it's possible to get faster than Clickhouse

That sounds like you are storing the data in a columnar format? Or do you do both row and columnar?

In a somewhat similar (yet also quite different) effort, I've been working on δx, a Postgres extension that compresses the data in a columnar format stored in normal Postgres tables (so replication, crash recovery, pg_dump, etc. still work normally). https://github.com/xataio/deltax

It is currently about 30-40% slower than ClickHouse (single node, ofc). The PR to add it to clickbench was just accepted, so you can see the comparison here: https://benchmark.clickhouse.com/#system=+liH|_etx|gQ|saB&ty...

6 comments

tudorg

Reply
malisper  2 days ago

Yep! The new version of pgrust supports batch based execution and a columnar format. I'm curious how you got δx to perform that well? From what I've seen a columnar layout only gets you part of the way and really good parallelism and really fast hash tables seem to make up a significant portion of why Clickhouse is faster.

  • tudorg  2 days ago

    Yeah, spent a lot of time on parallelism, vectorizing, pipelining, filter push-downs, bloom filters, all the tricks out there. It's really fun to make pretty steady progress on this.

  • FreakLegion  2 days ago

    pg_mooncake (now effectively abandoned due to being acquired by Databricks, but still up at https://github.com/Mooncake-Labs/pg_mooncake) pulled the DuckDB engine into Postgres wholesale, if I remember right.

    pg_lake also uses DuckDB but keeps it external, routing through Postgres and managing Iceberg tables (but not the data itself) there (https://github.com/Snowflake-Labs/pg_lake).

    Both of these were neck and neck with ClickHouse last time I tried them.