Comment by goerch
10 hours ago
Theoretically I would agree, but practically I still wonder why we need different database engines for row and columnar storage if supporting different types of indices is trivial(TM) for Postgres?
10 hours ago
Theoretically I would agree, but practically I still wonder why we need different database engines for row and columnar storage if supporting different types of indices is trivial(TM) for Postgres?
There are definitely hybrid OLTP-OLAP databases, or HTAP (Hybrid Transaction and Analytical Processing). Microsoft and, I believe, Oracle both have HTAP tech.
The most novel design, I think, is CedarDB (developed by Thomas Neumann's database group at TUM), which adaptively stores both row and column versions of the data [1], where some data is permanently compressed to columnar format and hot rows are converted "just in time" to columnar data as needed.
[1] https://cedardb.com/blog/colibri/
In theory, you don't. In practice, it's because the major SQL DBMS were architected around row-oriented storage and the technical effort to implement hybrid storage is large.
There are columnar storage engine extensions for many of the popular databases, though.
Interesting: I transferred the idea of a matrix being stored row or column wise to the database world and assumed this was a more physical than theoretical feature (not a native speaker here)?
Looking forward to check out `pg_duckdb`, yes.
there are hybrid engines too, so-called HTAP (as opposed to OLAP or OLTP) notable efforts are: SingleStore (commercial) and OceanBase (foss)
Microsoft SQL Server can use both row and column store for tables, in various combinations such as row store for the table with a columnar index, or vice versa.