Comment by hobofan
15 hours ago
Datafusion has all of the individual pieces, especially if you just want to data analysis, but is far from an embedded database.
Indexes, transactions, a first-class storage format are all things that come included with DuckDB, that you won't have with Datafusion.
indexes are very weak in duckdb, need to fit memory.
This fall, DuckDB will support larger than memory indexes! I'm really excited about it. https://duckdb.org/2026/08/17/duckdb-20-highlights#7-storage...
they don't say "this fall", they say later this year, I think it is some parallel effort, and it is actually going for few years already (rewriting index logic to have them in mmapped buffers and not in memory), finger crossed they will finish it finally.