Comment by ljosifov

Similar. Noticed DuckDB ever since an old article 'what db should I use' for local small data warehousing. The author was blown away that DuckDB seemed super naturally quick. It was I think columnar store + compression facilitated that? It made duckdb load compressed + on the fly decompress = faster than even reading the uncompressed data. Had forgotten most of it. But was used to mmap-ed files + columnar storing of Kdb. Was pleased that the author was clued to notice the power of that.

Then more recently I was given a somewhat random task to organise a motley collection of web scrapes, historic data, realtime data, data to be fetched on demand dispersed in semi-random collections. DuckDB as backing store + Claude Code that I discovered in Nov-2025 (with suitable skills and schemas) = a data agent where I could tell CC *in English* complicated data requests!? And CC would write glue shell and python code, write SQL and run it against DuckDB that housed most of the data, fetch new data, munge join filter, and present it to the user as "your data is in data slash blah". It seemed a miracle unfolded in front of my eyes! So yeah - fond of DuckDB. :-)

Latter I read this https://openai.com/index/inside-our-in-house-data-agent/ and thought "but of course".