Comment by zbentley
3 hours ago
I don’t think that extreme is compatible with the reporting/analytics use case of SQL DBs. Even though entire roles/companies may never touch that kind of SQL, there’s a massive quantity of it out there.
I once worked on a medical records system (with a pretty well designed but necessarily complex schema) where the primary “patient” data object used by most code was fetched by a query that, depending on what associated data you needed, had between 106 and more than 400 relations (across dozens to hundreds of tables) joined together.
And that was CRUDy data-path code. The OLAP/reporting side added zeros to those numbers. Query texts were often hundreds of kilobytes.
Could the same queries be 'planned', compiled down to pipelined KV operations, by the requester? I don't see that this is inherently less capable. You could even use an existing ORM – though I think you can do better when not compiling to something declarative, maybe more like polars.
I feel like databases effectively (/literally) add a JIT, which can mostly figure out what to do, even has accurate heuristics on the distribution of the data, but in exchange you get a less deterministic system, and less intuition for how to query or structure things. It's like, you know when to use a list/map/queue, but you want to focus on the business logic, so just use a smart collections which guess at runtime.
I think you can get this with FoundationDB, I should experiment rather than hypothesizing, but it feels like it should be better