Comment by liontwist
3 months ago
This sounds awful.
I would do almost any amount of iteration and index tuning to keep the query on a single machine rather than deal with a networked distributed system.
When you get slow queries the real problem is algorithmic complexity and linear workers only can do so much,
You can use duckdb on a single machine. It's also indexless (or more accurately, you don't have to explicitly create indexes)
You are rightfully proud of your skills!