Comment by hu3
1 day ago
From what I understand they basically couldn't scale writes in PostgreSQL to their needs and had to offload what they could to Azure's NoSQL database.
I wonder, is there another popular OLTP database solution that does this better?
> For write traffic, we’ve migrated shardable, write-heavy workloads to sharded systems such as Azure CosmosDB.
> Although PostgreSQL scales well for our read-heavy workloads, we still encounter challenges during periods of high write traffic. This is largely due to PostgreSQL’s multiversion concurrency control (MVCC) implementation, which makes it less efficient for write-heavy workloads. For example, when a query updates a tuple or even a single field, the entire row is copied to create a new version. Under heavy write loads, this results in significant write amplification. It also increases read amplification, since queries must scan through multiple tuple versions (dead tuples) to retrieve the latest one. MVCC introduces additional challenges such as table and index bloat, increased index maintenance overhead, and complex autovacuum tuning.
I was thinking about the same paragraph because write-amplification is exactly the problem solved by LSM trees _and_ they already have a solution for that in-house - one of the first acquisitions that OpenAI made is Rockset - a company that actually built the RocksDb at scale.
So, this is the part that actually made me left wondering why.
Tidb should handle it nice. I've wrote 200к inserts / sec for hour in peak. Underlying lsm works better for writes
That would mean it improved somewhat. We always got better write performance from mysql vs postgres, however that is a while ago; we then tried tidb to go further but it was basically rather slow. Again, a while ago.
When did you get your results, might be time to re-evaluate.
It was 1 year ago. Around 15 tikv serves, 32 cpu, 128 ram each, 4 tb nvme. In this case latency matters a lot. When i had load server in different region with ping of 3ms I got 70k inserts, when moved to the same region with sub ms ping it went to thousands