Comment by etler

9 months ago

I've lost count of how many "Migrating from X to Postgres" articles I've seen.

I don't think I've once seen a migrating away from Postgres article.

20 comments

etler

Reply
delish  9 months ago

Related: Oxide's podcast, "Whither CockroachDB," which reflects on experience with postgres at Joyent, then the choice to use cockroach in response to prior experiences with postgres.

https://www.youtube.com/watch?v=DNHMYp8M40k

I'm trying to avoid editorializing in my above summary, for fear of mischaracterizing their opinions or the current state of postgres. Their use of postgres was 10 years ago, they were using postgres for a high-availability use case -- so they (and I) don't think "postgres bad, cockroach good." But like Bryan Cantrill says, "No one cares about your workload like you do." So benchmark! Don't make technical decisions via "vibes!"

yakkomajuri  9 months ago

I think your point still stands, and I'm a big Postgres advocate/user myself btw.

But yeah we did migrate our _analytics_ data to ClickHouse (while still keeping Postgres for more transactional stuff) back when I was at PostHog.

Writeup: https://posthog.com/blog/how-we-turned-clickhouse-into-our-e...

  • mplanchard  9 months ago

    We also did this, using change data capture and kafka to stream data to clickhouse as it gets written to postgres.

    Clickhouse is incredible tech. We’ve been very pleased with it for OLAP queries, and it’s taken a lot of load off the postgres instance, so it can more easily handle the very high write load it gets subjected to.

evanelias  9 months ago

Not an article, and I have no direct knowledge of this either way, but I would strongly suspect that Instagram migrated off Postgres a while back. Probably to fb-mysql + myrocks, or some other RocksDB based solution.

The compression level is vastly superior to any available Postgres-based solution, and at Instagram’s scale it amounts to extremely compelling hardware cost savings.

Also if they were still primarily on pg, it would be one of the largest pg deployments in existence, and there would be obvious signs of the eng impact of that (conference talks, FOSS contributions, etc).

Bigger-picture: Postgres is an amazing database, and it’s often the right choice, but nothing in tech is always the best choice 100% of the time. There’s always trade-offs somewhere.

rakejake  9 months ago

Probably a corollary of the fact that most usecases can be served by an RDBMS running on a decently specced machine, or on different machines by sharding intelligently. The number of usecases for actual distributed DBs and transactions is probably not that high.

notTooFarGone  9 months ago

We migrated from postgres to ADX based on cost analysis done of the managed version on Azure.

Now we have lovely kql queries and pretty much start new with postgres again...