Comment by cortesoft
3 months ago
We use it because we are already running our own k8s clusters in our datacenters, and we have large storage requirements for tools that have native S3 integration, and running our own minio clusters in the same datacenter as the tools that generate and consume that data is a lot faster and cheaper than using S3.
For example, we were running a 20 node k8s cluster for our Cortex (distributed Prometheus) install, monitoring about 30k servers around the world, and it was generating a bit over a TB of data a day. It was a lot more cost effective and performant to create a minio cluster for that data than to use S3.
Also, you can get durability with minio with multi cluster replication.
Consider migrating to VictoriaMetrics and saving on storage costs and operations costs. You also won't need MinIO, since it stores data to local filesystem (aka to regular persistent volumes). See real-world reports from happy users who saved costs on a large-scale Prometheus-compatible monitoring - https://docs.victoriametrics.com/victoriametrics/casestudies...
I can't imagine switching at this point. We spent quite a while building up our Cortex and Minio infrastructure management, as well as our alerting and inventory management systems, and it is all very stable right now. We don't really touch it anymore, it just hums along.
We have already worked through all the pain points and it all works smoothly. No reason to change something that isn't a problem.