Comment by remram

16 hours ago

Bit of a tangent, but what I'm looking for is a S3-compatible server with transparent storage, ie storing each file (object) as an individual file on disk.

Minio used to do that but changed many years ago. Production-grade systems don't do that, for good reason. The only tool I've found is Rclone but it's not really meant to be exposed as a service.

Anyone knows of an option?

3 comments

remram

Reply
whispem  11 hours ago

minikv actually supports a fully S3-compatible API (PUT/GET/BATCH, including TTL extensions and real-time notifications). By default, the storage engine is segmented/append-only with object records in blob files, not “one file per object”. However, you can configure a backend (like the in-memory mode for dev/test, or Sled/RocksDB) and get predictable, transparent storage behavior for objects. Storing each object as an individual file isn’t the default — for durability and atomics, objects are grouped inside segment files to enable fast compaction, consistent snapshots, and better I/O performance.

If you need “one file per object” for a specific workflow, it’s possible to add a custom backend or tweak volume logic — but as you noted, most production systems move away from that model for robustness. That said, minikv’s flexible storage API makes experimentation possible if that’s what your use-case demands and you’re fine with the trade-offs.

Let me know what your usage scenario is, and I can advise on config or feature options!