← Back to context

Comment by gethly

3 months ago

There is https://github.com/seaweedfs/seaweedfs

I haver not used it but will be likely a good minio alternative for people who want to run a server and don't use minio just as s3 client.

15 comments

gethly

Reply
chrislusf  3 months ago

This is Chris and I am the creator of SeaweedFS. I am starting to work full time on SeaweedFS now. Just create issues on SeaweedFS if any.

Recently SeaweedFS is moving fast and added a lot more features, such as: * Server Side Encryption: SSE-S3, SSE-KMS, SSE-C * Object Versioning * Object Lock & Retention * IAM integration * a lot of integration tests

Also, SeaweedFS performance is the best in almost all categories in a user's test https://www.repoflow.io/blog/benchmarking-self-hosted-s3-com... And after that, there is a recent architectural change that increases performance even more, with write latency reduced by 30%.

  • written-beyond  3 months ago

    Congratulations on earning that opportunity!

    Thank you for your work. I was in a position where I had to choose between minio and seaweed FS and though seaweed FS was better in every way the lack of an includes dashboard or UI accessibility was a huge factor for me back then. I don't expect or even want you to make any roadmap changes but just wanted to let you know of a possible pain point.

lima  3 months ago

Is it stable now? Last time I checked, the amount of correctness bugs being fixed in the Git history wasn't very confidence-inspiring.

  • rednb  3 months ago

    Since storage is a critical component, I closely watched it and engaged with the project for about 2 years circa as i contemplated adding it to our project, but the project is still immature from a reliability perspective in my opinion.

    No test suite, plenty of regressions, and data loss bugs on core code paths that should have been battled tested after so many years. There are many moving parts, which is both its strength and its weakness as anything can break - and does break. Even Erasure Coding/Decoding has had problems, but a guy from Proton has contributed a lot of fixes in this area lately.

    One of the big positive in my opinion, is the maintainer. He is an extremely friendly and responsive gentleman. Seaweedfs is also the most lightweight storage system you can find, and it is extremely easy to set up, and can run on servers with very little hardware resources.

    Many people are happy with it, but you'd better be ready to understand their file format to fix corruption issues by hand. As far as i am concerned, i realized that after watching all these bugs, the idea of using seaweedfs was causing me more anxiety than peace of mind. Since we didn't need to store billions of files yet, not even millions, we went with creating a file storage API in ASP.NET Core in 1 or 2 hours, hosted on a VPS, that we can replicate using rsync without problem. Since i made this decision, i have peace of mind and no longer think about my storage system. Simplicity is often better, and OSes have long been optimized to cache and serve files natively.

    If you are not interested in contributing fixes and digging into the file format when a problem occurs, and if your data is important to you, unless you operate at the billions of files scalability tier Seaweedfs shines at, i'd suggest rolling your own boring storage system.

  • yahooguntu  3 months ago

    We're in the process of moving to it, and it does seem to have a lot of small bugfixes flying around, but the maintainer is EXTREMELY responsive. I think we'll just end up doing a bit of testing before upgrading to newer versions.

    For our use case (3 nodes, 61TB of NVMe) it seems like the best option out of what I looked at (GarageFS, JuiceFS, Ceph). If we had 5+ nodes I'd probably have gone with Ceph though.

nodesocket  3 months ago

I'm looking at deploying SeaWeedFS but the problem is cloud block storage costs. I need 3-4TB and Vultr costs $62.50/mo for 2.5TB. DigitalOcean $300/mo for 3TB. AWS using legacy magnetic EBS storage $150/mo... GCP persistent disk standard $120/mo.

Any alternatives besides racking own servers?

*EDIT* Did a little ChatGPT and it recommended tiny t4g.micro then use EBS of type cold HDD (sc1). Not gonna be fast, but for offsite backup will probably do the trick.

  • xyzzy123  3 months ago

    I'm confused why you would want to turn an expensive thing (cloud block storage) into a cheaper thing (cloud object storage) with worse durability in a way that is more effort to run?

    I'm not saying it's wrong since I don't know what it's for, I'm just wondering what the use-case could be.

    • nodesocket  3 months ago

      I've quickly come to this conclusion. Essentially looking for offsite backup of my NAS and currently paying around $15-$20/mo to Backblaze. I thought I might be able to roll my own object store for cheaper but that was idiotic. :-)

      1 reply →

hobofan  3 months ago

SeaweedFS has been our choice as a replacement for both local development and usage in our semi-ephemeral testing k8s cluster (both for its S3 interface). The switch went very smooth.

I can't really say anything about advanced features or operational stability though.

lionkor  3 months ago

Sadly there's nothing in the license of seaweedFS that would stop the maintainer from pulling a minio -- and this time without breaking the (at least spirit of the) terms of the project's license.

Not an issue at all until they do.