Comment by gschoeni

1 day ago

How big are your datasets? Working on an Open Source git-lfs replacement called "oxen" if you are interested.

https://github.com/Oxen-AI/Oxen

2 comments

gschoeni

Reply
GabeIsko  1 hour ago

Not working with datasets. Binary files aren't that large, and these tools are generally bad for my use case - because I am not concerned about datasets.

I need to track changes in binary files of very reasonable size. Total repo size is <1GB. But even at these small memory requirements it makes much more sense to self host with LFS. I have written this up too many times on the internet to go into great detail about how LFS isn't perfect and how I wish there was something better, but in practice it has worked extremely well for tracking a small amount of binary files. Kudos to the devs.