Comment by jesprenj
3 years ago
I do not know Go. But what if there are so many high level abstractions in the Go language that it operates on streams directly?
3 years ago
I do not know Go. But what if there are so many high level abstractions in the Go language that it operates on streams directly?
The standard convention is to slap bufio.Reader/bufio.Writer on streams to make them more performant.
Though how LFS ends up with ~50 byte chunks is probably something very, very, dumb in the LFS code itself. Better to fix that mistake than to paper over it.
bufio is for adding buffering regardless of source/dest. Better in this case is ReaderFrom (which will also be used transparently by io.Copy) to let the socket control the buffering and apply even more optimizations. For something like git-lfs I could expect sendfile to provide a huge improvement, depending on the underlying storage.