Comment by bencornia

1 day ago

> Grab’s engineering team went from 18 minutes for go get to 12 seconds after deploying a module proxy. That’s not a typo. Eighteen minutes down to twelve seconds.

> The problem was that go get needed to fetch each dependency’s source code just to read its go.mod file and resolve transitive dependencies. Cloning entire repositories to get a single file.

I have also had inconsistent performance with go get. Never enough to look closely at it. I wonder if I was running into the same issue?

> needed to fetch each dependency’s source code just to read its go.mod file and resolve transitive dependencies.

Python used to have this problem as well (technically still does, but a large majority of things are available as a wheel and PyPI generally publishes a separate .metadata file for those wheels), but at least it was only a question of downloading and unpacking an archive file, not cloning an entire repo. Sheesh.

Why would Go need to do that, though? Isn't the go.mod file in a specific place relative to the package root in the repo?

  • Go's lock files arrived at around the same time as the proxy, before then you didn't have transitive dependencies pre baked.