Comment by a_t48
1 day ago
It was half a day of work to make a drop in action.yml that does this. Saved a bunch of money (both in bandwidth and builder minutes), well worth the investment. It really wasn’t a lot of customization.
All our builds are on GHA definitions, there’s no way it’s worth it to swap us over to another build system, administer it, etc. Our team is small (two at the time, but hopefully doubling soon!), and there’s barely a dozen people in the whole engineering org. The next hit list item is to move from GH hosted builders to GCE workers to get a warmer docker cache (a bunch of our build time is spent pulling images that haven’t changed) - it will also save a chunk of change (GCE workers are 4x cheaper per minute and the caching will make for faster builds), but the opportunity cost for me tackling that is quite high.
Ah interesting, I was just curious. I’ve wasted some time setting up ci runners stuff on bare metal servers just because I’ve heard runners from gitlab/github can be expensive
Doesn’t the official docker build push action support caching with the GitHub Actions cache?
Yes but one image push for us is >10GB, due to ML dependencies. And even if it is intelligent and supports per layer caching, you can’t share between release branches - https://github.com/docker/build-push-action/issues/862.
And even if that did work, I’ve found it much more reliable to use the actual docker BuildX disk state than to try and get caching for complex multi stage builds working reliably. I have a case right now where there’s no combination of —cache-to/from flags that will give me a 100% cached rebuild starting from a fresh builder, using only remote cache. I should probably report it to the Docker team, but I don’t have a minimal repro right now and there’s a 10% chance it’s actually my fault.
You should try this with Depot [0]. I’m a founder of it and this is definitely one of the use cases we built it for. We persist your layer cache to a real NVMe device and reattach that automatically across builds. No more needing to save your layer cache via networks or resource constrained GitHub Actions cache.
[0] https://depot.dev