Comment by lamontcg
19 hours ago
The biggest thing that gems could do to make rubygems faster is to have a registry/database of files for each gem, so that rubygems didn't have to scan the filesystem on every `require` looking for which gem had which file in it.
That would mean that if you edited your gems directly, things would break. Add a file, and it wouldn't get found until the metadata got rehashed. The gem install, uninstall, etc commands would need to be modified to maintain that metadata. But really, you shouldn't be hacking up your gem library like that ith shellcommands anyway (and if you are doing manual surgery, having to regen the metadata isn't really that burdensome).
I wrote some code to do almost this many years ago (if I recall correctly, it doesn’t cache anything to disk, but builds the hash fresh each time, which can still result in massive speed up).
Probably obsolete and broken by now, but one of my favorite mini projects.
(And I just realized the graph is all but impossible to read in dark mode)
https://github.com/pmahoney/fastup
100%. Optimising "bundle install" etc. is optimizing the wrong thing. You don't even need this to work from gems in general. It'd have solved a lot of problems just to have it work for "bundle install" in standalone mode, where all the files are installed to a directory anyway.
But in general, one of the biggest problems with Ruby for me is how $LOAD_PATH causes combinatoric explosion when you add gems because every gem is added, due to the lack of any scoping of require's to packages.
The existence of multiple projects to cache this is an illustration that this is a real issue. I've had project in the past where starting the app took minutes purely due to require's, and where we shaved minutes off by crude manipulation of the load path, as most of that time was pointless stat calls.
At runtime rather than install time yes. I did some prototyping on this back in the day, one of the issues is the language lacks an efficient data structure to store that information and you can’t (easily) build one efficiently because instances are too heavy.
You are describing bootsnap.
And yes I proposed to integrate bootsnap into bundler ages ago, but got told to go away.
Perhaps its time to try again - bootsnap is definitely stable enough now, which it really wasn't early on.