Comment by franciscop
10 hours ago
> PEP 658 (2022) put package metadata directly in the Simple Repository API, so resolvers could fetch dependency information without downloading wheels at all.
> Fortunately RubyGems.org already provides the same information about gems.
> [...]
> After we unpack the gem, we can discover whether the gem is a native extension or not.
Why not adding the meta information of whether the gem is a native extension or not directly to rubygems.org? You could fully parallelize whole installation trees of dependencies then.
Had the same thought reading this but I suspect what's in the gemspec could accidentally differ from what's in the RubyGems.org metadata, although that should probably not be possible.
From working on RubyGems.org a long time ago I vaguely remember that the metadata extracted from the gemspec is version-specific. So if you add a new native_extension boolean you'd have to artificially reprocess those previously published gemspecs to change the metadata for all past versions.
Being able to mutate metadata for past versions is dangerous enough that I'd be surprised it's allowed or even possible. So that might not even be something Aaron considered here for that reason. That said, it seems reasonable to me to suggest this improvement going forward to make unpacking the gem unnecessary to know whether it'll affect installation order.
Just make the rule apply only to packages published after a given date, and then manually backfill that metadata into the service-backend DB with a one-time scrape through all packages from before that date.