Comment by senderista
7 hours ago
"Speed" is also ambiguous between latency and throughput. You seem to be using "speed" here as a synonym for throughput. Because of Little's Law, the memory consumed by deallocated objects is directly proportional to deallocation latency, so "low footprint" also generally means "low latency", while increasing throughput by amortizing deallocation overhead at the expense of latency increases memory usage for the same reason.
> so "low footprint" also generally means "low latency"
Not anymore.
You're absolutely right that one of the reasons moving collectors were not used more widely was that, while their throughput was always very impressive, their latency wasn't that great, but that changed a few years ago.
E.g. Generational ZGC in OpenJDK (released in September '23) introduces hiccups or "pauses" that are not dependent on the size of the liveset and are no larger than latency hiccups introduced by the OS (assuming no realtime kernel), i.e. <1ms (and typically <<1ms) up to heaps of 16TB. In fact, the latency can be smoother than approaches that have an explicit free operation and require maintaining a free list, as freeing a large object graph can be quite slow and occur in surprising places.
So modern moving GCs no longer have a latency penalty, but this is newer than even ChatGPT.
Where could one find resources about implementing such GCs in their own language?
https://www.taylorfrancis.com/books/mono/10.1201/97810035953...
Note that this is a pretty new technology. The first production-quality "pauseless" moving collector for commodity hardware was released in 2010 by Azul, but it was proprietary. The first open source implementation (that was also generational) was in JDK 21 (https://openjdk.org/jeps/439), i.e. it's less than three years old. The book I linked to is by one of the primary designers of that GC (and one of the world's foremost experts on memory management).
ZGC does no work in stop-the-world pauses; no marking, no compacting, not even root scanning.
Of course, if your language targets the JVM, it will automatically get to enjoy that amazing GC.