In this workload, it happened to be, in a lot of workloads it is not at all. This is quite likely because over time media sizes have increased greatly. Microsoft research on operating systems shows that contiguous chunks at 64MB don't really show any IOPS penalty in practical usage scenarios.
I work on workloads that work with thousands to millions of small files. 1k to 1MB. I can promise you that the IO effect is massive, with shortcuts like working directly on archive files/zips where they are read and written at once saves massively on hardware costs as IOPS tend to be expensive. This is with fast NVMe disks, no spinning platters involved. Also small files massively increase write amplification on SSDs.
There probably is significant savings in deduping 'tiny' files, but the moment you get past a particular threshold it drops off, and the vast majority of game assets have increased over that size.
You may misunderstand how poor hdd IOPs are and how much latency 4k random reads induce.
Random IO can be 100 to 200 times slower than sequential reads on small file workloads.
It literally says in the article the effect was inconsequential
> small number of players (11% last week) using mechanical hard drives will witness mission load times increase by only a few seconds in worst cases.
In this workload, it happened to be, in a lot of workloads it is not at all. This is quite likely because over time media sizes have increased greatly. Microsoft research on operating systems shows that contiguous chunks at 64MB don't really show any IOPS penalty in practical usage scenarios.
I work on workloads that work with thousands to millions of small files. 1k to 1MB. I can promise you that the IO effect is massive, with shortcuts like working directly on archive files/zips where they are read and written at once saves massively on hardware costs as IOPS tend to be expensive. This is with fast NVMe disks, no spinning platters involved. Also small files massively increase write amplification on SSDs.
There probably is significant savings in deduping 'tiny' files, but the moment you get past a particular threshold it drops off, and the vast majority of game assets have increased over that size.