Comment by chipdart
2 years ago
> Turns out there was zero difference, or negligible (Windows won), between compilation times.
I think there was something seriously flawed in your test. If you Google for a minute, you find multiple posts on how moving the same builds to Linux led to performance improvements in the range of 40% drops in build times.
Some anecdotes even compare doing the same builds in Ubunto with NTFS to see double-digit gains.
NTFS is notoriously awful in scenarios involving reading/writing many small projects. This is the bottleneck in Windows builds. There is a myriad of benchmarks documenting this problem.
Nowadays there are plenty of cross-platform projects to serve as benchmarks. Checking this can be as easy as checking out a project, start a full rebuild, and check how long it takes.
NTFS can be especially awful if you're used to Linux behaviour, port code over to Windows and expect the same performance.
Here's a talk on porting rustup to Windows: https://www.youtube.com/watch?v=qbKGw8MQ0i8
To begin with, it takes rustup 3m30s to install on Windows. After rejigging the code again and again away from its naive implementation which works fine on Linux, to perform sympathetically towards NTFS, it takes the same rustup 14s to install. That's quite a performance gain! But it needed a lot of changes to rustup, and to Windows itself.
Yeah you can work around the filesystem perf problems on Windows to some extend. Unity builds are not just popular because it reduces duplicated compiler work but also because it reduces filesystem access. There is a reason why Windows games tend to pack their assets into archives.
That's survivor bias. Fewer people blog about unsuccessful initiatives.
> That's survivor bias. Fewer people blog about unsuccessful initiatives.
I don't understand what point you're trying to make. Should we give equal credit to unsuccessful initiatives when their failure is due to screwing up a critical part of the project?
I mean, the successful attempts document what is possible. What do you expect to report when you couldn't even manage to get something working?
Say there are N initiatives to move to cross-compilation. Of those, there are M that deliver the hoped-for benefits, where M <= N. But practically, M < N significantly, say, M < 0.9 N.
Assume bloggers blog mainly about content that contains a positive message. I'm asserting that people blog more readily about their success than their failures.
So when you look at the blog literature, your population is not N, it is M. You don't see the failures because they don't tell the tale.
It seems possible that their pipeline just had large files, or something like that.
Java/dotnet pipelines don't have Windows penalty. Both have dependencies as compiled modules zipped into large jar/dll. While even simple node project have 10s of thousands of files inside node_modules folder.