Comment by setquk
7 years ago
Have to agree on all points.
You’ll find the slow file operations are entirely down to NTFS. Regardless of how you play around in fsutil it’s hopeless on lots of small files. This incidentally makes WSL unbearably slow. Hence why I still use VirtualBox and putty.
I was suspecting as much. I wonder why NTFS gets so much praise, especially compared to HFS+. Yes, HFS has less features, but at least it lets me get my work done.
NTFS deserves no praise. None at all. Not sure I've ever heard any to be honest.
edit: fine example as I got a downvote.
SVN checkout on windows NTFS 8 minutes 30 seconds
SVN checkout on same kit on ext4 48 seconds.
This problem scales to git as well, so is not SVN related. Anything that does lots of small writes, so basically anything unixy, suffers like this.
NTFS was much better than FAT, and was one of the first journaled filesystems in widespread use. It was praised perhaps 20 years ago.
2 replies →
> Anything that does lots of small writes, so basically anything unixy, suffers like this.
To be honest, when NTFS was conceived it probably didn't have as a design goal that common Unix patterns should be fast. After all, Unix applications use the file system for lots of things where Windows has different mechanisms for that purpose.
SVN sadly has to traverse the complete working copy and lock every single directory individually because every directory is also a working copy on its own. Most of the time SVN spends on, e.g., update, is spent on locking and unlocking. Git/hg only need to do this in one place and avoid that problem.
Never checked if it was NTFS issue, but it felt like logical explanation - when starting Stronghold Crusader through wine on Linux, loading took like 0.5s, compared to 10s or so on Windows.
Didn't bother to check the numbers, but it was order of magnitude for sure. Way more than just noticeable.
^ This is true. Dealing with lots of small files is SUPER SLOW in NTFS for some reason.
It's definitely not 'entirely'. I have a folder with a bunch of small files where explorer and other programs sometimes hitch for tens of minutes, and WSL can reliably process in seconds. It's too many seconds compared to native linux, but it's still seconds.
It's not NTFS -- it's Windows Defender's Realtime protection.
If you turn it off it still runs like ass. I have spent hours on this one even down to fsutil tuning and I can't get more than about a 10% improvement.
This is because the small files are stored on the MFT which is global read/write locked.