Comment by HiPhish
16 hours ago
Have you considered using OverlayFS[1] instead of installing all files into the game directory and tracking them with a database? Or maybe what GNU Stow is doing where it installs each package into its own directory and then uses symlinks which it tracks to "install" the files into the global file hierarchy?
Ooh overlayfs is quite interesting. I'll have to take a closer look at that. I imagine it's similar to what MO2 was doing on windows with the virtual filesytem to keep the game directory clean.
The stow approach is something that I considered but ultimately rejected for a couple of reasons around handling conflicts of game-installed files as well as how to ultimately handle the symlink lifecycle (eg wrapper to make the "non-running" state always clean or to let it always persist and then need to run manual cleanup/update steps). But if you're interested in that approach when I was applying for Nexus Mods approval I discovered https://github.com/Marc1326/Anvil-Organizer in the overall list of mod tools which I believe uses that strategy (though I haven't really looked too closely)
But basically my original idea to just install the files directly into the game directory stems from the fact that when I switched to linux for gaming and not having success with MO2 that's literally what I was doing. I would download the mod from nexus and unzip/tar it into the game directory manually. When I wanted to uninstall or update I'd find the original archive list the files in it and then delete them from my game directory. After doing this too much I realized that I was basically missing the functionality of a standard linux package manager (eg apt, pacman, etc)
Keep in mind that overlay file systems are designed to treat the lower layer as read only. Changes made in the merged view are written to the upper layer while the original lower files are untouched.
So if you need to persist changes into the lower layers, I think you may need to do tricks like taking snapshots and then swapping the bind mount (maybe with some diffing logic) or some other offline methods.