Comment by pjc50
19 hours ago
Completely different design. Git is intended to be fully distributed, so (a) every repo is supposed to have the full history of every file, and (b) locking is meaningless.
People should use the VCS that's appropriate for their project rather than insist on git everywhere.
> People should use the VCS that's appropriate for their project rather than insist on git everywhere.
A lot of people don't seem to realise this. I work in game dev and SVN or Perforce are far far better than Git for source control in this space.
In AA game dev a checkout (not the complete history, not the source art files) can easily get to 300GB of binary data. This is really pushing Subversion to it's limits.
In AAA gamedev you are looking at a full checkout of the latest assets (not the complete history, not the source art files) of at least 1TB and 2TB is becoming more and more common. The whole repo can easily come in at 100 TB. At this scale Perforce is really the only game in town (and they know this and charge through the nose for it).
In the movie industry you can multiply AAA gamedev by ~10.
Git has no hope of working at this scale as much as I'd like it to.
Perforce gets the job done but it's a major reason why build tooling is worse in games.
Github/gitlab is miles ahead of anything you can get with Perforce. People are not just pushing for git because they ux of it, they're pushing git so they can use the ecosystem.
What do you mean “build tooling is worse in games”? I’ve worked in games for 20 years and used both perforce and got extensively across small, medium, and large projects.
There’s nothing technically worse about perforce with regard to CI/CD, if that’s what you’re talking about, except that of course there are more options for git where you just click a button and enter your credentials and away you go. But that’s more a function of the fact that there aren’t as many options for perforce hosting and companies are more likely to host it internally themselves.
If companies are using perforce but aren’t running any CI/CD that’s because they’re lazy/small/don’t see the value.
I've been thinking of using git filter to split the huge asset files (that are just internally a collection of assets bundler to 200M-1GB files) into smaller ones. That way when artist modifies one sub-asset in a huge file only the small change is recorded in history. There is an example filter for doing this with zip files.
The above should work. But does git support multiple filters for a file? For example first the above asset split filter and then store the files in LFS which is another filter.
I mean it might work but you'll still get pull time-outs constantly with LFS. It's expensive to wait two or three days before you can start working on a project. Go way for two weeks, it will be a day before you can "pulled" up to date.
I hope this "new" system works but I think Perforce is safe for now.
> People should use the VCS that's appropriate for their project rather than insist on git everywhere.
Disagree. I really like the "de-facto standard" that git has become for open source. It means if I want to understand some new project's source code, there is one less hassle for me to deal with: I don't need to learn any new concepts just to access the source code and all the tooling is already right there.
The situation we have with package managers, dependency managers and package managers for package managers is worse enough. I really don't want a world in which every language or every project also comes with its own version control system and remote repo infrastructure.
A "proper" versioning system doesn't need to be learned since you literally only need a handful of straightforward operations (how do I get the latest version? how do I add a file? how do I commit changes?) - in svn that's 'svn update', 'svn add' and 'svn commit', that's all what's needed to get you through the day, no 'push', no 'staging area', no 'fetch' vs 'pull' and the inevitable merge-vs-rebase discussion... etc etc etc...)
It's only git which has this fractal feature set which requires expert knowledge to untangle.
But if all systems are so similar anyway, why would you need "the right tool for the job"?
If nothing else, you have to install it. There will also be subtle differences between concepts, e.g. git and svn both have versions and branches, but the concepts behave differently. I don't know about Mercurial, but I'm sure they have their own quirks as well.
Also, tooling: I have a VSCode plugin that visualizes the entire graph structure of a git repo really nicely. Right now, I can use that on 99% of all repos to get an overview of the branches, last commits, activity, etc.
If version systems were fragmented, I'd have to look for equivalent tools for every versioning system separately - if they exist at all. More likely, I'd be restricted just to the on-board tools of every system.
1 reply →
In the same way some might be discouraged from contributing to a project because they don't know the language well enough, I've given up on contributing to projects because I couldn't figure out mercurial, and I didn't care enough about the contribution to learn it.
To be clear, "fully distributed" also means "can use all of the features offline, and without needing commit access to the central repository".
I can't imagine living without that feature, but I also do a lot of OSS work so I'm probably biased.
How often are you fully offline though? A centralized version system could be updated to work in 'offline mode' by queueing pushed changes locally until internet is available again (and in SVN that would be quite trivial because it keeps the last known state of the repository in a local shadow copy anyway).
It's not terribly common, but more critically it means I can work even when the source forge is down. (And for corporate stuff it means I can work on stuff without connecting to the VPN.)
Also, designing around distribution meant that merges have to be fast and work well -- this is a problem that most centralised systems struggle with because it's not a key part of the workflow. Branching and merging are indispensable tools in version control and I'm shocked how long CVS and SVN survived despite having completely broken systems for one or both. Being able to do both (and do stuff like blames over the entire history) without needing to communicate with a server is something that changes the way you work.
My actual hot take (as a kernel developer) is that email patches are good, actually. I really wish more people were aware of how incredibly powerful they are -- I have yet to see a source forge that can get close to the resiliency and flexibility of email (though SourceHut gets very close). git-send-email has its quirks, but b4 honestly makes it incredibly easy.
(There's also the owning your own data benefits that I think often go overlooked -- reflog and local clones mean that you cannot practically lose data even if the server goes away or you get blocked. I suspect Russian or Iranian developers losing access to their full repo history because of US sanctions wouldn't share your views on centralised services.)
> by queueing pushed changes locally
And if you and another developer make conflicting changes while offline? What should happen when you return online?
1 reply →
All the time. If you use p4 and git you'll notice the difference just from trying to move clone git repos locally vs trying to make a new workspace.
> People should use the VCS that's appropriate for their project rather than insist on git everywhere.
But git is likely to be appropriate almost everywhere. You won’t just use svn just for big file purposes while git is better for everything else in the same project
The thing is, in a game project, easily 99% of all version controlled data is in large binary files, and text files are by far the minority (at least by size). Yet still people try to use git for version control in game projects just because it is the popular option elsewhere and git is all they know.
> text files are by far the minority (at least by size)
Well yeah because text files are small. Thinking text files are insignificant to games because they are small is a really dumb perspective.
> Yet still people try to use git for version control in game projects just because it is the popular option elsewhere and git is all they know.
Or perhaps it's because it works really well for text files, which are a significant part of most games, and because the tooling is much better than for other VCS's.
2 replies →
Not because it's popular, but because all the tooling (CI/CD) around git is the best.
In games a lot of the tooling assumes P4 so it's often a better choice, on the whole, but if git and LFS was as widely supported in art tooling it would be the clear choice.
> Git is intended to be fully distributed
Which is kinda funny because most people use git through Github or Gitlab, e.g. forcing git back into a centralized model ;)
> People should use the VCS that's appropriate for their project rather than insist on git everywhere.
Indeed, but I think that train has left long ago :)
I had to look it up after I wrote that paragraph about locking, but it looks like file locking is supported in Git (just weird that I need to press a button in the Gitlab UI to lock a file:
https://docs.gitlab.com/user/project/file_lock/
...and here it says it's only supported with git lfs (so still weird af):
https://docs.gitlab.com/topics/git/file_management/#file-loc...
...why not simply 'git lock [file]' and 'git push --locks' like it works with tags?
If you’re making local commits (or local branch, or local merge, etc), you’re leveraging the distributed nature of Git. With SVN all these actions required being done on the server. This meant if you were offline and were thinking of making a risky change you might want to back out of, there was no way on SVN to make an offline commit which you could revert to if you wanted.
Of course if you’re working with others you will want a central Git server you all synchronize local changes with. GitHub is just one of many server options.
Shortly after git became popular, the Subversion team released a tool that enabled replicating a SVN repository or any subset to a local independent repository, which could be used for local development and re-synced to the main repository at any time.
I'm pretty sure that if required, SVN could be updated to have an 'offline mode' where more offline operations would be allowed, since it keeps a local shadow copy of the last repository state from the server anyway. But offline mode simply isn't all that necessary anymore. How often is an average programmer in an area without any mobile coverage or other means of internet connectivity?
1 reply →
I very much dislike git (mostly because software hasn't been "just source" for many decades before git was hobbled together, and git's binary handling is a huge pain for me), but what does a lockfile achieve that a branch -> merge doesn't, practically, and even conceptually?
Binary files can't be meaningfully merged. E.g. if two people create separate branches and work on the same Photoshop file, good luck trying to merge those changes, you can at best pick one version and the other user just wasted a lot of time. A lock communicates to other users "hey I'm currently working on that file, don't touch it" - this also means that lockable files must be write-only all the time unless explicitly locked - so that people don't "accidentially" start working on the file without acquiring a lock first. Apparently (as far as I have gathered so far from googling) git-lfs supports file locking but not vanilla git.
7 replies →