| |||||||||||||
Subversion considered obsoleteSubversion considered obsoletePosted Apr 4, 2010 9:34 UTC (Sun) by epa (subscriber, #39769)In reply to: Subversion considered obsolete by wahern Parent article: A proposed Subversion vision and roadmap
I think you are right that because a git clone includes all history, it doesn't work well for really huge repositories. I did see a project (don't remeber its name) that let you use git in a more svn-like style, creating 'working copies' which do not include their own history, and committing from there back to a different repository.
(Log in to post comments)
Subversion considered obsolete Posted Apr 4, 2010 11:16 UTC (Sun) by peschmae (guest, #32292) [Link]
First of all you don't have to clone the entire history (see git-clone --depth)
Secondly the space used by git for storing the entire history is, in most cases, less than the space used by a working copy. i.e. for my linux kernel clone (with complete history going back 2.6.12) currently 400 MB for the .git directory as opposed to 450 MB for the actual checkout. Not really much of an issue in practice
Subversion considered obsolete Posted Apr 4, 2010 17:25 UTC (Sun) by RCL (guest, #63264) [Link]
Now try using gamedev repositories full of binary files (where code is a
tiny fraction of overall repo) and ... git can't even handle that (see my thread of comments below).
SHA calculation is what kills it. As I wrote, I didn't succeed in creating
Versioning really-big files Posted Apr 4, 2010 20:33 UTC (Sun) by smurf (subscriber, #17840) [Link]
Hmm. I can think of a simple way to fix the SHA1 problem (hash all (blockno-contents-of-block) tuples separately and XOR the result, or whatever; needs editor support to be effective).
The larger problem, however, is that you want a way to carry multiple versions of slowly-changing multi-GB files in your repo -- without paying the storage price of (a compressed version of) the whole blob, each time you check in a single-byte change. Same for network traffic when sending that change to a remote repository.
This is essentially a solved problem (rsync does it all the time) and just needs integration into the VCS-of-the-day. This problem is quite orthogonal to the question of whether said VCS-of-the-day is distributed or central, or whether it is named git or hg or bzr or whatever.
Yes, I know that the SVN people seem to have gotten this one mostly-right ("mostly" because their copy of the original file is not compressed). Hopefully, somebody will do the work for git or hg or whatever. It's not exactly rocket science.
Versioning really-big (binary) files Posted Apr 6, 2010 18:47 UTC (Tue) by vonbrand (subscriber, #4458) [Link]
Versioning really-big (binary) files Posted Apr 6, 2010 23:12 UTC (Tue) by dlang (✭ supporter ✭, #313) [Link]
the real problem is that in many cases when people say 'huge binary file that changes a bit' they really mean 'huge binary file where the meaning changes a little, but the actual file contents change a lot', usually due to a compression algorithm being used
even for images and audio, if you were to check them in uncompressed the git delta functionality would work well and diff the files against each other, but if you compress the file (jpeg, mp3, or even png) before checking it in, a small change to the uncompressed data results in a huge change to the compressed data. If it's a lossless compression (i.e. png) then it would be possible to have git uncompress it before checking for differences, but if it's a lossy compression you can't do this.
Versioning really-big (binary) files Posted Apr 7, 2010 7:53 UTC (Wed) by paulj (subscriber, #341) [Link]
The real problem is people thinking such files are suitable for checking
into an SCM. Just archive them somewhere.
Versioning really-big (binary) files Posted Apr 12, 2010 1:14 UTC (Mon) by vonbrand (subscriber, #4458) [Link] Not really. If the contents needs version control, it should be handled by a VCS. The size or format of the files could be a technical hurdle, sure; but it shouldn't be an excuse for not solving the problem.
Subversion considered obsolete Posted Apr 4, 2010 20:55 UTC (Sun) by simlo (subscriber, #10866) [Link]
Now you should not put (large) binaries into a VCS. VCS is for source code, not any form of compilation output. What you store in the VCS is a script which can pull the right, uniqely defined (version, sha1, etc.) binary from some server, or a build script, which produces the binary.
Where I work, we use subversion (and I use git svn :-). We have scripts which pulls the tar.gz files for various packages in specific versions from a server, unpacks, patches and crosscompiles them to our target. The only thing we have in subversion is the scripts and the files we have changed.
For the Linux kernel we tried to have the full thing in subversion, but it took way too much for subversion, so now we only have a makefile, which clones a git repository, when the source is needed.
Subversion considered obsolete Posted Apr 5, 2010 3:12 UTC (Mon) by RCL (guest, #63264) [Link]
Please understand that some software products rely on large amounts of data which is essentially a part of program and should be versioned together with source code.
It's almost like having very large firmware blobs in Linux kernels, much larger than they are currently...
See comments below where I elaborate on this interdependency between code and data in games.
Subversion considered obsolete Posted Apr 5, 2010 14:00 UTC (Mon) by simlo (subscriber, #10866) [Link]
It depends on wether you edit those data or not. Firmware files you don't edit. You grap a specific version, someone compiled for you, labeled and released.
On the other hand you can have data like maps and icons, which is also "source code" and belongs in the VCS if you edit them by using some program (some map editor, GIMP or whatever). But the overall system is badly designed if these files are "large". They ought to be seperated into small files, each containing seperate parts of the information and then "compiled" into larger files. This will usually make a more flexible and maintainable system (besides making life easier for the VCS). It is the same with C code: You don't make one big file but smaller ones, seperated by functionality.
Subversion considered obsolete Posted Apr 5, 2010 3:22 UTC (Mon) by martinfick (subscriber, #4455) [Link]
"We can do anything better than you"..."not X"..."you shouldn't do X", hmmm?
Subversion considered obsolete Posted Apr 8, 2010 10:08 UTC (Thu) by epa (subscriber, #39769) [Link]
It's remarkable how quickly 'my preferred tool is unable to do X' turns into the moral imperative 'you should not do X'.
Subversion considered obsolete Posted Apr 5, 2010 12:12 UTC (Mon) by peschmae (guest, #32292) [Link]
I agree, that in these cases (blobs that you'd like to have versioned along with your source code) git may not be the right choice for you. Still it does very well for large source code repositories.
Looks like you're stuck with perforce :-p
Subversion considered obsolete Posted Apr 5, 2010 17:21 UTC (Mon) by chad.netzer (✭ supporter ✭, #4257) [Link]
"git clone --depth" has some severe drawbacks, in that you cannot push or fetch from it. So any commits you make have to be made into patches and mailed to someone with a full repo (ie, a team of developers still all need full repos to work effectively with each other).
But yes, in practice, git stores source code repos very compactly, and by doing branch operations in place (rather than using separate directory copies for each "branch"), it uses much less space per client than SVN checkouts for a busy developer. And its also *much* faster for the same reason.
|
|||||||||||||