Recent Features
| |||||||||||||
Subversion considered obsoleteSubversion considered obsoletePosted Apr 4, 2010 17:25 UTC (Sun) by RCL (guest, #63264)In reply to: Subversion considered obsolete by peschmae Parent article: A proposed Subversion vision and roadmap
Now try using gamedev repositories full of binary files (where code is a
tiny fraction of overall repo) and ... git can't even handle that (see my thread of comments below).
SHA calculation is what kills it. As I wrote, I didn't succeed in creating
(Log in to post comments)
Versioning really-big files Posted Apr 4, 2010 20:33 UTC (Sun) by smurf (subscriber, #17840) [Link]
Hmm. I can think of a simple way to fix the SHA1 problem (hash all (blockno-contents-of-block) tuples separately and XOR the result, or whatever; needs editor support to be effective).
The larger problem, however, is that you want a way to carry multiple versions of slowly-changing multi-GB files in your repo -- without paying the storage price of (a compressed version of) the whole blob, each time you check in a single-byte change. Same for network traffic when sending that change to a remote repository.
This is essentially a solved problem (rsync does it all the time) and just needs integration into the VCS-of-the-day. This problem is quite orthogonal to the question of whether said VCS-of-the-day is distributed or central, or whether it is named git or hg or bzr or whatever.
Yes, I know that the SVN people seem to have gotten this one mostly-right ("mostly" because their copy of the original file is not compressed). Hopefully, somebody will do the work for git or hg or whatever. It's not exactly rocket science.
Versioning really-big (binary) files Posted Apr 6, 2010 18:47 UTC (Tue) by vonbrand (subscriber, #4458) [Link]
Versioning really-big (binary) files Posted Apr 6, 2010 23:12 UTC (Tue) by dlang (✭ supporter ✭, #313) [Link]
the real problem is that in many cases when people say 'huge binary file that changes a bit' they really mean 'huge binary file where the meaning changes a little, but the actual file contents change a lot', usually due to a compression algorithm being used
even for images and audio, if you were to check them in uncompressed the git delta functionality would work well and diff the files against each other, but if you compress the file (jpeg, mp3, or even png) before checking it in, a small change to the uncompressed data results in a huge change to the compressed data. If it's a lossless compression (i.e. png) then it would be possible to have git uncompress it before checking for differences, but if it's a lossy compression you can't do this.
Versioning really-big (binary) files Posted Apr 7, 2010 7:53 UTC (Wed) by paulj (subscriber, #341) [Link]
The real problem is people thinking such files are suitable for checking
into an SCM. Just archive them somewhere.
Versioning really-big (binary) files Posted Apr 12, 2010 1:14 UTC (Mon) by vonbrand (subscriber, #4458) [Link] Not really. If the contents needs version control, it should be handled by a VCS. The size or format of the files could be a technical hurdle, sure; but it shouldn't be an excuse for not solving the problem.
Subversion considered obsolete Posted Apr 4, 2010 20:55 UTC (Sun) by simlo (subscriber, #10866) [Link]
Now you should not put (large) binaries into a VCS. VCS is for source code, not any form of compilation output. What you store in the VCS is a script which can pull the right, uniqely defined (version, sha1, etc.) binary from some server, or a build script, which produces the binary.
Where I work, we use subversion (and I use git svn :-). We have scripts which pulls the tar.gz files for various packages in specific versions from a server, unpacks, patches and crosscompiles them to our target. The only thing we have in subversion is the scripts and the files we have changed.
For the Linux kernel we tried to have the full thing in subversion, but it took way too much for subversion, so now we only have a makefile, which clones a git repository, when the source is needed.
Subversion considered obsolete Posted Apr 5, 2010 3:12 UTC (Mon) by RCL (guest, #63264) [Link]
Please understand that some software products rely on large amounts of data which is essentially a part of program and should be versioned together with source code.
It's almost like having very large firmware blobs in Linux kernels, much larger than they are currently...
See comments below where I elaborate on this interdependency between code and data in games.
Subversion considered obsolete Posted Apr 5, 2010 14:00 UTC (Mon) by simlo (subscriber, #10866) [Link]
It depends on wether you edit those data or not. Firmware files you don't edit. You grap a specific version, someone compiled for you, labeled and released.
On the other hand you can have data like maps and icons, which is also "source code" and belongs in the VCS if you edit them by using some program (some map editor, GIMP or whatever). But the overall system is badly designed if these files are "large". They ought to be seperated into small files, each containing seperate parts of the information and then "compiled" into larger files. This will usually make a more flexible and maintainable system (besides making life easier for the VCS). It is the same with C code: You don't make one big file but smaller ones, seperated by functionality.
Subversion considered obsolete Posted Apr 5, 2010 3:22 UTC (Mon) by martinfick (subscriber, #4455) [Link]
"We can do anything better than you"..."not X"..."you shouldn't do X", hmmm?
Subversion considered obsolete Posted Apr 8, 2010 10:08 UTC (Thu) by epa (subscriber, #39769) [Link]
It's remarkable how quickly 'my preferred tool is unable to do X' turns into the moral imperative 'you should not do X'.
Subversion considered obsolete Posted Apr 5, 2010 12:12 UTC (Mon) by peschmae (guest, #32292) [Link]
I agree, that in these cases (blobs that you'd like to have versioned along with your source code) git may not be the right choice for you. Still it does very well for large source code repositories.
Looks like you're stuck with perforce :-p
|
|||||||||||||