Weekly edition	Kernel	Security	Distributions	Contact Us	Search
Archives	Calendar	Subscribe	Write for LWN	LWN.net FAQ	Sponsors

Versioning really-big (binary) files

Posted Apr 6, 2010 18:47 UTC (Tue) by vonbrand (subscriber, #4458)
In reply to: Versioning really-big files by smurf
Parent article: A proposed Subversion vision and roadmap

git uses delta compression by default (and has done so for a long time now), so the "huge binary files that change a bit" shouldn't be a problem. Please check with the latest version.

(Log in to post comments)

Posted Apr 6, 2010 23:12 UTC (Tue) by dlang (✭ supporter ✭, #313) [Link]

the real problem is that in many cases when people say 'huge binary file that changes a bit' they really mean 'huge binary file where the meaning changes a little, but the actual file contents change a lot', usually due to a compression algorithm being used

even for images and audio, if you were to check them in uncompressed the git delta functionality would work well and diff the files against each other, but if you compress the file (jpeg, mp3, or even png) before checking it in, a small change to the uncompressed data results in a huge change to the compressed data. If it's a lossless compression (i.e. png) then it would be possible to have git uncompress it before checking for differences, but if it's a lossy compression you can't do this.

Versioning really-big (binary) files

Posted Apr 7, 2010 7:53 UTC (Wed) by paulj (subscriber, #341) [Link]

The real problem is people thinking such files are suitable for checking
into an SCM. Just archive them somewhere.

Versioning really-big (binary) files

Posted Apr 12, 2010 1:14 UTC (Mon) by vonbrand (subscriber, #4458) [Link]

Not really. If the contents needs version control, it should be handled by a VCS. The size or format of the files could be a technical hurdle, sure; but it shouldn't be an excuse for not solving the problem.