LWN.net Logo

Versioning really-big (binary) files

Versioning really-big (binary) files

Posted Apr 6, 2010 23:12 UTC (Tue) by dlang (✭ supporter ✭, #313)
In reply to: Versioning really-big (binary) files by vonbrand
Parent article: A proposed Subversion vision and roadmap

the real problem is that in many cases when people say 'huge binary file that changes a bit' they really mean 'huge binary file where the meaning changes a little, but the actual file contents change a lot', usually due to a compression algorithm being used

even for images and audio, if you were to check them in uncompressed the git delta functionality would work well and diff the files against each other, but if you compress the file (jpeg, mp3, or even png) before checking it in, a small change to the uncompressed data results in a huge change to the compressed data. If it's a lossless compression (i.e. png) then it would be possible to have git uncompress it before checking for differences, but if it's a lossy compression you can't do this.


(Log in to post comments)

Versioning really-big (binary) files

Posted Apr 7, 2010 7:53 UTC (Wed) by paulj (subscriber, #341) [Link]

The real problem is people thinking such files are suitable for checking
into an SCM. Just archive them somewhere.

Versioning really-big (binary) files

Posted Apr 12, 2010 1:14 UTC (Mon) by vonbrand (subscriber, #4458) [Link]

Not really. If the contents needs version control, it should be handled by a VCS. The size or format of the files could be a technical hurdle, sure; but it shouldn't be an excuse for not solving the problem.

Copyright © 2013, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds