Append-only large files

Posted Dec 13, 2018 16:51 UTC (Thu) by anarcat (subscriber, #66354)
In reply to: Append-only large files by epa
Parent article: Large files with Git: LFS and git-annex

I'm not exactly sure as I haven't reviewed the source code behind git-pack-objects, only the manual page, which says:

In a packed archive, an object is either stored as a compressed whole or as a difference from some other object. The latter is often called a delta. [...]

--window=<n>, --depth=<n>
These two options affect how the objects contained in the pack are stored using delta compression. The objects are first internally sorted by type, size and optionally names and compared against the other objects within --window to see if using delta compression saves space. --depth limits the maximum delta depth; making it too deep affects the performance on the unpacker side, because delta data needs to be applied that many times to get to the necessary object. The default value for --window is 10 and --depth is 50. The maximum depth is 4095.

So yes, it can also "optionally" "sort by name", but it's unclear to me how that works or how effective that is. Besides, the window size is quite small as well, although it can be bumped up to make pack take all available memory with that parameter. :)