|
|
Subscribe / Log in / New account

Git archive generation meets Hyrum's law

Git archive generation meets Hyrum's law

Posted Feb 6, 2023 10:35 UTC (Mon) by farnz (subscriber, #17727)
In reply to: Git archive generation meets Hyrum's law by epa
Parent article: Git archive generation meets Hyrum's law

Or we could go one better; while making the compressor deterministic is hard, making the uncompressed form deterministic is not (when uncompressed, it's "just" a case of ensuring that everything is done in deterministic order). We then checksum the uncompressed form, and ship a compressed artefact without checksums.

Note in this context that HTTP supports "Content-Transfer" encodings: so we can compress for transfer, while still transferring and checksumming uncompressed data. And you can save the compressed form, so that you don't waste disk space - or even recompress to a higher compression if suitable.


to post comments

Git archive generation meets Hyrum's law

Posted Mar 25, 2023 12:47 UTC (Sat) by sammythesnake (guest, #17693) [Link]

If the archive contains, say, a100TB file of zeros, then you'd end up filling your hard drive before any opportunity to checksum it that way. If the archive is compressed before downloading, there's at least the option of doing something along the lines of zcat blah.tgz | sha...


Copyright © 2025, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds