Perforce [LWN.net]

Perforce

Posted Oct 21, 2009 9:25 UTC (Wed) by cortana (subscriber, #24596)
In reply to: Perforce by epa
Parent article: KS2009: How Google uses Linux

I just ran into one of the reasons yesterday. We were trying to check in a bunch of large binary
files, some several hundred megabytes large. Git ran out of memory with a fairly uninformative
error message while 'packing' objects, whatever that means...

Fortunately, a short trip to #git revealed the cause of the problem: git compresses objects
before sending them to a remote repository; it simply ran out of virtual memory while
compressing some of the larger files.

There were two fixes.

1. Use a 64-bit version of git. I'd be happy to, but there isn't an x64 binary download available
from the msysgit web site.

2. Tell git not to perform the compression; 'echo * -delta > .git/info/attributes'. Somewhat
undocumented, but at least I will be able to search for this LWN comment if I ever run into this
problem again. :)

(Log in to post comments)

-delta

Posted Oct 21, 2009 23:31 UTC (Wed) by joey (subscriber, #328) [Link]

Me and my 50 gb git repos thank you for that! But since finding LWN comments
in future is not my strong suite, I sent in a patch to document it on
gitattributes(1) ;)

Looks to me to make large object commits fast, but git pull will still
compress the objects, and still tends to run out of memory when they're
large.

-delta

Posted Oct 21, 2009 23:51 UTC (Wed) by cortana (subscriber, #24596) [Link]

Thanks so much for that. I would have suggested a patch, honest, but I'm super busy at work at the moment... ;)

Presumably git-pull running out of memory would be a server-side issue? And in that case, if you're not running a sensible 64-bit operating system on your server then you deserve what you get... ;)