User: Password:
|
|
Subscribe / Log in / New account

DVCS-autosync

DVCS-autosync

Posted May 16, 2011 21:12 UTC (Mon) by dlang (subscriber, #313)
In reply to: DVCS-autosync by njs
Parent article: DVCS-autosync

you have a valid point in terms of being able to mmap things (which is not quite the same as being able to fit them in memory). this is a limitation on 32 bit systems

however, how can you do version control if you don't keep a copy of the file somewhere else? if someone changes it, how can you get back what was there before without another copy?


(Log in to post comments)

DVCS-autosync

Posted May 16, 2011 21:34 UTC (Mon) by njs (guest, #40338) [Link]

Oh, does it use mmap exclusively for all file access now? Including repacking and everything? Neat.

You need two local copies if you want to do local version control, and also to let people edit files normally on disk (as opposed to, say, interposing a FUSE filesystem to observe edits as they happen). But the systems we're talking about are not trying to do local version control. They're trying to do remote backup and syncing!

*For this use-case*, you might almost be better off with CVS than with git. Its handling of binary files is dumb, but at least it wouldn't double your local storage requirements. Even better, of course, would be a system that stored the second copy on the remote server only, and then used something clever like rsync to upload the deltas.

Or maybe one could do something clever with libgit2 and librsync to let you directly and efficiently commit a local set of files to a remote bare repository...

DVCS-autosync

Posted May 16, 2011 21:53 UTC (Mon) by dlang (subscriber, #313) [Link]

I don't know that it always uses mmap, but it's the use of mmap that imposes the file size limit on individual files.

pack files are limited in that they use a 32 bit offset into them, but that's a matter of optimisation for files that can be diffed and compressed.

yes, for this use case you may be better of with CVS, but only until you have to reconcile differences between different locations. DVCS tools give you the framework (and many of the mechanisms) for doing this as part of their heritage

DVCS-autosync

Posted May 16, 2011 22:11 UTC (Mon) by njs (guest, #40338) [Link]

> yes, for this use case you may be better of with CVS, but only until you have to reconcile differences between different locations. DVCS tools give you the framework (and many of the mechanisms) for doing this as part of their heritage

Yes, of course. (Though in practice I'm not sure git's current merge mechanisms are well-optimized for the collection-of-large-binary-files case either.)

But that doesn't change the point, which is that git is not a perfect match for this problem, and a better tool that was similar to git in some ways but not in others could potentially do substantially better.

DVCS-autosync

Posted May 16, 2011 22:16 UTC (Mon) by dlang (subscriber, #313) [Link]

I agree that git does not do anything that useful for large binary files, and if that is all you are wanting to sync, then you are better looking at modifying git rather than using it directly.

but this depends in large part on what these binary files are. git supports configurable diff/merge engines, so if there is any sane way to merge your 'binary' files, git will allow you to use it.

please don't get me wrong, I'm not saying that git is perfect, just that it does a better job than anything else for the general case and brings a lot to the party. This makes basing a tool on git (or one of the other DVCS systems if you dislike git for some reason) a very reasonable thing to do rather than just developing your app from scratch.

DVCS-autosync

Posted May 17, 2011 14:45 UTC (Tue) by nye (guest, #51576) [Link]

You appear to be describing bup: https://github.com/apenwarr/bup


Copyright © 2017, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds