User: Password:
Subscribe / Log in / New account

symlinks, immutability, wrappers, hardlinks, mount-bind

symlinks, immutability, wrappers, hardlinks, mount-bind

Posted May 27, 2011 4:56 UTC (Fri) by Duncan (guest, #6647)
In reply to: DVCS-autosync by mathstuf
Parent article: DVCS-autosync

At least four independent techniques exist for keeping files in two different trees in sync, three of which have already been mentioned in other replies, but it's worth collecting them into one comment, and I didn't see the fourth mentioned at all, yet.

Symlinks: When I switched from MS dark side, symlinks quickly became my favorite *ix filesystem feature. =:^) Unfortunately, by themselves, they have one significant already mentioned limitation: apps that write a temp file and then mv it over the old permanent file, possibly mv-ing the old version to backup at the same time. (This is A good technique on its own as it better ensures integrity of either the file or at last resort its backup, in case of a crash at the wrong moment, but as implemented by many apps, it's not compatible with symlinking.)

In some cases, it might even be possible to reverse symlink, putting the symlinks in the sync-tree, if it's possible to tell whatever's syncing them (git by default, here, I don't know git well enough to know if it can handle that or not) to dereference symlinks instead of syncing the symlink itself.

Ext2/3/4 users at least might also be able to work around the rewriting problem by setting the immutable bit on the file, particularly in the mentioned scenario where an app is overwriting the file even when nothing changed. For filesystems that don't implement immutable, filesystem permissions can be used, setting the file, or its directory if setting only the file doesn't work, read-only, and if necessary, changing the ownership (assuming one has the privs to do so, of course). I didn't see these mentioned as options, yet.

Wrapper scripts: Someone mentioned application wrapper scripts as another option. Create a script by the same name and set it earlier in the path, have it copy the file in question out of the synced tree, then launch the desired app via absolute path name. The wrapper then waits around until the desired app exits, after which it copies the changed file back to the synced tree. (If the user doesn't want to save the changes, the app launch can use exec or similar to avoid having the wrapper stick around for the main app to finish.) Note that GUI launcher users may need to customize their GUI launcher to point at the wrapper, as it may already be using the absolute path or may not be using the same path ordering. FWIW I suspect many LWN users may already be using the wrapper technique for other purposes but may not have thought of using it as a solution for this problem.

Hardlinks: Another suggested solution is hardlinks. That can work for some things, particularly in combination with the immutable/read-only technique above, but hard links have their own negatives. A major one discouraging their use here is that hardlinks are same-filesystem-only, thus limiting one's filesystem management flexibility. A second negative is that files with multiple hard links simply aren't particularly intuitive to those who started on the MS side of things, another weakness of at least the FAT filesystems (I'm unsure about NTFS), where such linking tended to be the result of filesystem corruption and wasn't supported as a feature.

Mount-bind: Not yet mentioned, however, is the very useful mount --bind option, allowing one part of a filesystem, even an individual file, to be mounted elsewhere in the tree as well. =:^) This is a relatively new addition to the toolbox, and thus relatively unknown as it hasn't yet really entered the bank of common wisdom out there yet, as the Linux kernel and the mount command from the util-linux package haven't supported this for all that long. But at least for users with suitable mount privileges available, it is certainly one of the most powerful options, since it not only allows a user to make a directory or file existing in one location in the tree appear elsewhere either in addition or instead of the original location, but ALSO comes with all thue usual mount permissions restrictions ability, including making the mount read-only or read/write, nodev/noexec/nosuid, etc. Mount-binding a file as read-only in effect makes it immutable as addressed from that location, but without the usual filesystem-must-support-immutable limitations, as it's enforced by the kernel mount infrastruction instead of the filesystem. And while it makes little sense in the context of an individual file, directory/subtree level noexec/nodev/nosuid certainly has its uses, as well. Mount-bind functions in this way like an admin-level symlink, and I'm appreciating its benefits more and more as I use it, much as I did symlinks early on, when I first switched from the dark side.

Certainly there's no shortage of choices for keeping a sync-tree file in sync with a file elsewhere in the tree. Given all the choices available, /one/ of them should be workable.


(Log in to post comments)

Copyright © 2017, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds