User: Password:
|
|
Subscribe / Log in / New account

avoiding orphan files

avoiding orphan files

Posted Sep 19, 2011 12:06 UTC (Mon) by aeriksson (guest, #73840)
Parent article: Ensuring data reaches disk

The upshot of using the write/sync/rename workflow is of course that the original file is left untouched until there is a fully comitted replacment ready on disk. The downside is that you need to create a temporary file with a temporary filename while doing it. This is bad for crash recovery, where you'd leave orphan files on the filesystem.

Is there a way to sole that which I have overlooked?

fd=open("",O_UNNAMED);
....
rename_unnamed(fd,"/some/file");


(Log in to post comments)

avoiding orphan files

Posted Sep 19, 2011 17:15 UTC (Mon) by mathstuf (subscriber, #69389) [Link]

As someone who uses symlinks to manage dotfiles in another repository, the write/sync/rename workflow is annoying as all hell. Tin (since moved to slrn), gpg, pidgin, weechat (which is why I'm still using irssi), and more all force me to manually copy the file to the real location and remake the symlink. If there was a way to do the workflow and then replay the writes to an fd returned by open() on the original path, it would be much better. So far, it looks as if all of the above solutions still fail for me.

Also in the write/sync/rename workflow, what happens if the temp file is on a separate filesystem? There's a copy involved there, so there is a time when the file is not atomically replaced (unless I'm missing some guarantee by POSIX in this case).

avoiding orphan files

Posted Sep 19, 2011 18:17 UTC (Mon) by nybble41 (subscriber, #55106) [Link]

> Also in the write/sync/rename workflow, what happens if the temp file is on a separate filesystem?

In the write/sync/rename workflow, this is never supposed to occur. The temp file must always be on the same filesystem as the real file for the atomic-rename guarantee to apply.

Naturally, this can be extremely difficult to achieve in some cases. The file may be a symlink, which must be fully resolved to a symlink-free path to determine the real filesystem. The file may be the target of a bind mount, in which case I doubt there is any portable way to determine which filesystem it came from. And there there's the possibility that you can write to the file, but not the directory _containing_ the file...

The write/sync/rename process is hardly an ideal way to implement atomic replacement semantics. There are simply too many potential points of failure.

avoiding orphan files

Posted Jan 25, 2012 20:38 UTC (Wed) by droundy (subscriber, #4559) [Link]

> The write/sync/rename process is hardly an ideal way to implement atomic replacement semantics. There are simply too many potential points of failure.

True, but it's also the only one we've got, right?


Copyright © 2017, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds