User: Password:
|
|
Subscribe / Log in / New account

XFS: the filesystem of the future?

XFS: the filesystem of the future?

Posted Mar 5, 2012 21:36 UTC (Mon) by dlang (subscriber, #313)
In reply to: XFS: the filesystem of the future? by nybble41
Parent article: XFS: the filesystem of the future?

The process you list will badly fragment the file on disk, destroying performance (unless you re re-writing the entire file, in which case just writing a temporary file and renaming it will work)

It's also not clear that this is what the original poster was looking for when he said "Linux devs should really provide a proper solution (like O_ATOMIC) instead of blaming app devs for not doing the impossible."

I've been trying to make two points in this thread

1. it's not obvious what the "proper solution" that the kernel should provide looks like

2. it's not impossible to do this today (since there are many classes of programs that do this)


(Log in to post comments)

XFS: the filesystem of the future?

Posted Mar 5, 2012 22:45 UTC (Mon) by nybble41 (subscriber, #55106) [Link]

>> This does assume _complete_ replacement, i.e. O_ATOMIC implies O_TRUNC.
> The process you list will badly fragment the file on disk, destroying performance (***unless you re re-writing the entire file***, in which case just writing a temporary file and renaming it will work)

Emphasis added. Yes, I assumed the entire file was being rewritten. If not, one could add an online defragmentation step after the metadata update, though online defragmentation introduces atomicity issues of its own.

Creating and renaming a temporary file has its own issues, which have already been mentioned, particularly relating to symlinks and hard links. Even for ordinary files, it's not guaranteed that there exists a directory on the same filesystem where you have permission to create a temporary file. Bind mounts (which can apply to individual files) are another potential sore spot. How much work should applications be expected to do just to find a place to put their temporary file such that rename() can be guaranteed atomic?

An O_ATOMIC option to open() would ensure that you are really replacing the original file, and that the temporary space comes from the same filesystem.

1) No, there are no obvious solutions yet, but there have been several reasonable proposals.

2) Current applications can do atomic replacement in common but limited circumstances using rename(). They all depend on creating a temporary file on the same filesystem, which assumes both that you can locate that filesystem (see: bind mounts) and that you can create new files there. They also tend to break hard links, which may or may not be a desired behavior. There is no general, straightforward solution to the problem of atomically replacing the data associated with a specific inode.

XFS: the filesystem of the future?

Posted Mar 5, 2012 23:37 UTC (Mon) by dlang (subscriber, #313) [Link]

> it's not guaranteed that there exists a directory on the same filesystem where you have permission to create a temporary file.

this sounds like a red hearing to me.

If you aren't allowed to create a new file in the directory of the file, are you sure you have permission to overwrite the file you are trying to modify?

as for the symlink/hardlink 'issue', in my sysadmin experience, more problems are caused by editors that modify files in place (not using temp files and renaming them) than by breaking links. Editors that break links when modifying a file are referred to as 'well behaved' in this area.

XFS: the filesystem of the future?

Posted Mar 6, 2012 0:28 UTC (Tue) by nybble41 (subscriber, #55106) [Link]

> If you aren't allowed to create a new file in the directory of the file, are you sure you have permission to overwrite the file you are trying to modify?

These are orthogonal permissions. To overwrite the file you need write permission on the file. To create a new file in the same directory you need write permission on the directory. It's easily possible to have one without the other:

root# mkdir a
root# touch a/b
root# chown user a/b

user$ echo test > a/b # no error
user$ touch a/c
touch: cannot touch `a/c': Permission denied

> as for the symlink/hardlink 'issue', in my sysadmin experience, more problems are caused by editors that modify files in place (not using temp files and renaming them) than by breaking links.

Obviously that would remain an option. The point is that it should be an *option*, alongside the ability to atomically update a file in place. Symlinks are often used to add version control over configuration files (without putting the entire home / etc directory in the repository), while bind mounts are more often used with namespaces and chroot environments. Usually you want the latter to be read-only, but if the file is read/write it makes sense to allow it to be updated without breaking the link. (You can't just delete or rename over a bind target, either; it has to be unmounted.)


Copyright © 2017, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds