Create an account

Subscribe to LWN

LWN.net Weekly Edition for May 23, 2013

An "enum" for Python 3

An unexpected perf feature

LWN.net Weekly Edition for May 16, 2013

A look at the PyPy 2.0 release

Weekly edition	Kernel	Security	Distributions	Contact Us	Search
Archives	Calendar	Subscribe	Write for LWN	LWN.net FAQ	Sponsors

Better get the basics straight

Better get the basics straight

Posted Mar 17, 2009 22:42 UTC (Tue) by butlerm (subscriber, #13312)
In reply to: Better get the basics straight by job
Parent article: Better than POSIX?

I agree. I am coming around to the position that all self respecting
filesystems should provide ordered (but not necessarily durable) renames by
default.

Then, as suggested by a commenter elsewhere, we could add something like a
"rename_unordered" function for those relatively unusual cases where a user
is willing to risk severe data loss to get better performance. In addition,
for portability reasons, we should have an option so that an application
can discover whether or not an fsync is required to get ordered renames.
fsync is very expensive when you don't need (synchronous) durability
semantics.

(Log in to post comments)

Better get the basics straight

Posted Mar 17, 2009 22:53 UTC (Tue) by quotemstr (subscriber, #45331) [Link]

POSIX has a mechanism to do precisely that:

SYNOPSIS long fpathconf(int fildes, int name); long pathconf(const char *path, int name); DESCRIPTION The fpathconf() and pathconf() functions shall determine the current value of a configurable limit or option (variable) that is associated with a file or directory.

I don't see a problem with adding platform-specific values for name: if an conscientious application asks for _LINUX_SAFE_RENAME on a system that doesn't even know it exists, pathconf will just return -1 and the application will say, "oh, okay. I need to use fsync."

If the POSIX people ever get around to standardizing safe semantics for rename, then Linux's pathconf can just support both the original and the standard name.

Better get the basics straight

Posted Mar 18, 2009 1:00 UTC (Wed) by bojan (subscriber, #14302) [Link]

You'll probably need this on a per-FS basis, because you could mount an FS on Linux that doesn't support this, although the system may generally support it.

Better get the basics straight

Posted Mar 18, 2009 1:04 UTC (Wed) by quotemstr (subscriber, #45331) [Link]

You'll probably need this on a per-FS basis

Of course it's per-filesystem. That's the whole point of using pathconf instead of sysconf.

Better get the basics straight

Posted Mar 18, 2009 1:11 UTC (Wed) by bojan (subscriber, #14302) [Link]

OOPS! My bad, sorry :-(

fpathconf

Posted Mar 18, 2009 4:41 UTC (Wed) by butlerm (subscriber, #13312) [Link]

Thanks for posting that, that is very helpful.

Better get the basics straight

Posted Mar 19, 2009 7:27 UTC (Thu) by job (guest, #670) [Link]

That would be one way of dealing with different filesystem semantics.

I would hope that the operation "write data to file" gets less complex, not more. There is already a little dance of calls to be made (Ted writes about it). If we add logic on the application level to handle that some filesystems expect fsync on the directory, some on only the file and some manage without, it becomes even more so. In tens of thousands of applications.

But this is only vaguely related to the data ordering issue. In an interactive program or where performance is critical you may not want to wait until data is commited to disk. Latency kills.

Copyright © 2013, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds