fsync and scripts [LWN.net]

fsync and scripts

Posted Apr 7, 2009 20:46 UTC (Tue) by ncm (guest, #165)
Parent article: Linux Storage and Filesystem workshop, day 1

It seems customary for file system people to excoriate application coders who fail to call fsync() at appropriate moments. When, though, has calling fsync() at the right moment ever been possible in a shell script? Is it the opinion that one cannot write a well-behaved reliable, shell script anyway, so effort to accommodate the needs of scripts would be wasted?

fsync and scripts

Posted Apr 7, 2009 22:10 UTC (Tue) by Chousuke (subscriber, #54562) [Link] (4 responses)

Can't you just use 'sync' in a shell script?

fsync and scripts

Posted Apr 8, 2009 1:49 UTC (Wed) by mjg59 (subscriber, #23239) [Link] (3 responses)

No, since sync isn't provided by POSIX. Even if it was, POSIX doesn't require sync() to be synchronous.

fsync and scripts

Posted Apr 8, 2009 13:33 UTC (Wed) by vaurora (guest, #38407) [Link] (2 responses)

Actually, sync() is POSIX, but you are correct that the definition does not specify that the data on disk by the time it returns, just that it's been scheduled to be written out However, if you restrict your scripts to Linux and other sane operating systems, you will get the expected behavior modulo disk caching. From the sync(2) man page:

       According to the standard specification (e.g., POSIX.1-2001), sync() schedules the writes,
       but may return before the actual writing is done.  However,  since  version  1.3.20  Linux
       does  actually  wait.   (This  still  does not guarantee data integrity: modern disks have
       large caches.)

UFS on Solaris is the only case I know of that actually takes advantage of this hole and returns before the data hits disk. The last I heard, ZFS did the sane thing and waited.

fsync and scripts

Posted Apr 8, 2009 13:44 UTC (Wed) by mjg59 (subscriber, #23239) [Link] (1 responses)

The sync() system call is POSIX, but as far as I can tell the sync command isn't. Whether this is actually a real world issue, I have no idea, but since half of these conversations are about being POSIXly correct...

fsync and scripts

Posted Apr 14, 2009 7:10 UTC (Tue) by dlang (guest, #313) [Link]

I remember reading at one point that on some older systems sync could exit as soon as the fsync started, but if you did sync;sync the second one could not start until the first finished.

so sync was not enough sync;sync would do the job, but sync;sync;sync is what people ended up using.

this could be yet another myth, but it seems to match the facts that I have run across on the topic

fsync and scripts

Posted Apr 7, 2009 23:34 UTC (Tue) by bojan (subscriber, #14302) [Link]

FreeBSD has fsync command. GNU dd has options to do the same. See conv=fsync and conv=fdatasync options.