Garrett: ext4, application expectations and power management

Wow. This thread is soaking up a huge portion of my work day. I posted this on ted's blog:

>> in order to get very high performance levels, file systems usually combine 5-30 seconds worth of >> file system operations into a single transaction commit.

Ted, thanks for continuing the dialogue here. It’s been educational. Thanks for putting the rename ‘kludge’ in, but I do think it’s absolutely necessary. I will give you my use case.

I use the write-close-rename all the time with configuration files that I don’t care if I lose the last N seconds of change to. This is easy and thread safe and has worked for years on ext3. At the same time, I have a daemon that is continually writing out large log files. We also run on cheap IDE hardware due to cost pressures. It seems that fsyncs will force your above described ‘mega transaction’ to complete, which involves seeking all over the disk to our other dirty log files. If I made multiple changes to a conf file during a 5-30 second ext4 flush interval, fsync will cause more seeks than not using it, which will wear out our disks.

You are right that the FS is not a database, certain things I do not care about instant durability. So we can:
1. have the FS support write barriers on rename or other new api
2. do writes to a user space cache daemon that only flushes when necessary
3. make every app much more complicated and cache its own data

#1 already works.