Solving the ext3 latency problem

OK, so there are multiple issues when people talk about "safety" and data=writeback. First of all, for ext3. In the security dimension, with data=writeback, if after a crash where the filesystem was not cleanly unmounted, files that were written right before the crash might contain unitialized data. This data could be from another user, although on a single-user system, this is obviously not very likely. How severely you consider this really depends on how paranoid you are. Even on a single-user system, this data might contain information that you don't want to send out publically, and if you don't notice that a file contained something other than what you thought before you send it out to someone else, there is a potential for a security exposure. Obviously, though on a single user system this is much less of an issue than on timesharing system.

In the data loss department, if you have an application that didn't use fsync(), and the system crashes, with data=writeback there is the chance for dataloss. In 2.6.30, Linus accepted patches which will cause an implied flush operation when a hueristic detects an application trying to replace an existing file via the replace-via-truncate and replace-via-rename cases patterns. This largely reduces the problems for non-fsync-using applications. It doesn't solve the problem for a freshly written file, but the system could have easily crashed five seconds earlier.

OK, so how does ext4 change things. By default ext4 on modern kernels (ignoring the technology preview on RHEL 5 and Fedora 10) performs delayed allocation. This means that the data blocks are not allocated right away when you write the file, but only when they are forced out, either explicitly via fsync(), or via the page writeback algorithms in the VM, which will tend to push things out after 30-45 seconds (ignoring laptop mode) and perhaps sooner if the system is short on memory.

In the security dimension, what this means is that even in data=writeback mode, in general on a crash the file will be truncated or zero-length instead of containing uninitialized data. In ext4 with delayed allocation and data=writeback, there *is* a very tiny race condition where if a transaction closes right between when the pdflush daemon allocates the filesystem block and before it has a chance to trigger the page writeback, that you might end up with uninitialized garbage. This chance is very small, but it is non-zero. In this case, ext4 data=ordered will force the write to disk, so it is technically safer in the security dimension, although this race is very hard to exploit, and very rare that it gets hit in practice. (This is also why the overhead of data=ordered and data=writeback is much less for ext4, thanks to delayed allocation --- the difference between the two is not the same, however!)

In the safety against applications that don't use fsync department, as of 2.6.30, ext4 will always do an implied allocation and flush for data=ordered and data=writeback. So there is no real difference here between data=ordered and data=writeback.

The bottom line is that while there is some performance benefit in going with data=writeback with ext4, the differences between data=ordered and data=writeback are much smaller with ext4, in both the cost and benefit dimensions.

Chris Mason is also working on a data=guarded mode, which will cause files to be truncated (much like delayed allocation) on a crash with ext3. I will look into porting this mode into ext4, if it proves to be enough of a performance advantage for ext4 over data=ordered, and yet providing a tiny bit more safety than data=writeback. It's not clear to me that it will be worth it for ext4, however.

I hope this helps answers the questions between ext3 and ext4, and data=ordered versus data=writeback.

Regards,

Ted.