LWN.net Logo

Ext4 data corruption trouble [Updated]

Ext4 data corruption trouble [Updated]

Posted Oct 25, 2012 4:33 UTC (Thu) by butlerm (subscriber, #13312)
Parent article: Ext4 data corruption trouble [Updated]

Shouldn't quiescing all mounted filesystems and putting them into a read only state be something that the filesystem itself have responsibility to do? In the kernel, when instructed that the system is about to reboot/sleep/power off?

Expecting user code to track down all mounted filesystems and unmount them in reverse topological order doesn't sound like the sort of thing one would want filesystem integrity to depend on. It sounds like an ugly hack of the first magnitude.


(Log in to post comments)

Ext4 data corruption trouble [Updated]

Posted Oct 25, 2012 11:19 UTC (Thu) by nix (subscriber, #2304) [Link]

Ext4 data corruption trouble [Updated]

Posted Oct 25, 2012 22:16 UTC (Thu) by butlerm (subscriber, #13312) [Link]

A 'umountall' system call looks like it could do the job nicely, with one exception.

It might be helpful to have a way to put all filesystems in a read only state, for the benefit of shutdown code that only requires (file level) read access, such as code to shutdown RAID devices.

A more general problem is that you might have loopback mounts and nested block devices, so what you really need is a combined operation that does the topological sort and quiesces filesystems and block devices in reverse stacking order.

Ext4 data corruption trouble [Updated]

Posted Oct 25, 2012 16:02 UTC (Thu) by wahern (subscriber, #37304) [Link]

I still have an ingrained habit of typing `sync' at idle moments in my shell, picked up in the early days of ext2. Re-downloading the Slackware floppy set (because invariably one of the disks of a previous downloaded set would be go bad) over a 2400 baud modem was not fun times. Because things were generally less stable back then, and you never knew when the system might crash and leave the disk corrupt and unbootable, vigorous and frequent syncing was the only alternative.

Ext4 data corruption trouble [Updated]

Posted Oct 26, 2012 0:44 UTC (Fri) by jhardin (guest, #3297) [Link]

I still have an ingrained habit of typing `sync' at idle moments in my shell, picked up in the early days of ext2.
+1, except I picked up the habit on SCO Xenix.

sync;sync;sync

Ext4 data corruption trouble [Updated]

Posted Oct 26, 2012 1:22 UTC (Fri) by pr1268 (subscriber, #24648) [Link]

I like this solution:

/* 3sync.c */
#include <unistd.h> /* for sync(2) */
#include <time.h>   /* for struct timespec and nanosleep(2) */

int main()
{
    struct timespec ts = { 0, 1000000L };
    sync();
    (void) nanosleep(&ts, 0);
    sync();
    (void) nanosleep(&ts, 0);
    sync();
    return 0;
}

Compiled and placed in $HOME/bin (which is in $PATH), and now it gets used quite frequently in other scripts I run. Which either (1) is horribly inefficient, and/or (2) shows how paranoid I am with data corruption. Sigh.

Ext4 data corruption trouble [Updated]

Posted Oct 26, 2012 0:21 UTC (Fri) by ncm (subscriber, #165) [Link]

Maybe I'm missing something. Why not start by remounting all the file systems to a synchronous-write mode, first? Then, sync. After that, you can execute a HCF(*) instruction, wait long enough for the lying drives to drain their internal queues, power down, and everything should be fine. Right?

(*) "Halt and Catch Fire"

Ext4 data corruption trouble [Updated]

Posted Oct 26, 2012 0:26 UTC (Fri) by nix (subscriber, #2304) [Link]

That requires that you can *find* all the file systems. In the presence of PID and fs namespaces, no single process can necessarily do that (nor can any single process necessarily talk to any group of processes that can do that, even indirectly).

This is somewhat unlikely, it is true.

Ext4 data corruption trouble [Updated]

Posted Oct 26, 2012 0:45 UTC (Fri) by neilbrown (subscriber, #359) [Link]

echo S > /proc/sysrq-trigger
echo U > /proc/sysrq-trigger
??

Ext4 data corruption trouble [Updated]

Posted Oct 26, 2012 14:04 UTC (Fri) by nix (subscriber, #2304) [Link]

Oh. Yeah. That would work. I completely forgot that sysrq-trigger even existed. :)

New syscall, who needs it, though relying on sysrq-trigger for something as fundamental as shutting down seems a little icky.

Ext4 data corruption trouble [Updated]

Posted Oct 26, 2012 14:37 UTC (Fri) by butlerm (subscriber, #13312) [Link]

Does that work with loopback mounted filesystems, i.e. is the sysrq handler smart enough to unmount a loopback mounted filesystem before the filesystem that holds its backing store?

Ext4 data corruption trouble [Updated]

Posted Oct 27, 2012 20:17 UTC (Sat) by nix (subscriber, #2304) [Link]

Sysrq-U remounts read-only, it doesn't unmount.

(But, still, you do want to remount the loopback-mounted filesystem first, or that umount won't be able to do e.g. journal flushes...)

And the answer is no: it ends up calling do_emergency_remount(), which does a straight iteration over all super_blocks: there is no dependency analysis of any kind: I'd expect (given the way super_blocks is built) to unmount the backing store fs *before* the loopback-mounted fs.

(Perhaps do_emergency_remount() should iterate over super_blocks in reverse order?)

Copyright © 2013, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds