Improving ext4: bigalloc, inline data, and metadata checksums

What you're trying to do with moving back to ext3 is what the Jargon File calls shotgun debugging: trying out some radical move in hopes that this will fix your problem.

That's not shotgun debugging (and not what the Jargon File calls it). The salient property of a shotgun isn't that it makes radical changes, but that it makes widespread changes. So you hit what you want to hit without aiming at it.

Shotgun debugging is trying lots of little things, none that you particularly believe will fix the bug.

In this case, the fallback to ext3 is fairly well targeted: the problem came contemporaneously with this one major and known change to the system, so it's not unreasonable to try undoing that change.

The other comments give good reason to believe this is not the best way forward, but it isn't because it's shotgun debugging.

There must be a term for the debugging mistake in which you give too much weight to the one recent change you know about in the area; I don't know what it is. (I've lost count of how many people accused me of breaking their Windows system because after I used it, there was a Putty icon on the desktop and something broke soon after that).

Thanks for the pointer, and thanks also to yoe's reply above. But, my music collection (currently over 10,000 files) has existed for almost four years, ever since I converted the entire collection from MP3 to OGG (via a homemade script which took about a week to run).¹ (I've never converted from FLAC to OGG, although I do have a couple of FLAC files.) I never noticed any corruption in the OGG files until a few months ago, shortly after I did a clean OS re-install (Slackware 13.37) on bare disks (including copying the music files)². I'm all too eager to blame the corruption on ext4 and/or LVM, since those were the only two things that changed immediately prior to the corruption, but you both bring up a good point that maybe I should dig a little deeper into finding the root cause before I jump to conclusions.

¹ I've had this collection of (legitimately acquired) songs for years prior, even having it on NTFS back in my Win2000/XP days. I abandoned Windows (including NTFS) in August 2004, and my music collection was entirely MP3 format (at 320 kbit) since I got my first 200GB hard disk. After seeing the benefits of the OGG Vorbis format, I decided to switch.

² I have four physical disks (volumes) in which I've set up PV set spanning across all disks for fast I/O performance. I'm not totally impressed at the performance—it is somewhat faster—but that's a whole other discussion.

The big problem with 'ext4' is that its only reason to be is to allow Red Hat customers to upgrade in place existing systems, and what Red Hat wants, Red Hat gets (also because they usually pay for that and the community is very grateful).

I also see that I was making some sort of horrible mistake by installing ext4 on all my newer systems, but you never make clear what that mistake might have been.

In particular JFS should have been the "default" Linux filesystem instead of ext[23] for a long time. Not making JFS the default was probably the single worst strategic decision for Linux (but it can be argued that letting GKH near the kernel was even worse).

I've been wracking my brains and I can't think of one thing Greg has done that has come to public knowledge and could be considered bad. So this looks like groundless personal animosity to me.

$ ls -ld /dev/tty9
crw--w---- 1 root tty 4, 9 2011-11-28 14:03 /dev/tty9
$ cat /sys/class/tty/tty9/dev
4:9

Thanks for the suggestion; I'll give it a try sometime if/when I find a corrupt OGG file.

Just to bring some closure to this discussion, I wish to make a few points:

1. In 13 years of using Linux, I've not once ever experienced data corruption on any filesystem running in Linux (prior to the random OGG errors I encountered a few months ago on ext4 and LVM). This includes ext2, ext3, Reiser(3)fs, and Linux's fat and vfat implementations. (Hardware failures are not included, but I've been fortunate not to have much experience with those.)
2. My existing hard drives are error-free (according to smartctl -l as suggested above). I have two IDE disks and two SATA disks, all Seagate brand (not that this matters, but I do admire their 5-year warranty).
3. My OGG files are all on their own filesystem/partition (ext4), for which only root has write privileges, and thus my non-privileged account can't accidentally be writing to this filesystem (or so I'd hope).
4. I've since created an oggcheck.sh script which scans the whole filesystem containing the OGG files and reports any errors found to a log.
5. Running this script lately (within past 1 month or so) has yielded no errors, so I'm wondering if (hoping that) this is some weird anomaly which has passed.
6. Any errors experienced on the OGG filesystem would imply the possibility of similar errors on my / filesystem (which includes /home), but I haven't noticed any such errors as of yet (and I hope that none exist!).
7. I'm all too eager to blame this on ext4 or LVM (or the combination of both), but I'm equally eager to blame this on some strange operator error. Trust me, I'm good at creating these!

Many thanks to everyone's discussion above; I always learn a lot from the comments here on LWN.

It is very cheap insurance.

ECCRAM is worthwhile, but it is not at all cheap once you factor all that in.