| |||||||||||||
|
| |||||||||||||
Filesystems (ext3, reiser, xfs, jfs) comparison on Debian Etch (Debian Administration)Filesystems (ext3, reiser, xfs, jfs) comparison on Debian Etch (Debian Administration)Posted Apr 25, 2006 22:50 UTC (Tue) by dwheeler (guest, #1216)Parent article: Filesystems (ext3, reiser, xfs, jfs) comparison on Debian Etch (Debian Administration) Nice filesystem survey article! But it omits some other trades. In particular, according to Theodore Ts'o (at FISL 2005), there's a serious "data loss in power fail" condition in XFS. SGI's XFS was originally designed for SGIs, and SGIs were specifically designed so hard drives would stop writing when power began to fail... to prevent catastrophe. XFS was designed with this presumption (that the hard drives would stop first). Unfortunately, x86 systems don't do that... the memory usually goes first, so you end up writing garbage to the disk on an x86. That can be a BIG BIG problem. Ts'o claimed that the other filesystems had much less risk and data loss when power failed and you're using a commodity x86 design. If your system is on a UPS, and you trust it, I understand that XFS works very nicely in many circumstances. But it may not be a good solution without one. Perhaps this is obsolete information, I'd love to know if this is no longer true.
(Log in to post comments)
Filesystems (ext3, reiser, xfs, jfs) comparison on Debian Etch (Debian Administration) Posted Apr 25, 2006 23:16 UTC (Tue) by hch (guest, #5625) [Link] This has never been true. People have been trying to talk Ted out of that for at least five years, but he keeps on repeating it without backing thefacts unfortunately.
Filesystems (ext3, reiser, xfs, jfs) comparison on Debian Etch (Debian Posted Apr 25, 2006 23:56 UTC (Tue) by khim (subscriber, #9252) [Link] I was biten by this bug myself quite a few times: system looked "just fine" at first glance but SHA1sums of files were different. Unfortunatelly I had SHA1sums for some files but not for all of them and this was HUGE mess. In the end I've converted all servers to ReiserFS and never looked back. Sorry, but "XFS is flacky when powered off under load" is NOT a Ted's delusion. I never was able to reproduce such problem without terabytes of data and high load so I know debugging is hard, but if you don't have high load and terabytes of data then you can use any filesystem and it'll not produce any difference, right ? Conclusion: do not use XFS... Ever,..
Filesystems (ext3, reiser, xfs, jfs) comparison on Debian Etch (Debian Posted Apr 26, 2006 0:54 UTC (Wed) by hawk (subscriber, #3195) [Link] On the other hand, with machines with terabytes of data and high load, one might consider having UPS... :-)
Filesystems (ext3, reiser, xfs, jfs) comparison on Debian Etch (Debian Posted Apr 26, 2006 1:10 UTC (Wed) by sbergman27 (subscriber, #10767) [Link] So why have journalling filesystems at all? So much development time.... just wasted. And all we needed to do was buy a UPS! ;-)
Filesystems (ext3, reiser, xfs, jfs) comparison on Debian Etch (Debian Posted Apr 26, 2006 15:15 UTC (Wed) by NightMonkey (subscriber, #23051) [Link] Certainly consider it, but a UPS doesn't do much in the presence of a short on the MB, or all power supplies failing, or even the power connector on the hard drive coming loose, etc....
Filesystems (ext3, reiser, xfs, jfs) comparison on Debian Etch (Debian Posted Apr 26, 2006 7:14 UTC (Wed) by nix (subscriber, #2304) [Link] You converted filesystems to *reiserfs* for *stability*?
Filesystems (ext3, reiser, xfs, jfs) comparison on Debian Etch (Debian Posted Apr 26, 2006 13:01 UTC (Wed) by arafel (subscriber, #18557) [Link] Some people like to live dangerously...
Filesystems (ext3, reiser, xfs, jfs) comparison on Debian Etch (Debian Posted Apr 26, 2006 21:18 UTC (Wed) by nix (subscriber, #2304) [Link] God knows why. It's not even especially fast, even compared to something like ext3 with dir_index, IME.
(A shame: I like tree-structured filesystems a lot. Just not reiserfs --- or NTFS, for that matter, a filesystem which reiserfs definitely *is* a hell of a lot faster than.)
(And who can forget those reiserfs design docs, with the somewhat grandiose claims (most of which I agree with, but they were oddly put), and the blue men... a work of immortal genius, or perhaps hallucinogenic drugs ;) )
Filesystems (ext3, reiser, xfs, jfs) comparison on Debian Etch (Debian Posted Apr 26, 2006 13:46 UTC (Wed) by dmantione (guest, #4640) [Link] Stop trolling please. For terabytes of data, xfs and reiserfs are therealistic choices. Reiserfs has excellent reliability, I'd say better than xfs, and excellent resize and fsck tools.
Filesystems (ext3, reiser, xfs, jfs) comparison on Debian Etch (Debian Posted Apr 26, 2006 16:56 UTC (Wed) by nevyn (subscriber, #33129) [Link] Feel free to read about how bad fsck.reiferfs is. If you want to use reiserfs for the speed, feel free to do so ... just don't pretend you don't need a really good backup strategy.
Filesystems (ext3, reiser, xfs, jfs) comparison on Debian Etch (Debian Posted Apr 26, 2006 21:21 UTC (Wed) by nix (subscriber, #2304) [Link] Oh, yes, I spotted that when it went by for the first time. The idea of `stitch reiserfsish blocks together' makes a lot of sense until you consider loop...
(One possible fix would be to put an fs-specific uuid in every block, but I hope anyone actually trying this realises how crazy it is and shoots themselves for the sake of our disk space.)
Filesystems (ext3, reiser, xfs, jfs) comparison on Debian Etch (Debian Posted Apr 27, 2006 6:06 UTC (Thu) by dmantione (guest, #4640) [Link] That link you post is a big troll. Reiserfsck can indeed rebuild thefilesystem (--rebuild-tree) from scratch by searching the disk. You only use it when no other recovery is possible. However, if no other recovery is possible, the recovery chance is still near 100%, unless you indeed had reiserfs images on your disk, but I doubt that is the case for many people.
No, I don't use reiserfs for speed. It just happens that it contains a
Filesystems (ext3, reiser, xfs, jfs) comparison on Debian Etch (Debian Posted Apr 27, 2006 13:08 UTC (Thu) by erich (subscriber, #7127) [Link] --rebuild-tree didn't work for me, and reiserfsck made things actually _worse_. It's crap. Or it was back then, when I stopped using reiserfs.
Filesystems (ext3, reiser, xfs, jfs) comparison on Debian Etch (Debian Posted Apr 27, 2006 13:54 UTC (Thu) by dmantione (guest, #4640) [Link] Yes, --rebuild-tree rebuilds the entire filesystem, so if it fails thefilesystem is not accessible because the old tree is no longer available. This situation remains until a successfull rebuild has been done, so investigate what the cause of the failure is and try again.
Filesystems (ext3, reiser, xfs, jfs) comparison on Debian Etch (Debian Posted Apr 29, 2006 10:56 UTC (Sat) by nix (subscriber, #2304) [Link] This is, of course, an extremely... peculiar design. The robust approach would be to build a new tree in parallel with the old (as long as space was available and the old tree was undamaged enough to determine which blocks were free), then switch over to the old atomically.
This is harder, but it does make the thing fail-safe.
Filesystems (ext3, reiser, xfs, jfs) comparison on Debian Etch (Debian Posted Apr 27, 2006 13:06 UTC (Thu) by erich (subscriber, #7127) [Link] reiserfs may have excellent reliability...... unless it crashes and trashes your whole filesystem, that is.
Happended to me, and if you do a poll on a major linux channel you'll find tons of people burnt by reiserfs.
Filesystems (ext3, reiser, xfs, jfs) comparison on Debian Etch (Debian Posted Apr 28, 2006 4:43 UTC (Fri) by zooko (subscriber, #2589) [Link] The trouble is that "doing a poll", or more realistically chatting back and forth and swapping war stories, is a terrible way to figure out the truth about things. Humankind has recently developed a set of alternate techniques to figure out the truth about things, which collectively go under the rubric of "science". The older technique would best be titled "folklore".
It is part of Linux culture to sit around and swap stories and form sort of a group consensus on things which are otherwise not measured or analyzed. Does the group consensus usually settle on the truth? Who knows. Folklore is often right. Sometimes not. I'll withhold judgment until I see something better.
(The article linked to in this thread that did fault injection and source analysis of myriad possible failures is an example of something better.)
Regards,
Zooko
ReiserFS's stability is actually quite good Posted Apr 26, 2006 14:07 UTC (Wed) by gmaxwell (subscriber, #30048) [Link] Please read http://www.cs.wisc.edu/adsl/Publications/iron-sosp05.pdf.
ReiserFS's handling of exceptional conditions are actually head and shoulders above the competition. The conservative policy of panicking rather than running with corrupted data when something unexpected happen, combined with the dislike of Hans by some of the more notable linux personalities has resulted in a perception which is diametrically opposed to reality.
ReiserFS's stability is not actually quite good Posted Apr 26, 2006 21:16 UTC (Wed) by nix (subscriber, #2304) [Link] I'm thinking more of the three completely chewed-up news filesystems reiser3 gifted me with (each accompanied by a nice panic-with-no-reboot-even-though-I-asked-for-it, killing a colo box which otherwise went to great lengths to avoid anything that might take it down; UPS, RAID, the lot), and the half-a-dozen panics for no obvious reason that I've also been hit with (on completely functional hardware). I don't dare use reiserfs on anything but news fsen, and even there have stopped using it on any machines which I care about them not panicking under any load (like, uh, an expiry run). It's caused me more trouble in the few places I've used it than every other filesystem I've ever used, on *any* Unix system.
reiserv3 is nearly unmaintained, and shows it. My experience has been that it's an unreliable and decidedly dangerous fs which does more to reduce system stability than any other major filesystem you could use (no, umsdos is not major in that sense).
And as for that vaunted failure case: panicking on any tiny error is *utter* stupidity if the only flaw is on a relatively unimportant filesystem! The only advantage of panicking on failure is that it spares the fs developers from having to do any analysis of failure modes to determine if the system could potentially stay up under a given mode. Other filesystems manage it. Why can't reiserfs?
ReiserFS's stability is not actually quite good Posted Apr 26, 2006 21:24 UTC (Wed) by nix (subscriber, #2304) [Link] Lest I seem excessively harsh, I'm only talking, oh, ten panics in four years here. But still that's ten panics more than ext2, ext3, or xfs have ever hit me with. (ext3 even coped on a machine with RAM so bad you couldn't md5sum a 10Mb file twice in a row and get the same answer!)
(I had major disk corruption on ext3 once, but when your disk drive decides to put blocks somewhere other than where the filesystem asked for them to go, you have little option but to get a new disk and restore from backup, really...)
Even with reiserfs, Linux is damned stable. But reiserfs doesn't help.
ReiserFS's stability is not actually quite good Posted Apr 27, 2006 13:36 UTC (Thu) by dmantione (guest, #4640) [Link] <I>I had major disk corruption on ext3 once, but when your disk drivedecides to put blocks somewhere other than where the filesystem asked for them to go, you have little option but to get a new disk and restore from backup, really...</I>
Yes, you would prefer to restore from backup.
However, if you did not backup the data, either due to cost/benefit
In such a situation the reiserfsck --rebuild-tree option, which you
* Connect a new disk with the same size or larger as the failing disk.
Your actual data loss depends on the severity of the corruption. I've had
It is easy to bash the --rebuild-tree option because it is not 100% safe
ReiserFS's stability is not actually quite good Posted Apr 27, 2006 23:43 UTC (Thu) by dlang (✭ supporter ✭, #313) [Link] so you are saying that reiserfs knows better then the admin. personally I won't use software that decides it knows what I should do better then I do (good defaults are one thing, engineering the product to not allow for choice in another)
you are overlooking the fatal flaw with rebuild tree, namely it grabs ALL blocks that look like they are reiserfs tree blocks, unfortunantly with loop mounted filesystems you can have many filesystems on one partition (the main one, and then the ones that are images in files), rebuild tree will try to combine them all.
ReiserFS's stability is not actually quite good Posted Apr 28, 2006 5:32 UTC (Fri) by dmantione (guest, #4640) [Link] Since it is an optional feature, and you are not required to use it, I donot see the problem.
ReiserFS's stability is not actually quite good Posted Apr 29, 2006 11:00 UTC (Sat) by nix (subscriber, #2304) [Link] I can't imagine *any* filesystem saving my data if the disk has been placing blocks in the wrong place for five minutes during intense writes (which was what happened here).
reiserfsck is not going to be any luckier here than any other repair tool because the data is *overwritten*. --rebuild-tree is not magical (any tool which can be misled by the *contents of files* deserves a rather different word to describe it).
ReiserFS's stability is not actually quite good Posted Apr 28, 2006 4:46 UTC (Fri) by zooko (subscriber, #2589) [Link] You would prefer that the filesystem optimistically ignore errors when it can and the admin can restore from backups if that goes bad. This is a reasonable strategy for some workloads, where availability is more important than correctness in the short term, and the admin can manually restore correctness in the long term. If you have such a workload, you should probably not user ReiserFS. If on the other hand you have a workload where correctness is more important than availability, then ReiserFS would be a good tool for that use.
Regards,
Zooko
ReiserFS's stability is not actually quite good Posted Apr 29, 2006 11:02 UTC (Sat) by nix (subscriber, #2304) [Link] No, of course not. If an error is detected, the FS should go read-only as far as is possible so that data recovery can continue and the system can at least stay mostly running.
This is trivially possible: ext2fs and ext3fs both do it. That reiserfs does not is not to its credit.
ReiserFS's stability is not actually quite good Posted May 8, 2008 15:57 UTC (Thu) by zooko (subscriber, #2589) [Link] Hello. Two years ago was the last comment in this thread, and here I am to add another one. :-) My comment is that this fault-injection analysis: https://www.cs.wisc.edu/wind/Publications/iron-sosp05.pdf says that what ext2fs and ext3fs do for many of the errors that they measured is nothing, i.e. carry on as if nothing happened. What reiserfs does for those same errors is stop. You could argue that switching to read-only mode would be better than stopping. I don't know about that. Perhaps the "propagate" option in iron-sosp05.pdf would be better than the "stop" option because then outer layer code (i.e. the kernel or even userland) can detect the error and remount read-only. But, as far as comparing the safety of ext2fs and ext3fs vs. reiserfs, the iron-sosp05.pdf document seems to make it clean that ext2fs and ext3fs err on the side of increased availability at the cost of higher risk of corruption, where reiserfs errs on the side of increased correctness at the cost of higher risk of unavailability.
ReiserFS's stability is not actually quite good Posted May 8, 2008 22:18 UTC (Thu) by nix (subscriber, #2304) [Link] I'm still here :) That increased availability is important. As I see it, there are two types of file storage one might be interested in. There's files for which availability is more important than correctness-of-content-under-errors, and there are files where integrity is all. The former case should be handled by detecting errors and going read-only (i.e. what ext2+ does, only perhaps with added integrity hashes so you can spot more failure cases). The latter case should be handled by making *the filesystem objects that are corrupted* unreadable (not the whole disk unless there's so much corruption that you can't be sure of anything). Files that satisfy the former constraint are far more common than those that satisfy the latter, because you almost always want the fs to be mountable so you can recover as much as possible before hitting the backups. Files that satisfy the latter constraint... well, I'm trying to think of any and I'm coming up cryptographic keys. (Definitely not financial data: I work in that industry, and what matters there is availability above all. If one bit is flipped you cope and carry on, you don't go unavailable: after all, your competitors haven't stopped just because you're having system problems...) Of course neither fs satisfies these constaints: reiserfs stops too hard (and panics the whole machine!), ext* doesn't spot enough failures before they cascade into something horrid. Recently I had a failure mode where the heads didn't bother to move after a journal write, leading to rubbish dumped into the journal. The drive went more demented shortly after that, mashed another fs, and the machine got rebooted... and ext3 thought ooh, we can just roll the journal forward! Oops, wrong, that scattered more corruption around, and because no fsck had been done the first we knew of it was when a dozen NFS mounts from that filesystem suddenly went read-only. e2fsck did a sterling job and got essentially everything back, even though we had part of an inode table claimed simultaneously by the journal inode and by a bunch of logfiles, with a bunch of logfile output written into it in place of inodes. (e2fsck wasn't stupid and realised that this was in an inode table so the other two files must be liars. Obviously we lost the metadata for most of those files, but all the important stuff came back OK. Amazing.)
Filesystems (ext3, reiser, xfs, jfs) comparison on Debian Etch (Debian Posted Apr 26, 2006 10:32 UTC (Wed) by ikm (subscriber, #493) [Link] > I was biten by this bug myself quite a few times: system looked "just fine"> at first glance but SHA1sums of files were different.
So what? This is a documented behaviour. Only the metadata is preserved, the actual data may be lost. This is also true for most of the other journalled filesystems. And XFS here actually has an advantage over many of them: it guarantees that the data that was lost will be filled with zeroes, instead of any garbage, which can leak sensitive information and so on.
Filesystems (ext3, reiser, xfs, jfs) comparison on Debian Etch (Debian Posted Apr 26, 2006 11:53 UTC (Wed) by forthy (guest, #1525) [Link] > So what? This is a documented behaviour. Only the metadata is> preserved, the actual data may be lost. From a user's point of view, this is not acceptable. And this is neither true for ext3 nor for ReiserFS, which can keep both data and metadata consistent. Just take off your filesystem developer's hat, and take the user's hat on: What's a file worth that sits there, has the correct name, the correct length, the correct date, no warning sign, and just garbage in? Nothing! It's worse than having that file deleted, because if it's deleted, you know it's gone. If it's just tampered, and you don't know it is, you are toast. So stay away from filesystems that promise you just "metadata integrity", because it buys you nothing. Either go full way (data+metadata journaling or at least ordering), or no way (completely fscking random rubbish in case of a crash, because then you at least know that it's completely fscking random rubbish, and you have to dig out your backup).
Filesystems (ext3, reiser, xfs, jfs) comparison on Debian Etch (Debian Posted Apr 26, 2006 13:02 UTC (Wed) by ikm (subscriber, #493) [Link] > From a user's point of view, this is not acceptable.
Sure, I actually agree. Just that it's not a bug, but a documented behaviour. OTOH, it should say it more clearly, and the fs itself should not be PRed as a powerfail-safe solution.
Filesystems (ext3, reiser, xfs, jfs) comparison on Debian Etch (Debian Posted Apr 27, 2006 13:16 UTC (Thu) by erich (subscriber, #7127) [Link] Metadata integrity buys you fsck time. Thats what it is about, not having to fsck your terabytes (whatever) after a crash.
Ext3 and I guess reiserfs are usually not using full data journalling either, because that is much slower. ext3 can, but I think it comes with a serious speed penalty.
Just get a log of your fsck on reboot, and you should have information on which files were trashed, and restore them from backup if they aren't okay.
If an application needs to ensure data integrity, it should be handled in the application, not the filesystem. The application can usually do this much more efficiently. Or at all.
A database file will be changing all the time, probably only being in a completely "consistent" state only when the database server is actually shut down properly. What does data journalling buy you, if you need to be able to handle application crashes and such anyway, too?
Filesystems (ext3, reiser, xfs, jfs) comparison on Debian Etch (Debian Posted May 6, 2006 13:30 UTC (Sat) by anton (subscriber, #25547) [Link] Metadata integrity buys you fsck time. Thats what it is about, not having to fsck your terabytes (whatever) after a crash. I don't think that fsck could do anything for data integrity, so that sounds like nonsense. Also, with journaling, fsck time is much less important. With conventional and journaling file systems, metadata-only integrity is a way to achieve better performance and/or lower complexity. Ext3 and I guess reiserfs are usually not using full data journalling either, because that is much slower. ext3 can, but I think it comes with a serious speed penalty. And last time I looked, full data journaling was discouraged for ext3 (apparently that part was no longer maintained). If an application needs to ensure data integrity, it should be handled in the application, not the filesystem. The application can usually do this much more efficiently. Or at all. What does data journalling buy you, if you need to be able to handle application crashes and such anyway, too?What happens on an application crash is completely different from what happens on an system crash (power outage, OS crash etc). You can write an application such that it will not lose data on an application crash, but it can still lose data in case of a system crash (e.g., if the programmer forgot some fsync()s). And resilience against application crashes can be tested much better than resilience against system crashes. So what I would like to see is a file system with in-order semantics, i.e., where there is no difference between what happens on an application crash and what happens on a system crash. And with the right file system, providing this can be more efficient than littering the applications with fsync()s.
Filesystems (ext3, reiser, xfs, jfs) comparison on Debian Etch (Debian Posted Apr 26, 2006 15:59 UTC (Wed) by khim (subscriber, #9252) [Link] So what? This is a documented behaviour. Only the metadata is preserved, the actual data may be lost.
Then why have logging filesystem at all ? Note: I'm not talking about "fresh" files - they are not expected to survive. But when files created days and weeks ago perish in crash... sorry - this is not what I expect from logging filesystem. In few cases I even had corrupted files which survived first crash but not second one - gosh. Worse then EXT2 without any logging: at least there you can be sure files are there - or not...
judging from what? Administration) Posted Apr 27, 2006 6:35 UTC (Thu) by gvy (guest, #11981) [Link] If from MLs, maybe "never been". In my experience, even if I like and use XFS, it's very reasonable explanation of some data losses experienced when UPSes went out (ISP facility, we weren't -- and still aren't -- able to get notified that the batteries run low).
Filesystems (ext3, reiser, xfs, jfs) comparison on Debian Etch (Debian Administration) Posted Apr 26, 2006 0:31 UTC (Wed) by sbergman27 (subscriber, #10767) [Link] What this article leaves out... what most *every* fs benchmark article leaves out is the fact that EXT3 gives you a much greater level of journalling protection than the others by default. By default, EXT3 gives you metadata journalling and ordered writes of metadata and data. The others only give you metadata journalling by default. And, of the others, only reiserfs gives you the option of anything higher.
XFS, JFS, EXT3, and Resierfs all offer plain metadata journalling. This level basically gives you no more protection than a nonjournalling filesystem beyond the fact that you don't have to run fsck after a power failure. After an unclean shutdown, you are guaranteed that the filesystem structure will be intact, but some of your file data may be garbage. This is the default for XFS, JFS, and Reiserfs.
Reiserfs and EXT3 offer full data journalling. In general, this is the slowest of the journalling levels because you are writing the data twice. Once to the journal (which is fast because the journal is always streamed out sequentially) and once to the ultimate location. It gives you the greatest level of protection. In fact, it gives the same level of protection as mounting the filesystem synchronously. If the application was is told that the data was written, it's as good as written. After a power failure, the filesystem structure is guaranteed to be intact. And the file data is guaranteed to be intact. None of the tested filesystems default to this level.
EXT3 offers an intermediate option called "ordered" mode. It is basically metadata journalling, but adds a constraint that the writes to disk will be ordered in such a way that it gives some an extra guarantee. Not only is the filesystem structure guaranteed to be intact, but the data is also guaranteed to be intact. It may not be the *latest* data that the app was told was written to disk. But it is guaranteed n0t t0 b3 g@rb@g3. i.e. it may be the data that was there before the write request. There is a significant performance penalty, but not as great as for full data journalling. This is (wisely in my opinion) EXT3's default.
Another thing that the article misses is the fact that ext3 reserves 5% of space for its fragmentation avoidance algorithms. So that 92.77% partition capacity really should be 97.77%. You can set the reserved space to 0% if you really want. (But keep in mind that the 5% reserve, like the performance penalty for ordered journalling, buys you something.)
I firmly believe that if ext3 had been designed and maintained by people with a little more marketing savvy, and a little less concern with doing what makes sense from a technical standpoint (like reis... *cough*... er), ext3 would have a much better performance reputation. And I'm glad it wasn't.
(David, I know you know all this, but some budding filesystem benchmarker might read this and get it right.)
Filesystems (ext3, reiser, xfs, jfs) comparison on Debian Etch (Debian Administration) Posted Apr 26, 2006 1:47 UTC (Wed) by dlang (✭ supporter ✭, #313) [Link] keep in mind that these benifits you are listing only count if you are useing a hard drive that allows you to disable the write cache on it (i.e. no IDE drives qualify). if the drive can tell you that the write is completed while it's in the drives memory it's still vundrable to being lost.
since there is a huge performance penalty for doing this it's seldom done even on the drives that support it.
David Lang
P.S. and no, the drives don't use their platter energy to drive the electronics long enough to write their data. as an excercise consider how many seeks could be required to write all the data in a 16M buffer, and how long that could take. then add in the problems of writing to a platter that's slowing down as you write and you start to realize why nobody does that anymore.
Filesystems (ext3, reiser, xfs, jfs) comparison on Debian Etch (Debian Administration) Posted Apr 26, 2006 2:49 UTC (Wed) by sbergman27 (subscriber, #10767) [Link] There is a difference between the benefits "not counting" and the guarantees not being absolute.
But you are correct. Hardware write caching does figure in.
You *can* turn write caching off on most IDE drives but, as you say, there is a considerable performance penalty due to the insane limitation on write request size imposed by the ATA standard. So far as i know, SCSI drives are not so hampered.
P.S. Did I say something about the drive using its platter energy for something?
Filesystems (ext3, reiser, xfs, jfs) comparison on Debian Etch (Debian Administration) Posted Apr 26, 2006 7:28 UTC (Wed) by dlang (✭ supporter ✭, #313) [Link] no, you didn't say anything aobut platter energy, but it's a common misconception that people have about drives and the cache on them
Filesystems (ext3, reiser, xfs, jfs) comparison on Debian Etch (Debian Administration) Posted Apr 26, 2006 15:14 UTC (Wed) by sbergman27 (subscriber, #10767) [Link] I saw refernces to that misconception twice yesterday. Oddly, the last time I encountered it (that I remember) was in 1979 when my first computer science instructor made reference to the "fact" that the drive did that. I believed it at the time, and then later decided it was urban legend.
I notice in your previous post you indicate that they don't do that "anymore". Did they really used to do that?
Also of note, and sorry, I don't have a ready link, I was once struck by a thread on lkml in which someone was crash testing different filesystems under Linux. He had a filesystem with lots of writes going on and knew what all the md5sums should be or something like that and would then pull the plug at random times and observe what happened. His question to the list was for someone to please point out what he was "doing wrong". You see, all the other filesystems in the test corrupted files with more or less the same frequency. All except for ext3, that is. It seemed never (or very rarely) to ever corrupt files and he felt that there must be a problem with his methodology. It was explained that ext3 defaults to data=ordered mode and that such behavior was really expected.
My summary of the thread is probably not completely accurate because it's been a while since I read it, but I was struck by the fact that someone observed the difference even though they were not expecting it.
If I can dig up a link, I'll post it.
Filesystems (ext3, reiser, xfs, jfs) comparison on Debian Etch (Debian Administration) Posted Apr 26, 2006 16:15 UTC (Wed) by dlang (✭ supporter ✭, #313) [Link] >I notice in your previous post you indicate that they don't do that "anymore". Did they really used to do that?
I don't know for sure, but every urban legend needs to get started somewhere right? :-)
thinking about it from a practical standpoint, at one point drives had very small buffers (ram was too expensive, and too large to put much on a drive) along with a lot of rotating mass, so it would have been possible to flush that buffer immediatly on power-loss (and the tolorances of the data on disk were loose enough to accept the slight distortion that would result).
Filesystems (ext3, reiser, xfs, jfs) comparison on Debian Etch (Debian Administration) Posted Apr 26, 2006 6:46 UTC (Wed) by joib (guest, #8541) [Link] keep in mind that these benifits you are listing only count if you are useing a hard drive that allows you to disable the write cache on it (i.e. no IDE drives qualify).As was already mentioned, most IDE drives allow you to disable write-back cache. However, many manufacturers consider this operation a warranty-voiding one, since disabling write-back caching causes much more physical writes which significantly reduces the life of the drive. if the drive can tell you that the write is completed while it's in the drives memory it's still vundrable to being lost. since there is a huge performance penalty for doing this it's seldom done even on the drives that support it. Fortunately, you can have your cake and eat it too. The trick is to implement IO barriers using the CACHE FLUSH and/or FUA commands. That way you can have the performance and MTBF benefits of write-back caching while still having a safe fsync() (safe as in doesn't return before data is on the platters). Also note that the IO barrier rewrite referenced above was included only from 2.6.16+; I don't know how previous kernels did it.
Filesystems (ext3, reiser, xfs, jfs) comparison on Debian Etch (Debian Administration) Posted Apr 26, 2006 7:29 UTC (Wed) by dlang (✭ supporter ✭, #313) [Link] I hadn't caught the fact that the IO barriers had made it into the kernel, I knew they were being worked on. prior to that going in the only option the kernel had was to stop all IO to the drive while issueing a full flush to it.
Filesystems (ext3, reiser, xfs, jfs) comparison on Debian Etch (Debian Administration) Posted May 6, 2006 13:56 UTC (Sat) by anton (subscriber, #25547) [Link] >hard drive that allows you to disable the write cache on it (i.e. no>IDE drives qualify).
hdparm -W0 works on any IDE drive I have tried it on (and we run all
>since there is a huge performance penalty for doing this
Well, I recently tried it:
Disk: SAMSUNG SV1204H, ATA DISK drive
So the penalty was a factor of 2 in this case.
>as an excercise consider how many seeks could be required to write all
The drive could have a 16MB spare area for just this case, and dump
Filesystems (ext3, reiser, xfs, jfs) comparison on Debian Etch (Debian Administration) Posted Apr 26, 2006 16:05 UTC (Wed) by Duncan (guest, #6647) [Link] > What this article leaves out... what most *every*> fs benchmark article leaves out is the fact that > EXT3 gives you a much greater level of journalling > protection than the others by default. By default, > EXT3 gives you metadata journalling and ordered > writes of metadata and data. The others only give > you metadata journalling by default. And, of the > others, only reiserfs gives you the option of > anything higher.
> Reiserfs and EXT3 offer full data journalling.
> EXT3 offers an intermediate option called "ordered"
Actually, reiserfs offers ordered mode as well. It didn't originally, but
That said, if you didn't know it was there (I knew in part due to LWN
ReiserFS: md_d1p1: using ordered data mode
I remember looking to see that I was using ordered shortly after
Oh, for anyone interested, those patches /did/ make reiserfs far more
Duncan
Filesystems (ext3, reiser, xfs, jfs) comparison on Debian Etch (DebianAdministration) Posted Apr 26, 2006 21:41 UTC (Wed) by nix (subscriber, #2304) [Link] Yeah, I haven't lost any reiserfs filesystems since then anyway.
(It's just panicked instead. Three times.)
Filesystems (ext3, reiser, xfs, jfs) comparison on Debian Etch (DebianAdministration) Posted Apr 27, 2006 9:23 UTC (Thu) by Wol (guest, #4433) [Link] I believe there's also an experimental filesystem called TuxFS. It guarantees integrity without journaling :-)
Basically, it never overwrites a live file. Any changes, it writes the modified block out in full somewhere else. Then it rewrites the updated file header out in full somewhere else. Then the directory header ...
Until finally it rewrites the root block. The only time there's any danger is if it goes down while writing the root block. At all other times, the root block is pointing at a completely valid filesystem. If the system crashes, all updates since the last root block update are lost because the modified blocks are orphaned. But all previous data is safe, because data is never modified "in situ".
Filesystems (ext3, reiser, xfs, jfs) comparison on Debian Etch (DebianAdministration) Posted Apr 27, 2006 12:01 UTC (Thu) by nix (subscriber, #2304) [Link] Downside: a minor problem with fragmentation. :)))
(The extra disk overhead can be disregarded as long as you have decent amounts of cache, because disk writes are generally localized in the directory tree anyway. At least they are if atime updates are disabled. Using a filesystem like this with atime updates enabled strikes me as... perhaps unwise.)
XFS is not for the masses Posted Apr 26, 2006 13:11 UTC (Wed) by hmh (subscriber, #3838) [Link] XFS' data loss on powerloss scenarios is still true. It is caused by (as others said) the fact that data writes are not protected unless you call fsync(), and I am not really sure they are fully protected even then.
What people often forget is that XFS has delayed write allocation, which increases the window for such data loss even more as a trade-off for performance. Given that making sure that XFS isn't corrupted requires xfs_repair and is not easy to do on the / partition on most default installs, I would never recommend anyone who doesn't really know what he is doing to use it (and yes, XFS does get corrupted although it is a very rare event, and usually due to bad memory on the system board).
Ext3 will also have corruption/data loss issues on powerloss, depending on your disks and their caches, but the window is much smaller. At least it is very fsck-on-boot-friendly...
Maybe the new SATA drives (which seem to be better at honouring cache flush requests) and IO barriers will put an end to most ext3 corruption on the typical desktop, but I wouldn't hold my breath.
I won't comment on reiserfs, I have little experience with it.
XFS is not for the masses Posted Apr 26, 2006 21:43 UTC (Wed) by nix (subscriber, #2304) [Link] Ah well, with early userspace it's easy to fsck before mounting /, and when early userspace lands for everyone, hopefully there'll be a fsck in there, or klibc will be capable enough for e2fsprogs's fsck...
XFS is not for the masses Posted Apr 26, 2006 21:54 UTC (Wed) by hmh (subscriber, #3838) [Link] You can fsck / with ext2/ext3 just fine. It is XFS' utter useless fsck semanthics (do nothing) which are trouble. You need xfs_repair in the early/repair userland to deal with that.
I'd be much better if XFS' (and anyone else also doing this) would just stop with the impress-the-management-with-lies politics and implement fsck as it is meant to be implemented, instead of as a NOP.
XFS is not for the masses Posted Apr 27, 2006 6:23 UTC (Thu) by nix (subscriber, #2304) [Link] Why not wrap up xfs_repair with a fsck wrapper? (I thought someone had done that, actually.)
XFS is not for the masses Posted Apr 27, 2006 12:02 UTC (Thu) by nix (subscriber, #2304) [Link] There is something unpleasant about fscking *any* live filesystem to me: the 'recommend reboot' return code from e2fsck is a symptom of this unpleasant special case. Well, now we can zap it :)
|
|
||||||||||||