Weekly Edition
Return to the Distributions page
| |||||||||||||
Ext4 to be standard for Fedora 11, Btrfs also included (heise online)
heise online
notes plans to include Ext4 and Btrfs
in Fedora 11.
"According to current plans, version 11 of Fedora, which is expected to arrive in late May, will use Ext4 as its standard file system. That's what the Fedora Engineering Steering Committee (FESCo) recently decided, following a heated discussion in an IRC meeting. If however Ext3's successor encounters big problems with the pre-release versions of Fedora 11, the developers will dump that plan and revert to Ext3.
So the Fedora Project is going one step beyond Ubuntu version 9.04 (Jaunty Jackalope), which as things currently stand will offer Ext4 as an install time option, though the installer will still use Ext3 as its default file system."
(Log in to post comments)
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 22, 2009 23:03 UTC (Thu) by BillyCrook (guest, #53586) [Link]
"Fedora Project is going one step beyond Ubuntu"
That's hardly news. Fedora has always taken the lead, and always will.
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 23, 2009 3:33 UTC (Fri) by salimma (subscriber, #34460) [Link]
There are some features that Ubuntu get first -- for example, fast user switching and upstart (which is developed by Ubuntu).
When it comes to kernel features, though, your observation is more or less true. Red Hat employs more core kernel developers than other distributions, and as a result, Fedora tends to get them first (though they tend to favor the extN filesystems, so this is not necessarily true for other file systems).
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 23, 2009 7:46 UTC (Fri) by xoddam (subscriber, #2322) [Link]
> "fast user switching"
What, like "startx -- :1" followed by Ctl-Alt Fn to switch? This is a feature of X. Predates both Fedora and Ubuntu, I wouldn't be surprised if it worked on UnixWare pre-Linux.
That you can now achieve the same in Ubuntu with a mouse is not exciting.
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 23, 2009 9:08 UTC (Fri) by RobWilco (guest, #40828) [Link]
I'll tell my Mom that she can use "startx -- :1" followed by Ctl-Alt Fn to switch" instead of her mouse. I will also tell her that ext4 and btrfs are in Fedora and she wil be so happy she will cook a cake for me.
Ok, Kudos for Fedora for this audacious step, it is a good thing. Kudos for Ubuntu for putting efforts in the notification system. It is needed too.
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 23, 2009 12:15 UTC (Fri) by epa (subscriber, #39769) [Link]
FWIW, my mother has been using 'gdmflexiserver --no-lock' (run from a panel icon) to do fast user switching for several years now. The improvement in recent Fedora releases was to make fast user switching more polished somehow, not to make it possible.
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 25, 2009 21:49 UTC (Sun) by salimma (subscriber, #34460) [Link]
That feature was specifically disabled from Fedora's GNOME, because it previously did not integrate well with Fedora's management of system devices (only the user logged on locally get access to devices such as the sound card).
It is now turned on, after the development of PolicyKit.
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 23, 2009 9:05 UTC (Fri) by buchanmilne (guest, #42315) [Link] fast user switching KDE has had this feature since 2003, I think SUSE was the first distribution to ship with it, all before Ubuntu even existed. If you mean support in GNOME, well, it seems that Fedora was concerned with taking the security issues into account, which Ubuntu didn't (at the time KDE got this feature in 2003, there was no ConsoleKit, less restrictive pam_console permissions and device-groups were the only means to make this work in a semi-secure way). Fedora has also added ConsoleKit support to KDM.
upstart (which is developed by Ubuntu)Starting this discussion (who develops what) is dangerous for Ubuntu supporters ...
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 23, 2009 17:51 UTC (Fri) by jdahlin (guest, #14990) [Link]
Besides Upstart was originally developed by an Ubuntu developer in his free time, if that make a difference.
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 23, 2009 23:51 UTC (Fri) by Burgundavia (guest, #25172) [Link]
So was PackageKit and whole host of RH-driven things...
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 25, 2009 22:11 UTC (Sun) by jspaleta (subscriber, #50639) [Link]
You mean it was not originally developed on Canonical paid time?
Hmm.
-jef
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 26, 2009 0:04 UTC (Mon) by nix (subscriber, #2304) [Link]
*sigh* why the *hell* does this matter? *Linux* wasn't developed on 'paid
time'. Being developed on 'paid time' says nothing about the quality of the software in question (actually, I'd tend to trust it less if it was paid, because there were probably deadline pressures).
I suspect everyone on LWN would breathe a sigh of relief if you just
For goodness sake *give it a rest*.
(The annoying thing is I can't even killfile you because a few percent of
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 26, 2009 17:29 UTC (Mon) by jspaleta (subscriber, #50639) [Link]
Linux isn't developed on paid time? Noone spends time contributing to the linux kernel while on the clock for some corporate entity?
hmm.
-jef
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 26, 2009 22:29 UTC (Mon) by nix (subscriber, #2304) [Link]
Linux didn't start as a paid project, which is *exactly* what you
were 'hmmm'ing about with regard to upstart.
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 26, 2009 23:27 UTC (Mon) by jspaleta (subscriber, #50639) [Link]
Yes you are right, apologizes for the last comment as it was reading what you wrote in the wrong context.
Yes "Originally developed" was the phrase used. So yes I was hmm'ing about Canonical's role there. Why am I hmm'ing? Because I was under the impression that this was a Canonical developed codebase. Why was I under that impression?
My reading of http://upstart.ubuntu.com/wiki/CopyrightAssignment
The developer even talks about the copyright assignment to Canonical here:
Saying its the same assignment required by bzr, and again my understanding is that bzr was originally developed on Canonical's dime. This doesn't prove that Canonical paid for the original Upstart development time, but its what I took away from the requiring the Canonical copyright assignment.
And there's nothing inherently wrong with the copyright assignment either. If the original developer wants to sign over copyright assignment to Canonical and give Canonical the power to re-license the codebase on whatever terms it sees fits as a corporate entity, even if the codebase originated on volunteer time, that's totally within his rights to do.
Also my previous reading of
specifically lists the original developer as a Canonical employee instead of just a Ubuntu community member and that again implied to me that it was a Canonical development.
Why does it matter? I fully expect that the History of Ubuntu to be a category on Jeopardy in ten years so I want to make sure I get all the trivia correct.
-jef
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 27, 2009 5:34 UTC (Tue) by paulj (subscriber, #341) [Link]
Please could you shut up about Canonical/Ubuntu? You're an embarrassment to Fedora with your behaviour here
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 27, 2009 6:50 UTC (Tue) by jspaleta (subscriber, #50639) [Link]
Sadly I'm also an embarrassment to my parents too.
Look I didn't bring it up, I was just expressing the fact that the assertion that Upstart was initially a volunteer effort without Canonical backing seemed odd. If I were angling to beat up Canonical about it, I'd be on the other side of the discussion, trying to argue that this is an example of Canonical not contributing...which the exact opposite of what I thought the situation was.
And point of fact, another person has also chimed in that the original assertion is false. Unlike most everyone else, I've putting my opinion in context by citing the reference material which influenced how I formed that opinion. It would have been nice if the original assertion and come with an authoritative reference. Are you okay with people making unsupported assertions and treating them like facts? I'm more than happy to be shown an authoritative source material that Canonical didn't pay for the development time for Upstart, but until then I'm giving Canonical the benefit of the doubt.
-jef
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 27, 2009 7:35 UTC (Tue) by paulj (subscriber, #341) [Link]
Jef,
I have no doubt that you believe you're arguing objectively and in good faith, on important questions regarding Canonical.
However, it's completely missing the point: You are a prominent member of the Fedora community (possibly even an officer of it of some sort, I don't know). It reflects very badly on you - and anybody you represent, by extension - when you wage a bad-mouthing campaign against rivals on forums. However well-founded your bad-mouthing might be doesn't matter[1].
Your socio-political standing means it is NOT YOUR PLACE to launch such direct criticisms. Rather make them indirectly by promoting those *positive* aspects of your distro which contrast with what you feel is lacking in others. E.g. "More developers get paid to work on Fedora", or whatever your point is. Leave any direct criticism to 3rd parties (e.g. LWN editors) who have at least semblence of impartiality.
Please, please stop..
1. On the flip-side: if it's not well-founded, it reflects even more poorly on you. Though I suspect many people now ignore any of your comments containing the words "Canonical" or "Ubuntu"..
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 27, 2009 8:02 UTC (Tue) by jspaleta (subscriber, #50639) [Link]
I know my place... my place is in the kitchen...barefoot.
-jef
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 29, 2009 14:18 UTC (Thu) by hummassa (subscriber, #307) [Link]
> Your socio-political standing means it is NOT YOUR PLACE to launch such direct criticisms.
I hadn't noticed that Fedora was in a state of war, or any kind of emergency that precludes its members, albeit high-standing ones, of criticizing it.
Lucky Debian, in which _really_ high-standing ones can criticize the project at will and with impunity.
Meh.
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 27, 2009 23:12 UTC (Tue) by cry_regarder (subscriber, #50545) [Link]
Child: Mommy Mommy! The teacher is picking on me again!
Mom: What about?
Child: Not doing my homework!
Mom: Did you do your homework?
Child: Ummm. No...
Mom: Well?
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 27, 2009 0:02 UTC (Tue) by keybuk (subscriber, #18473) [Link]
No it wasn't.
It was originally developed as a feature for Ubuntu 6.10, and very much in paid time by a Canonical employee.
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 23, 2009 8:33 UTC (Fri) by michaeljt (subscriber, #39183) [Link]
I would be really happy as an Ubuntu user if Fedora always took the lead and Ubuntu concentrated on being second with improved reliability (both would be valuable services to the community). To quote Seymour Cray - "One of the problems of being a pioneer is you always make mistakes and I never, never want to be a pioneer. ItÂ’s always best to come second when you can look at the mistakes the pioneers made." Unfortunately, Ubuntu don't seem to agree with me :)
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 23, 2009 9:44 UTC (Fri) by Janne (guest, #40891) [Link]
Which is more important: To take cutting-edge features and shipping it, or making sure that the not-so-cutting-edge features work and that the entire system is smooth and easy to use? Different people might have different answers to that.
Why do we only consider technical features as proper features, whereas polish and ease of use are not?
Features vs. non-functionals... Posted Jan 23, 2009 10:08 UTC (Fri) by Cato (subscriber, #7643) [Link]
Unfortunately this 'features matter most' attitude is endemic to the whole computer industry. Almost everyone ignores the 'non-functional requirements' such as reliability, security, usability, etc. A couple of examples:
- Microsoft Office beat the competition largely by adding features more quickly - since software reviews focus on what they can easily measure in a few days usage, which means features and ease of learning the new features (not long term usability), it tended to always win the reviews and gain market share until it was dominant.
- in the Ubuntu world, many people jump on the latest release even if they don't really need the new features - good thing for testing, but not so good if they simply need a reliable desktop system, for which the Ubuntu stable releases would be more appropriate.
- traditionally Windows had more features than Linux, but Linux was much more reliable and secure - however, most people chose Windows based on features. Linux is approaching feature parity for desktops so this is no longer so valid.
If it was common practice to quantitatively measure and review software based on these non-functionals, the world would be a different place, but it would also take a lot longer to review software...
Features vs. non-functionals... Posted Jan 24, 2009 5:15 UTC (Sat) by dkite (guest, #4577) [Link]
>traditionally Windows had more features than Linux, but Linux was much
more reliable and secure - however, most people chose Windows based on features
Quite regularly someone shows me something that they did with windows that
I would be interested in a survey of users showing how the capabilities of
Derek
Features vs. non-functionals... Posted Jan 29, 2009 10:45 UTC (Thu) by Cato (subscriber, #7643) [Link]
I can't think of many things you can do with Windows that you can't do with Linux - really I was making the point that out of the box, Windows has some additional features including hardware support, good usability, consistent UI, sound that always works, etc.
Generally I can load up a Linux box with far more apps and it's still stable. For example, my main Linux desktop can do email, browsing, video, VoIP, VMware, web serving and OpenNMS all at the same time, but I wouldn't dare do all that on Windows. Hardware support can be more problematic if you don't choose the kit carefully, but generally most things work "out of the box".
Linux also scales down much better of course - I've recently been hugely impressed by SliTaz, which needs only a tiny amount of RAM yet still supports Firefox and other modern apps, and has a very light Debian-like package system, and by Crunchbang, which is a very light Ubuntu (much less than Xubuntu) while also coming with Java and Flash pre-installed. Either of these is great for older PCs with 100MB or 200MB of RAM respectively, on which XP would really struggle and Vista would not even install.
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 22, 2009 23:11 UTC (Thu) by giggls (subscriber, #48434) [Link]
Using xfs for many years now, I was wondering if Ext4 does have any advantages compared to xfs.
One thing I always found a very handy thing to have is a very stable dump implementation in form of the xfsdump utility while the generic dump is somewhat infamous for not working for most of the time.
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 22, 2009 23:18 UTC (Thu) by sumC (subscriber, #1262) [Link]
Performance wise it does have some advantages. Here is a brief benchmark between EXT4, EXT3, XFS, JFS, and ReiserFS on Ubuntu.
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 23, 2009 0:22 UTC (Fri) by ncm (subscriber, #165) [Link]
It appears these tests were all run on non-rotating storage. Perhaps that's the wave of the future, but the fact needs to be noted prominently. Tests using rotating storage would be more immediately helpful.
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 24, 2009 18:25 UTC (Sat) by Los__D (guest, #15263) [Link]
There is a link to a rotating-storage test on the page.
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 22, 2009 23:29 UTC (Thu) by jwb (guest, #15467) [Link]
ext4 has metadata performance roughly 6-10x faster than xfs in most scenarios. On my machine with a large raid, the time required to copy a kernel source try is 27s with ext4 and 194s for xfs.
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 22, 2009 23:36 UTC (Thu) by kev009 (subscriber, #43906) [Link]
It should at least match performance of XFS in most workloads (similar features including extents, delayed alloc, etc. Search this site for the details you need.), but has the benefit of way more experienced devs than XFS ever will, and is home grown to the Linux kernel. Suffice to say, once the dust settles and ext4 has been incorporated into the major distros it will be far more stable and reliable than XFS could ever dream, even though the latter has been around for quite a while.
This seems to be how FOSS works, where the most popular projects attract the best devs and tend to take the lead and run with it even if it isn't [originally] the best technology. That isn't to say that niche projects don't exist, but for a general purpose FS, XFS really has no advantage over ext4. The more advanced XFS features like freezing are being ported to other FSes anyways.
In summery, ext4 is a worthy contender in the FS space and should handle most user's needs better than any of the other general purpose FS, at least until Btrfs stabilizes. Btrfs might not even benchmark as well as ext4, but it will allow for novel features like efficient/safe backup, Time Machine work-a-like, easy multi-volume management, and all the hype of ZFS and co. This tandem of ext4 and Btrfs should keep us content for a few years yet.
I'll link dump to my blog with more FS thoughts: http://www.kev009.com/wp/2008/11/on-file-systems/
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 23, 2009 3:35 UTC (Fri) by salimma (subscriber, #34460) [Link]
I can't wait to run an ext4 root / btrfs home combination. One wonders if Sun's time machine-like slider on OpenSolaris' Nautilus file manager has been contributed back to the GNOME project, and can be used with a different file system backend (i.e. replacing ZFS with Btrfs).
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 25, 2009 8:42 UTC (Sun) by dgc (subscriber, #6611) [Link]
> [ext4] ... has the benefit of way more experienced devs than
> XFS ever will ...
Oh, that's a bit harsh.
Yes, the ext4 developers are experienced and talented.
But who is more talented or experienced *does not matter*.
What matters is that they put their time and effort into
Respect them for providing you with a choice in filesystems
The fact that some filesystem doesn't meet your particular
FYI: Yes, I'm an XFS developer.
ext4 over xfs Posted Jan 23, 2009 3:07 UTC (Fri) by rfunk (subscriber, #4054) [Link]
If ext3 is anywhere near as reliable as ext3, then ext4's big advantage
over xfs is much lower risk of losing your entire filesystem when something goes wrong.
ext4 over xfs Posted Jan 23, 2009 3:54 UTC (Fri) by clump (subscriber, #27801) [Link]
I believe you're referring to mounting ordered mode by default. It's the main reason I use EXT3 nearly everywhere. EXT4 has it, and it's the default just like EXT3.
Found this nugget and others in 2.6.28's "ext4.txt" documentation.
Nope - there are other problems with xfs Posted Jan 23, 2009 10:03 UTC (Fri) by khim (subscriber, #9252) [Link] Actually it all is related to any stress-conditions. Fill the XFS partition with bittorrent-downloaded files (no pre-allocate!) and the only way to save this partition is to backup everything and restore from backup. Yes, it's nasty thing to do with the FS (bittorrent with pre-allocate more- or-less equals to creation of files from pieces in random order and if you fill the FS to 100% there are no way to radically optimize layout - because all files must be sparse or else they'll not fit), but I still expect that any sane filesystem will be slow after that, not dead. XFS failed this test consistently for a few years in row and eventually I've just stopped caring and stopped using it...
ext4 over xfs Posted Jan 23, 2009 13:45 UTC (Fri) by rfunk (subscriber, #4054) [Link]
Well, I was specifically thinking of the other side, how badly xfs has
screwed up filesystems of mine in the past. And I started out as a fan of xfs, having used it on SGIs. But between losing an entire filesystem on an external drive, and getting strings of zero-bytes strewn without warning throughout files on another filesystem, I'm done with xfs.
(BTW, I meant "if *ext4* is anywhere near as reliable as ext3" in my
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 23, 2009 4:10 UTC (Fri) by dlang (✭ supporter ✭, #313) [Link]
to those saying that ext3 is so much faster than XFS, is this with ext3 disabling barriers (as it does by default in the name of performance) while XFS honors them?
to those who say ext3 is so much safer than XFS, are you taking the absense of barriers into account?
Barriers, shmarriers... Posted Jan 23, 2009 10:15 UTC (Fri) by khim (subscriber, #9252) [Link] to those who say ext3 is so much safer than XFS, are you taking the absense of barriers into account? Nope. Why should I? I'm not talking about internals or some theory. I'm talking about practice. When files untouched for months are corrupted after energency power switch (this was the case with XFS for many, many years) - this is reason enough to stop using it. It never happened for me with ext3, it happended few times with xfs (when I've tried to actually use it): power down your system at some random point (I did this not on purpose: power in my home sometimes was switched off), check iа your MP3 collection is still Ok (it's not - most files are Ok and have the same sha1sum, but few are destryed) and start reripping stuff from CDs. And if files untouched for months (Ok, they were played, but never modified) were randomly corrupt - how can I trust this filesystem? What can I put on it? Ok, may be this mysterious spring 2007 fix solved this problem - but by then I've stopped caring about XFS and don't have guts to go back and check again. It works in lab tests, it does not work in real life - that's the problem...
Barriers, shmarriers... Posted Jan 25, 2009 11:19 UTC (Sun) by man_ls (subscriber, #15091) [Link] Right, barriers are only a means for an end; as our kind editor told us some time ago corruption is not very likely except in contrived tests, and in practice few people have ever seen it. OTOH it seems that all XFS users have seen corrupted data on a power off, and it hurts. So in a way yes, we are taking the absence of barriers into account.
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 23, 2009 7:54 UTC (Fri) by muwlgr (guest, #35359) [Link]
For long time, there were rumors that XFS is prone to data-loss on crashes (zeroed file blocks instead of written data, etc.). Afraid of this, I never used XFS by myself. The performance is said to be great, but the codebase is said to be too complex, large, and requiring special adaptation to run on Linux.
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 23, 2009 8:32 UTC (Fri) by cyrus (guest, #36858) [Link]
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 23, 2009 9:46 UTC (Fri) by GhePeU (subscriber, #56133) [Link]
The benchmarks I've seen until now (with Phoronix as the only exception, but, well, often their benchmarks are a bit 'off' and not well targeted toward what they want to measure) show similar performances; usually XFS and EXT4 are paired, in some operations one of them is faster than the other but never by significant margins. I'm going to do some limited testing on my own, but I couldn't find the time yet.
With regard to XFS reputation: you can find people complaining that the filesystem ate their data for every single filesystem out there. I've been using XFS for at least two and a half year, on two different machines; my system is pretty stable, but there were a few blackouts and I suffered many lockups when I was testing the bleeding edge X stack last year, and I never experienced data losses or filesystem corruptions.
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 23, 2009 10:13 UTC (Fri) by petegn (guest, #847) [Link]
I have had several cabling problems with hard drives those darn stupid SATA connectors the designer of which needs castrating without any pain relief , but never lost a single bit of information when the stupid cables were re seated using XFS Ext* FS's thou have been an absolute nightmare so much so that everyting i have has been moved to either Reiserfs or XFS and never had major problems
Bring on BTRFS sounds great just hope opensuse adopt it now
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 23, 2009 10:36 UTC (Fri) by Seegras (subscriber, #20463) [Link]
XFS is nice, until the hardware breaks.
Ext2/3/4 has a badblock-list, which XFS hasn't. If the superblock or some inode-table ends up in a bad block, you can't just tell it the block is bad, it should use another one and go on with the fsck.
No, the only way is to copy the whole fs with dd_rescue to somewhere else and fsck there. Very funny if this happens with a really big filesystem (as it happened to me).
And speaking of fsck; to fsck an XFS you need roughly a megabyte per gigabyte in the filesystem. Which also is real fun with huge filesystems; especially if you needed to dd_rescue the fs onto another raid-array on another machine which just happens not to have as much memory as the other one.
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 23, 2009 10:54 UTC (Fri) by dlang (✭ supporter ✭, #313) [Link]
given that modern drives will remap bad blocks on the disk to hide this from the OS, how much use does such a feature get nowdays?
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 23, 2009 17:53 UTC (Fri) by martinfick (subscriber, #4455) [Link]
Only until the HD runs out of spare blocks, then the FS sees it.
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 23, 2009 18:37 UTC (Fri) by tialaramex (subscriber, #21167) [Link]
But the disks are designed to have enough such blocks to last the lifetime of the disk. It will show up in SMART under that "old age" category, like the slow loss of motor performance caused by wearing of the tiny moving parts.
If you're interested because you run disks way beyond their design lifespan then you're in a niche, like the Bad RAM patch that lets Linux map out ranges of DRAM known to be damaged or faulty so that you can use DIMMS which failed QA.
Everybody else ought to be replacing disks that approach the limit on remapped blocks. That's a pretty clear sign something's wrong. Maybe it's not fatal. but it's definitely not good, so replace the disk.
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 23, 2009 19:19 UTC (Fri) by martinfick (subscriber, #4455) [Link]
You are assuming modern disks that even have SMART. I still have disks that don't. The disk can be fine until suddenly one day: bad block!
But, even with modern disks, do you really think the average consumer has even heard of SMART? Do you think they have any clue that their disk needs to be replaced? Do any distros even enable SMART monitoring* by default? Do you think bad blocks only ever appear gradually?
Unless you have a specific good reason to not have bad block relocation, why would you not want it? You shouldn't need checksums either, but they are a good idea. A modern FS should certainly be able to handle bad blocks, I would find it hard to call it modern of it can't.
* I am kind of curious when distros will become proactive and install all sorts of monitoring/smtp stuff by default?
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 23, 2009 20:31 UTC (Fri) by dlang (✭ supporter ✭, #313) [Link]
when did you last purchase a drive that doesn't do bad-block remapping?
The users don't need to know about SMART, they need to know that if the OS ever sees a bad block on the drive that the drive should be replaced. It hasn't been the case for _many_ years that it was normal to have some number of user-visible bad blocks on a drive. you almost have to go back to the days of MFM/RLL/ESDI drives (I think this was 12+ years ago)
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 23, 2009 21:31 UTC (Fri) by martinfick (subscriber, #4455) [Link]
I think my whole point must have gone right over your head? :)
Even my old drives do bad block remapping, but there is no way to know about it until they run out since they do not have SMART!! Given enough time, they will run out or the drive will crash entirely. I have older drives that do remap blocks, but have run out of spare blocks. I no longer use them since they are obviously risky to use, but in the meantime I am happy to have not lost any data until I realized the problem.
Your OS point is idealistic as I tried to explain previously. I don't know of any linux distro that enables smartmon by default, so, no, the average user will not know when the drive is going bad even thought the tools exists to find out. Even if they did have smartmon enabled, do you think the average user looks through logs to see if there are any problems? How would a linux desktop GUI only user ever know?
Not to mention, that you still have not provided a logical reason to NOT perform bad block allocation except for your stance that it is not really needed.
Just to add to this, my personal opinion is that the HD is the wrong place for this feature anyway. It requires you to waste an arbitrary number of blocks as spares. There is no way to get that number right, either you will have allocated too many or too few. Neither case is ideal. Since this really is a software feature (remapping), let the OS know and don't pretend it's a hardware feature. To be honest, it probably belongs at the block layer since it will not make sense to do it at the FS layer if using LVM.
Perhaps LVM should take care of this? If I use LVM, I should be able to have my spare blocks on a separate disk. In fact LVM should probably have thresholds that make it remap the entire disk when it starts to see too many bad blocks! I guess ideally all the layers should be able to handle badblocks somehow. :)
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 23, 2009 22:03 UTC (Fri) by dlang (✭ supporter ✭, #313) [Link]
one point, when a bad block gets remapped, you do (stand a very good chance that you) loose the data on that block.
the problem with having the OS do bad-block mapping is that people use drives with different OS's, and they would all have to know about all the bad blocks, what they were remapped to, etc.
drives used to work this way, the reliability of having the OS do this (even in cases where you weren't dual-booting) was bad enough that at the time it migrated into the drives so that the OS could forget about this, everyone considered this a step forward.
it's far more complicated for the drive to do the remapping than to let the OS do it, so it wasn't just a whim that change things so that every drive manufactured does it internally.
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 24, 2009 12:43 UTC (Sat) by Los__D (guest, #15263) [Link]
When the spares runs out, the drive should be replaced immediately, if you care about your data.
No OS bad block remapping scheme will change that.
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 24, 2009 13:36 UTC (Sat) by nix (subscriber, #2304) [Link]
Yes: but you can see bad blocks appear before any spares have been used at
all, if you only saw them on read, rather than after a rewrite.
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 24, 2009 13:51 UTC (Sat) by dlang (✭ supporter ✭, #313) [Link]
but in that case the correct action isn't to mark the sector as bad, but to try to write to it (so that the sector will be remapped)
however, if the drive really is going bad, doing so could further damage your data
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 23, 2009 23:58 UTC (Fri) by nix (subscriber, #2304) [Link]
If the OS sees a bad block on a drive *after a write* the drive should be
replaced. If it's just a read, then the drive hasn't had a chance to do any bad block remapping yet, so nothing unexpected is going on.
(Using RAID will let you do that rewrite without losing the data that was
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 24, 2009 1:08 UTC (Sat) by EzDi (guest, #56297) [Link]
Exactly, I've had this problem with several drives where I get a bad sector somewhere, often underneath swap, a journal, or some other file system bit, because that's what's being dealt with before the incident that caused it.
Drives often don't remap UNTIL YOU DO A WRITE. So I keep getting errors while trying to mount, fsck, or when doing something to an adjacent sector, until I manually zero out that sector. Then my hard drive is perfect again from the OS's viewpoint.
Of course using bad block lists can often not do the right thing, because they only do a read test...
SMART is not that usefull Posted Jan 24, 2009 13:51 UTC (Sat) by zuki (subscriber, #41808) [Link]
Google did a large survey (100000s disks) of disk failures,
(http://research.google.com/archive/disk_failures.pdf, sec. 3.5.6) and about 1/3 report no SMART errors at all before failing, and only a minority report failures that would allow for early replacement.
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 23, 2009 12:22 UTC (Fri) by ledow (guest, #11753) [Link]
Alright, maybe it's just me, but personally I don't touch any filesystem that isn't years old (or heavily based on one that is years old) or that can't easily be recovered in some fashion to a working filesystem. Data loss is one of the killers for me when it comes to computing... the best solution we have so far is "make copies of it everywhere", which is hardly technical.
I am more than happy to use FAT32 on a hard drive if I know it's going to be used for basic data storage and not much else. I have FAT32 partitions still on my Linux servers. I always have the safest ext? filesystem on any partitions that I boot from or need linux permissioning on. This, at the moment, is ext3. Give it a couple of years and it might be ext4. In the absence of either option, I would choose ext2.
NTFS is an overly complicated solution to certain problems but the fact that I now have read-write safe tools that run on Linux too also starts to fold this into the "useable for certain purposes" area. (I never even *touched* CaptiveNTFS because it was such a hack, something was bound to break).
However, ALL other filesystems, whatever their origin, are to me nothing more than experimental versions that I don't consider my data safe on. I admin school networks for a living and if I lose data in any way, shape or form, it could easily prevent the school from opening that day (you have no idea how much stuff runs off school systems now) - that would be my job out of the window.
So, obviously, we have backups galore and nice seperation between systems but do you really think that ANYTHING is going to go onto a filesystem that I can't read back in Linux and at least have a stab at fixing? I'm wary even of read-write drivers for the filesystems I trust that don't have an established history.
ext3 > ext2 > FAT32 > NTFS > anything else
at the moment, with ext4 languishing in the last category until it becomes a mainstream hit.
Special purpose filesystems? They're for special purposes and you run your own risks there. And to be honest, for 99% of stuff, one of the above would do just as well.
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 23, 2009 20:32 UTC (Fri) by kev009 (subscriber, #43906) [Link]
Thinking NTFS on anything other than Windows is more stable than ext2,3, or even 4 is mad. The read/write is fairly recent. I've never hit bugs in any file system I've used, but Linux r/w NTFS speed is terrible to this day.
And Fat32? Not only is it behind the times and inefficient, but it too is not a native file system to Linux which would worry me a bit.
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 23, 2009 20:53 UTC (Fri) by dlang (✭ supporter ✭, #313) [Link]
ext4 (especially with the new disk format and new features) was only declared 'stable' by it's developers in this last kernel release. In spite of the fact that it shares three characters of it's name with more common filesystems there is a huge amount of new code in it that has not had extensive real-world testing yet. I think that we're only in the 6-9 month range since the developers started trusting their own data to it.
the fact that Fedora (a bleeding edge distro) is considering using it for it's next release (in 4 months or so) just means that they think that it's good enough to test.
this is a FAR cry from saying that it's a reliable filesystem that is production ready.
NTFS speed may be terrible, but it's been around long enough to have let experience find the bugs that the developers missed.
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 23, 2009 23:50 UTC (Fri) by rahulsundaram (subscriber, #21946) [Link]
Some additional information:
Upstream has dropped the development tag from Ext4 in the latest Linux kernel release which is a indication that they believe it is good for more wide spread usage.
Red Hat has kernel filesystem developers working on Ext 3/4, XFS and Btrfs and Fedora has supported Ext4 as a experimental option right from the Fedora 9 release. Red Hat has included Ext4 as a technology preview in the recently released RHEL 5.3 as well.
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 29, 2009 10:29 UTC (Thu) by Cato (subscriber, #7643) [Link]
There are quite a few real-world bugs still in NTFS-3G, I just saw one this morning from an Ubuntu user. While NTFS in its Windows implementations seems very stable, NTFS-3G has had less time to mature, so it's best to limit its use if possible - e.g. for a dual boot system put most of the shared data on FAT32 so that most NTFS writes are from Windows.
NTFS has advantages when Windows is running (e.g. it supports shadow copies for consistent backups of open files) but not for Linux.
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 30, 2009 15:15 UTC (Fri) by szaka (subscriber, #12740) [Link]
We are not aware of any NTFS-3G bug. If you check out http://ntfs-3g.org/support.html then you can find that there are many problems which are attributed to NTFS-3G when in fact they are not related to any file system in any way.
About the maturity of the 9 years old NTFS-3G code base see http://lwn.net/Articles/316471/
Unfortunately we, NTFS-3G developers, don't share your view about the stability of the Windows NTFS driver. There are many ways to crash Windows via the NTFS file system driver. In fact Microsoft documents that system crash is expected for instance if NTFS is corrupted. This is completely unacceptable for us. NTFS-3G survives millions of fsfuzz iterations.
I think our test suite is one of the most extensive ones in the industry (it's not complete list): http://ntfs-3g.org/quality.html Besides we are maintaining the PJD POSIX file system test suit as well: http://ntfs-3g.org/pjd-fstest.html what of course our advanced driver fully passes.
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 24, 2009 0:59 UTC (Sat) by szaka (subscriber, #12740) [Link]
NTFS read-write is actually one of the oldest and probably also one of the most mature (I don't mean the original NTFS project from 1995 but the one which restarted in 2000). Its code base is 9 years old. The core of the write functionality is actively used since 2003 in quite many softwares. Not necessarily as a full-feature read-write driver but as part of other softwares which had a very good code coverage of the today complete driver.
We are not aware of terrible NTFS performance. Many performance related issues are documented at
In fact some people reported that it's sometimes faster than ext3. This is not so surprising if one compares the file system designs. Ext4 solved the ext3 performance issues. In all our recent tests ext4 performs better than NTFS-3G.
There is also a performance oriented NTFS-3G driver, mainly for consumer electronic device makers which is 3-20 times faster. On a T9300@2.5GHz it performs 1.375 GB/s write speed. No typo, it's gigabyte per second and it's still far to be fully optimized.
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 24, 2009 8:02 UTC (Sat) by tzafrir (subscriber, #11501) [Link]
> but personally I don't touch any filesystem
> that isn't years old (or heavily based on one that > is years old) or that can't easily be recovered in > some fashion to a working filesystem.
And XFS and JFS don't match your criteria because?
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 23, 2009 14:30 UTC (Fri) by jengelh (subscriber, #33263) [Link]
It's amazing to see users still debating about "foofs ate my files, so I switched to barfs".
FWIW, XFS had important Ext4 features for years. Extents, delayed allocations, and whatever else is under the hood. What Ext4 *still* does not have: mount-time quota scanning and activation. It is so much faster than quotascan and safer because there is no window between mounting and turning quotas on (where a user could dump a huge file to gobble up space for himself).
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 23, 2009 22:48 UTC (Fri) by drag (subscriber, #31333) [Link]
Sorry but:
Safe Data > Features.
Between choosing to have a file system that is much more reliable on shitty (aka commodity) hardware vs having mount-time quota scanning I'll choose the safer FS.
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 23, 2009 23:10 UTC (Fri) by dlang (✭ supporter ✭, #313) [Link]
which filesystem are you considering unsafe on commodity hardware?
if you are thinking that XFS requires systems with special power supplies, that is a myth, and that bog that was fixed in may 2007 (linked to elsewhere in this topic) was the fix for it.
if anything, the fact that ext3 ignores write barriers by default while XFS honors them would make me say that XFS is safer on commodity hardware :-)
Write barriers Posted Jan 24, 2009 1:49 UTC (Sat) by i3839 (guest, #31386) [Link]
> if anything, the fact that ext3 ignores write barriers by
> default while XFS honors them would make me say that XFS > is safer on commodity hardware :-)
Depends. If it ignores write barriers because it is reliable without them in some other way, then it's not less safe. Similarly, if some drives ignore write barriers, then not counting on them in the fs is more reliable as well.
Write barriers Posted Jan 24, 2009 2:21 UTC (Sat) by dlang (✭ supporter ✭, #313) [Link]
in this case it is known that ext3 ignoring barriers puts data at risk, but the performance overhead of turning them on by default (something like 30%) was deemed 'too expensive' by one person, so they remain off
I think you will find arguments where Ted Tso is arguing that they need to be on for data safety and Andrew says that it's unacceptable to introduce the performance regression that would result from doing so.
Write barriers Posted Jan 26, 2009 20:29 UTC (Mon) by zlynx (subscriber, #2285) [Link]
Ext3 generally works without barriers because hard drive write caches usually write the journal first for whatever reasons. From what I recall reading, because the journal is all in one chunk at the beginning of the drive far away from the data, and that write caches usually flush in-order from lowest block to highest.
Note all the generally and usually qualifiers in there.
I recall reading about Ext3 journal corruption issues when the power fails as the journal is wrapping around, because the drive writes the end of the journal, some data, then back to the beginning of the journal instead of writing end of the journal, start of the journal, some data.
Write barriers Posted Jan 29, 2009 16:47 UTC (Thu) by anton (guest, #25547) [Link] Who knows what drives do or don't do when in write-caching mode? The last time I tested this 10 years ago the drives I used apparently wrote the blocks in-order, except when a block was overwritten; then the data might never reach the platter (if it's overwritten again and again). I should do some experiments with my new drives.Also, why should writing to the journal first be safer than writing the other stuff first? With meta-data journaling as typically used in ext3 this would lead to typical meta-data update bugs. Actually the data=ordered mode writes the data before the corresponding journal entries to prevent that from happening. So if the disks then write the journal entries before the data thanks to write-caching disks without barriers, that subverts the safety data=ordered provides. If the disks write stuff out-of-order in other ways, other failure modes are possible. I guess that the reason we don't see lots of reports of data destroyed by ext3 without barriers is that most people don't have lots of write traffic going on (but then they won't notice any performance penalty from barriers either), and you expect to lose the file you write when the system dies anyway.
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 25, 2009 19:29 UTC (Sun) by Nelson (subscriber, #21712) [Link] I think the data safety claims are spurious at best, XFS is a serious filesystem with serious developers, made by a serious company. There simply aren't semantics for UNIX defined for what is supposed to happen when you lose power mid-write. My experience, they are both par. All the stories seem to be anecdotal, and the fact of the matter is whatever filesystem you're running when you lose data is going to become a defacto POS in your mind after that.The big difference between the ext(x) family and XFS and JFS is ext(x) seem to be supported, really well. The others are still kind of bolt-ons to various distributions. I ran an all XFS Mandrake/Mandriva and Fedora systems for years, you'd be shocked how many times you'd' install a kernel upgrade and have a non-bootable system. I got to the point I'd never even consider doing a full distribution upgrade. The situation might be better now but for a very long time it looked like just a handful of kernel devs so much as cared about XFS, let alone enhanced it and kept it fresh with the rest of the kernel.
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 25, 2009 20:35 UTC (Sun) by cantsin (guest, #4420) [Link] What doesn't seem to be anecdotal is that XFS caches data more aggressively and over longer time in RAM than other filesystems including ext3. This results in performance advantages, but reliability issues on any other but high quality server hardware with ECC RAM. The anecdotal evidence mostly comes from users running PCs with standard (cheap) RAM and in non-data center-/USV-usage scenarios where loss of power is more likely or even common (such as on laptops).
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 25, 2009 20:39 UTC (Sun) by jengelh (subscriber, #33263) [Link]
I have to object - laptops have statistically more batteries than desktop computers :-)
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 26, 2009 2:25 UTC (Mon) by salimma (subscriber, #34460) [Link]
But, in countries with relatively stable electricity supplies, laptops are more likely than desktops to be uncleanly shut down because the battery ran out.
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 25, 2009 23:56 UTC (Sun) by nix (subscriber, #2304) [Link]
UNIX doesn't define semantics for what happens when you lose power
mid-write. That means that what a filesystem does when that happens is a quality-of-implementation issue: but given that power supplies are not perfectly reliable it's a fairly important one, especially in countries like the USA with a third-world electricity distribution network (as a UK resident I'm privileged to have a fairly reliable power supply and the most badass electrical regulations on earth: it's one of the few things the UK's done right infrastructure-wise).
However, I'm afraid I consider 'files you were writing right now might be
I too saw this behaviour when I used XFS (years ago now): I've never seen
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Jan 26, 2009 23:44 UTC (Mon) by lmb (subscriber, #39048) [Link]
Unix/POSIX doesn't define what happens to data in when you crash "mid-write", but it has one hell of an opinion on writes that have been committed and where the write()/fsync()/... calls have returned. So the durable state of the data is quite defined, unless the application is too dumb to make proper use of those features. (And, of course, unless the fs bites the dust.)
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Feb 2, 2009 23:32 UTC (Mon) by dirtyepic (subscriber, #30178) [Link]
i just wanted to say that i think barfs is an awesome name for a file system.
Ext4 to be standard for Fedora 11, Btrfs also included (heise online) Posted Feb 3, 2009 0:39 UTC (Tue) by nix (subscriber, #2304) [Link]
And if you've been drinking vodka too long from the barfs, you might go
home and write pohmelfs. I think yo might have a poin there.
|
|||||||||||||