Weekly Edition
Return to the Front page
| |||||||||||||
Reiser4 and kernel inclusion
Filesystem developer Hans Reiser cannot be accused of giving up quickly.
His current request that the reiser4
filesystem be included in the 2.6.14 kernel has created a lengthy
discussion, but that's not where the story starts. In fact, Hans first asked that reiser4 be merged back
in July - of 2003. For more than two years, Hans has repeated his request
and made changes in response to feedback from the kernel development
community. Yet reiser4 remains out of the mainline, and its chances of
getting into 2.6.14 appear small.
Reiser4 is an interesting filesystem. It comes with claims of improved speed and space utilization; those are welcome, but they are beside the real point. Reiser4 includes a "wandering log" mechanism which provides journaling capability without the need for a separate journal. The ability to perform multi-step transactions is built into the filesystem, though not yet completely exposed to user space. Multi-stream files (including file-like access to file metadata) are supported, though this feature is turned off for the moment as well. A flexible plugin architecture makes it easy to add new features (such as encryption, compression, different formats, etc.) to the filesystem. And so on. Hans Reiser and his developers at Namesys are trying to change how people work with filesystems - and with computers as a whole. The underlying vision is one where the filesystem implements the entire namespace used by the system; everything truly is a file. In the Reiser view of the future, applications like relational database managers need not exist; such tasks will be handled in the filesystem itself. What it comes down to is that reiser4 represents some of the most innovative work being done with filesystems for Linux - or for any other system. So one might well wonder why inclusion is proving to be such a challenge. Some of the reasons are straightforward: there were genuine issues with the code. The "files as directories" capability opened the door to trivial, user-initiated kernel deadlocks - a feature which can absolutely ruin those performance benchmarks. The multiplexed sys_reiser4() system call - to be used for managing transactions, among other things - is just the sort of call that the Linux developers are trying to get away from (and its use of an in-kernel command interpreter did not help). A number of other things needed to be fixed; the Namesys hackers have been working through the list, but a few items remain. The real point, however, is that getting code into the kernel is an increasingly hard thing to do. In the early days of Linux, almost any code which made things work or added new features was welcome - we needed it all. In more recent times, it is often hard to argue that new features are truly needed, especially at the kernel level. So each new addition must be weighed against the costs that will be incurred when it is added. The result is that the standards for new kernel code have gone up considerably over the years. Reiser4 has run into these standards, and objections have been raised to code which duplicates features found elsewhere in the kernel, is hard to read, violates the layering rules, has unclear locking schemes, or which uses obsolete interfaces. The point is that, in order to be merged, the reiser4 code must be understandable by people other than its original developers. As Alan Cox put it:
It doesn't matter if reiser4 causes crashes. It matters that people
can fix them, that they are actively fixed and the code is
maintainable. It will have bugs, all complex code has bugs. Hans
team have demonstrated the ability to fix some of those bugs fast,
but we also all remember what happened with reiser3 later on
despite early fast fixing.
"What happened later on" is that the reiser3 developers moved on to reiser4; not only did they stop maintaining the code, but they actively opposed updates made to the code by other developers. At this point, reiser3 is almost entirely maintained by non-Namesys developers. In the future, the same thing may well happen with reiser4. The crux is this: the Linux kernel has been around for 14 years, and is expected to last for quite a few more. The kernel hackers understand that, if they are insufficiently careful about what they merge now, they will have a big mess to deal with five years down the road. Many developers, working in all areas of the kernel, have had seemingly good code turned away because the development community was worried about maintaining the code in the future. The process is most frustrating for the people involved, but it is absolutely essential if we want to continue to use Linux into the future. To many, the difficulties encountered by reiser4 (and FUSE, and the realtime LSM, and class-based kernel resource management, and ...) represent the kernel development process at its worst, but the opposite is true. That said, reiser4 has had a harder time and more microscopes applied to its code than many other developments. Mr. Reiser's approach to community relations, which strikes many as occasionally belligerent and paranoiac, certainly has not helped here. This issue has been discussed often, but there is another issue which deserves airing: some people are clearly uncomfortable with the vision behind the ongoing Reiser filesystem effort. It doesn't quite look like the Unix systems we grew up with. Linux is not an experimental or research-oriented system, so the inclusion of radically different ideas of how the system should work must be carefully considered. But Linux must also evolve, or risk irrelevance in the future. Hans Reiser's efforts to push that evolution are a good thing; the community discourages such work at its peril. So perhaps the time has come to let reiser4 in; the wider community can then get to work dealing with any remaining issues. (Log in to post comments)
Small syntax error Posted Sep 22, 2005 1:54 UTC (Thu) by jhs (guest, #12429) [Link] Paragraph 2, sentence 3:Reiser4 includes a "wandering log" mechanism provides journaling capability without the need for a separate journal.I think a relative pronoun got omitted. I wouldn't say anything on any other sites, but LWN is pretty consistantly well-written, in addition to having good content.
Reiser4 and kernel inclusion Posted Sep 22, 2005 2:29 UTC (Thu) by sbergman27 (subscriber, #10767) [Link] One other thing I will point out. Hans leans heavily on Reiser4's performance benchmarks when advocating its inclusion in mainline. There are times when it appears that he thinks it should go in for that reason alone.
So it was particularly distressing when some months ago, perhaps as much as a year (I can't remember exactly) Hans was going on (on lkml) about Reiser4 leaving ext3 "in the dust" and having the benchmarks to prove it (btw, I think that "in the dust" is probably a good place to start if you want to google for the thread) when former employee Nikita Danilov spoke up to remind Hans that he was the one who had conducted those benchmarks and that furthermore, when he went to Hans to show him how dismally Reiser4 performed compared to ext3 in some phases of the test, Hans told him to simply remove the results which were unfavorable to Reiser4 and publish the rest.
Hans admitted on the list that he had done so and stated that he would make a note on his site about the omitted results. To this day he has not done so.
He had some ideas at the time as to how to get around the performance problems, but his own employees were telling him (also on lkml) that his ideas would not work.
I try to keep all that in mind when trying to evaluate the veracity of Hans' statements.
Reiser4 and kernel inclusion Posted Sep 22, 2005 3:41 UTC (Thu) by sbergman27 (subscriber, #10767) [Link] For reference, here is the thread:
http://www.uwsg.iu.edu/hypermail/linux/kernel/0408.3/1429...
Hans Posted Sep 22, 2005 5:10 UTC (Thu) by ccyoung (guest, #16340) [Link] Reiser is not the smooth politician of Linus, has problems seeing faults outside his the closure of his vision, and - the more you read of his the more you realize - he's a kid. He's trying to carry a great weight on his shoulders, and is enormously in debt trying to fulfill these obligations. I personally have nothing but praise and respect for him.
If you're looking for faults with him it's shooting fish in a barrel. But so what. To misquote Al Davis, His ideas are going to take us to the next level, baby. It's going to be a royal pain in the ass, and it's the right thing to do.
Hans Posted Sep 22, 2005 17:02 UTC (Thu) by smeg4brains (guest, #207) [Link] If you're looking for faults with him it's shooting fish in a barrel. But so what. To misquote Al Davis, His ideas are going to take us to the next level, baby. It's going to be a royal pain in the ass, and it's the right thing to do.I couldn't have said it better myself. Linux is often accused of lacking in innovation. Now we have someone that's really trying to innovate, and his code is getting ignored because he's sometimes an immature brat. None of the childish things he's done really erase the fact that he's basically devoted his life to a charity cause that is intended to help us all. You can hate him, if you like, but why keep his code out? It doesn't hurt anyone that decides not to use it, and most people that I've talked to are excited to give it a try (even the crazy crap that is currently disabled). Mark it "experimental" or if they've got a little Hans in them, they can mark it "very experimental, dangerous++" and make the description "This will most likely delete all your files in the morning"... but for the users sake, just put it in.
Reiser4 and kernel inclusion Posted Sep 22, 2005 6:30 UTC (Thu) by edomaur (subscriber, #14520) [Link] Personally I very much like to see Reiser4 merged. There are some interesting fs out there that do some changes in the way the VFS layout could be seen. v9fs and Reiser4 are two of them, but you can bet that these kind of things will arise more and more often. Better begin to work on some new VFS functionnalities now, when there is only a limited set of exotic fs to cope with.
As for the personnality of Hans Reiser, well, he is quite abrasive, it's a fact. So what ? He is not the only one, and he is not the most abrasive.
Reiser4 and kernel inclusion Posted Sep 22, 2005 19:15 UTC (Thu) by cventers (guest, #31465) [Link] Agreed. I am shocked by the volume of people who refuse to use DanielBernstein's software on the grounds that he is an asshole - even though his software is clearly *very* robust and *very* secure. Hans seems to be another case of "punish the person by refusing to accept their charity on its own merits."
Bernstein is a different beast Posted Sep 23, 2005 0:25 UTC (Fri) by pm101 (guest, #3011) [Link] The problem with Bernstein's software is the ultrarestrictive license. It is very robust and very secure, but if Bernstein gets hit by a bus tomorrow, there will be no more updates. More likely, if Bernstein has a philosophical swing of some sort tomorrow, he is free to modify his software appropriately, and you're either stuck with no more updates, have to switch platforms, or have to buy into his new software. He's working on a replacement for the SMTP protocol. Lets say that ten years down the line, 75% of the mail servers out there support it, and he decides to remove SMTP support from qmail to force the remaining 25% to switch. What do you do? Let's say that he decides enough people adopted qmail, and now he wants to make some cash off of it, and the new versions are only available for $1k per seat. What do you do?
Assholes are okay, so long as they use the GPL. Assholes with proprietary licenses are a different beast. I don't want my infrastructure to be dependend on some unstable guy's mood swings. See BitKeeper for an example of what can happen when you do.
Bernstein is a different beast Posted Sep 23, 2005 11:31 UTC (Fri) by job (guest, #670) [Link] That is not true. Bernsteins software is not more non-free than Firefox. What would happend if Bernstein is hit by a bus is that his projects would be forked under new names -- just like netqmail is a fork of qmail with more modern functionality.
License? Posted Sep 23, 2005 13:44 UTC (Fri) by pm101 (guest, #3011) [Link] This is slightly outside of the scope of this discussion. Nevertheless, I believe I need to be more clear, since the comment posted was factually inaccurate. qmail itself does not come with a license. This is because of djb's philosophy. Given just this, you would be allowed to modify your local copy, but not make copies of the software for others. In addition, he grants rights of redistribution for specific, unmodified versions that he released. They must have the same MD5 checksum.As a result, it is possible to, for instance, distribute the basic qmail distribution, a minor patch, and some method of combining them, in the way that netqmail does. Nevertheless, it is illegal to distibute binary builds of netqmail, or sources in a format other than the one described above. This practically limits the scope of modifications you can make -- while it is possible to write a better version of diff/patch, at the moment, you cannot rename a file, since then the text of the file would be in the patch. You cannot combine two files, or split a file into two for similar reasons. We can get around some of this with a better version of diff/patch, but there will almost certainly always be limitations to the types of modifications you can make. This also limits how the software may be packaged. It is still non-trivial to patch binaries, so every user must make a local build. You cannot make a standard .deb or .rpm of netqmail (you can do the installer thing some .debs of proprietery software do, that grabs it and compiles at, at the addition of some amount of build time). If you have 50 servers in your location, you cannot install the same hard drive image on all of them if the hard drive image contains netqmail. You must compile each copy of netqmail locally. There are many other similar limitations. If djb goes off the deep end, or dies, users of his software are SOL. They'll get patches with security fixes for some time, but the software will be fundamentally unmaintainable in the long term.
Bernstein is a different beast Posted Sep 23, 2005 15:39 UTC (Fri) by Ross (guest, #4065) [Link] The problem is that the license terms are unknown. You could say "well, but it should be ok anyway because everyone can be expected to act reasonably". But that's where his, erm, personality traits, come into play with decision making. Also, while the software is well written and secure, it makes things difficult to do in normal ways (startup scripts, directory layout, etc.).
Reiser4 and kernel inclusion Posted Sep 22, 2005 9:20 UTC (Thu) by ringlord (subscriber, #6309) [Link] The Open Source community is often criticised for not innovating, butinstead playing catch up. While there may be some good reasons for not including reiserfs4, at least it seems like it's a very innovative approach to filesystems. Also, if it is not "traditional Unix", so what? You'll still have the other filesystems, but maybe it is time to explore new ways of doing things?
Reiser4 and kernel inclusion Posted Sep 22, 2005 11:14 UTC (Thu) by NAR (subscriber, #1313) [Link] While there may be some good reasons for not including reiserfs4, at least it seems like it's a very innovative approach to filesystems.I don't know the reiser4fs filesystem, but one thing caught my attention in the article: In the Reiser view of the future, applications like relational database managers need not exist; such tasks will be handled in the filesystem itself. I seem to remember that the VMS filesystem (some 20 years ago) had "relational database-like" features: files can consist of strongly typed records and there is record-level locking.
Reiser4 and kernel inclusion Posted Sep 22, 2005 15:20 UTC (Thu) by vonbrand (subscriber, #4458) [Link] Right, the Database is in the filesystem has been tried numerous times. Wonder why none of them is still around...
Reiser4 and kernel inclusion Posted Sep 22, 2005 15:54 UTC (Thu) by zlynx (subscriber, #2285) [Link] How many times did Edison try to build a lightbulb?
Just because it hasn't worked before does not mean it won't work now or will never work in the future. It's important to learn from past mistakes, but the key word there is "learn." And then try it again.
Reiser4 and kernel inclusion Posted Sep 23, 2005 12:32 UTC (Fri) by NAR (subscriber, #1313) [Link] How many times did Edison try to build a lightbulb?Your analogy is wrong: "database in a file" is not a question of "can be done" or "can't be done". It has been done already at least 20 years ago and there are at least two implementations (VMS filesystem and Berkley DB), so there's nothing new with that. The real question is that should this be implemented in the kernel or in user space. As I mentioned, the VMS filesystem has a feature like this. One of the recurring problem was that users backuped their MAIL.MAI files (the equivalent to /var/mail/<username>) via FTP to an another computer. Then when they restored the file, they saw that they lost their mail, because this MAIL.MAI file is a "database in file" and its structure is stored in the filesystem, in attributes. Of course, when the file was restored via FTP, the FTP server had no idea that this file should have any special attributes, so the file was created as a binary stream, instead of the special indexed format. The UNIX filesystem has been designed and used for 35 years with the basic concept that a file is just a sequence of bytes. I'm afraid there are a lot of applications that depend on this fact implicitly and we could be in for some nasty surprises like the above mentioned scenario if this assumption is changed. My other problem with the "database in file" concept has been addressed elsewhere in this thread - code of this complexity shall be done in user space, not kernel space. It's easier to debug and if it crashes, it doesn't bring down the whole computer.
Reiser4 and kernel inclusion Posted Sep 23, 2005 14:11 UTC (Fri) by zlynx (subscriber, #2285) [Link] Apparently my analogy is just about perfect. Edison's lightbulb attempts didn't all simply fail to work. Some were too dim. Some burned out too quickly.
Problems appear and are fixed, until it works. You demonstrated one problem with database filesystems. Do you think that no one else has noticed that problem or thought of ways to fix it? No so!
Are you aware that the "should be done in userspace" argument is exactly why Linux was supposed to fail? It isn't a microkernel. Microkernels were supposed to be the greatest thing ever. Well, they aren't. The performance penalties for all that protection are too great.
Reiser4 and kernel inclusion Posted Sep 23, 2005 14:46 UTC (Fri) by dvdeug (subscriber, #10998) [Link] What should be done in user space is not simply a microkernel/normal-Unix kernel question, though. Every new feature is a question of whether it should be done in user space. Should a kernel handle video or leave it to user space? NT handles it in kernel, a much criticised decision. Linux offers a kernel-space implementation and a more powerful user-space implementation. Should emulation be done in kernel? Floating point emulation and supporting instructions from newer CPUs have been contested, but I think both have gone into released kernels. Should the kernel include a JVM? If not, should it continue supporting OSF executables on Alpha and x86 executables on AMD-64?
User-space or not is just not a question that should be answered with a knee-jerk response.
Reiser4 and kernel inclusion Posted Sep 30, 2005 11:09 UTC (Fri) by forthy (guest, #1525) [Link] For the video question: I think yes, it is a mistake to have the Xserverdo the card settings. Really. The Xserver should have exposed interfaces to the card, and the kernel should know about the state, so that switching VTs, resetting state after X crash or suspend will work. For the drawing commands itself, something like the DRM module should be used for 2D drawing, too.
Reiser4 and kernel inclusion Posted Sep 29, 2005 7:46 UTC (Thu) by Wol (guest, #4433) [Link] Even better - Edison finally fixed his problems by ditching his ideas and "borrowing" someone else's.
I think you'll find that the lightbulb, as finally produced by Edison, is a pretty close match to the lightbulb as patented by Joseph Swan. Oh - and that patent is dated the same or earlier than Edison's patents.
Cheers,
Reiser4 and kernel inclusion Posted Sep 23, 2005 16:45 UTC (Fri) by IkeTo (subscriber, #2122) [Link] > "database in a file" is not a question of "can be done" or "can't> be done". It has been done already at least 20 years ago and there > are at least two implementations
Um... I might be wrong, but I really think that "database in a file" isn't quite what ReiserFS achieves. Instead, what it achieved is "filesystem in a database". The whole filesystem is implemented by using database technologies that simply holds a single huge B+-tree, with journaling features and such. And the database is being backed by a raw block device. Because it is "a filesystem in a database", of course it has all the database features. It is just the question of whether it exposes the filesystem features of itself into the user space (ReiserFS 4) or not (ReiserFS 3).
Reiser4 and kernel inclusion Posted Sep 23, 2005 19:13 UTC (Fri) by corbet (editor, #1) [Link] In fact, "database in a file" is far removed indeed from what reiser4 is trying to do; comparisons with VMS are misplaced here. I suggest a read through the files on namesys.com to get an understanding of the ideas and vision behind reiser4. It will not be a quick read, and you have to deal with some of the stranger artwork I've ever seen in technical material, but it can be worth the effort regardless.
Reiser4 and kernel inclusion Posted Sep 29, 2005 7:38 UTC (Thu) by Wol (guest, #4433) [Link] At least 20 years ago?
Make that 40. Pick was born in 1967. Oh - and it's still around. It powers some of the biggest data warehouses in the world - far bigger than relational can handle.
Cheers,
Reiser4 and kernel inclusion Posted Sep 23, 2005 11:28 UTC (Fri) by job (guest, #670) [Link] The file system *is* a database. It is a hierarchical database which maps keys to BLOBs, essentially, with some meta data on the side. There has been some evolution in the file system space during the last decade, but things move very slowly.
For example is it mainstream among Linux distributions to index the keys properly now which is a very database-ish thing to do (which reiserfs 3 pioneered in Linux by the way). Also the BeFS had some very interesting queries, which may be a niche market today but far from dead.
The next step would be to have a better interface for open queries that scales (perhaps inotify?) and extensible metadata with proper indexing. Reiserfs3 also has tail packing which is a very nice feature that I hope spreads to other file systems. Reiser4 is a very interesting next step, and I would be sad to see it vanish just because of implementation.
Reiser4 and kernel inclusion Posted Sep 29, 2005 7:43 UTC (Thu) by Wol (guest, #4433) [Link] Right, the Database is in the filesystem has been tried numerous times. Wonder why none of them is still around... Probably because people nowadays equate "database" with "relational database". Relational theory is a mathematical theory, and as such there is no evidence that it actually works in the real world. Occam's Razor says it doesn't .... The original razor (translated) is, I believe, "don't multiply entities unnecessarily". What is a relational database but "a maze of twisty little tables, all different" :-) A relational database is just too bloody complicated to be a good solution... Cheers,
time for (another) change? Posted Sep 22, 2005 9:58 UTC (Thu) by zmi (guest, #4829) [Link] Reiser4 should be merged. People can choose if they want to use it or not.If the namesys people will not fix bugs, mark it "experimental" in the kernel options. But merge it! Linux should be innovative, and reiser4 could be. It sounds like a good step in a new field. I'm still missing the very nice and simple to understand filesystem features that Novell's Netware provided already back in at least 1996 (Netware 4). With a few clicks, users or groups could be assigned all types of accesses. ACLs do the trick now, but are not that easy to use as those on Netware. Linux should try to become more user friendly (still), and reiser4 could be a good step.
What you all fail to acknowledge ... Posted Sep 27, 2005 2:21 UTC (Tue) by kirkengaard (subscriber, #15022) [Link] ... is that merging a filesystem is unlike merging a device driver in non-trivial ways. The filesystem used has a dramatic effect on system functionality, in ways that even a networking driver does not. Moreover, merging a filesystem that agrees with the design principles and specs of the operating system is much easier than merging revolutionary works like reiser4, because the compliant filesystem necessitates orders of magnitude fewer changes to the way everything else expects a filesystem to work. The "Just adopt it already" position ignores the fact that reiser4 expects things that the kernel will natively disagree on in use, and the kernel expects things that reiser4 will screw up. Work is ongoing; we wouldn't be having this debate if reiser4 was being rejected.
Hans refuses to acknowledge the fact that this whole argument is really him winning kernel support for reiser4, since accomodations *are being made* to do so. Full inclusion will happen when everything is kernel-maintainable and functionally stable. Experimental features belong in -mm until those two criteria are met.
Also, just to say it plainly, namesys will not support 4 past the release of 5, just as they don't support 3 now that 4 is out. Hans Reiser does not want non-namesys-employed hands working in "his" codebase, and so protests efforts to change his work to conform to a) kernel code requirements, and b) user-desired features and bugfixes added in a completely kosher fashion (a la GPL). Until Hans thinks like the rest of the GPL world thinks, namesys products will continue to fail the expectations of F/OSS -- and especially Linux kernel -- programmers. That's at least half of the adoption hurdle, ignoring the equally applicable ad hominem attacks -- and if you think social graces shouldn't matter, tell that to your boss/loan officer/HR manager next time you see him/her/them.
What you all fail to acknowledge ... Posted Sep 27, 2005 6:47 UTC (Tue) by zmi (guest, #4829) [Link] > Experimental features belong in -mm until those two criteria are met.If it's "just" this, then a delay is of course necessary. Only when stability is more or less to be expected (don't dare to say "guaranteed" ;-), an inclusion is appropriate. > Hans Reiser does not want non-namesys-employed hands working in "his" > codebase, and so protests efforts to change his work to conform to a) > kernel code requirements, and b) user-desired features and bugfixes > added in a completely kosher fashion (a la GPL). Why can there be a discussion about this? If they don't support it, somebody else should. He should fix it or shut up. Progress should be done, in order have more features. Although I personally am happy with reiser3, it just works and is stable. But it's becoming slow after some time working with it, just like NTFS. Hopefully reiser4 with it's builtin defragmentation can help there. mfg zmi
Reiser4 and kernel inclusion Posted Sep 22, 2005 11:21 UTC (Thu) by duck (guest, #4444) [Link] Hi,I am all for an inclusion of Reiser4, since I like the vision and the opportunities it offers _very_ much. Of course nobody can predict whether it will be maintained for long enough, or whether its plugin-architecture will ever be used. But this is irrelevant as far as I am concerned, because it is "only" a YAFS (yet another file system). The difference is that is tries to provides so much more than the other file systems. Of course there will be penalties for the features it provides, maybe the speed or processor usage will suffer. But if the vision becomes reality, and instead of storing files in a DB, I can just save them on an reiser4 partition, then one might be willing to pay a little penalty. So what if it does not work, or is never really used in innovative ways, or not properly maintained? Just remove it from the kernel, an tell the people to use a different YAFS.... The only reason to oppose its inclusion are long-lasting, painfull changes to the kernel itself that reiser4 needs. As far as I understood, people are trying to minimize those. I wish them Good Luck Peter
Reiser4 and kernel inclusion Posted Sep 22, 2005 15:59 UTC (Thu) by zlynx (subscriber, #2285) [Link] "The only reason to oppose its inclusion are long-lasting, painfull changesto the kernel itself that reiser4 needs."
Which is a bit ironic, I think. The first proposed versions of Reiser4 were very self-contained, which is where the complaints about layering violations came from.
Reiser4 and kernel inclusion Posted Sep 22, 2005 15:55 UTC (Thu) by zooko (subscriber, #2589) [Link] "That said, reiser4 has had a harder time and more microscopes applied to its code than many other developments. Mr. Reiser's approach to community relations, which strikes many as occasionally belligerent and paranoiac, certainly has not helped here."
In other words, some of the kernel developers are giving reiser4 an extra hard time of it because Hans Reiser has angered them and this is how they can use their power to punish him. Is that what's happening?
Or is it that Hans's alleged belligerence and paranoia has somehow contributed to technical issues which need to be solved before reiser4 can be included?
Reiser4 and kernel inclusion Posted Sep 22, 2005 16:01 UTC (Thu) by zooko (subscriber, #2589) [Link] Following up to myself because I realized that the original article suggests a third possibility: the alleged belligerence and paranoia has led kernel developers to suspect that Hans Reiser and/or his employees are unreliable maintainers of the code, and they are hesitant to include code which might be unreliably maintained by the original authors.
Honestly, having read a few of the threads on lkml myself, I still lean toward the first hypothesis: some people are angry at him, and are thus motivated to criticize his works more harshly. The latter hypothesis -- "are namesys reliable maintainers?" -- makes sense in the abstract, but if you consider the way the Linux dev process already includes plenty of personalities who are at least as belligerent as Hans is, and plenty of code that is worse-maintained than reiser3 or reiser4, I don't think that's really the explanation.
Reiser4 and kernel inclusion Posted Sep 22, 2005 17:09 UTC (Thu) by southey (subscriber, #9466) [Link] My understanding of the kernel development (from reading) is that the kernel does not seem to discriminate between people but rather on the actual code quality. It has been very clear that bad code (really how the code affects the complete kernel) does not enter or stay in the kernel regardless of who wrote it.
I think that the other filesystems like JFS also went through the same process (relative to the state of the kernel) except that these ports than new filesystems. These also had to undergo some changes that the respective developers made. I recall similar issues to reiser4 for when reiser3 started to be merged but reiser4 changes appear to be more invasive.
As for the current status, I think (with no knowledge of kernels) that the apparent rigid criteria applied to reiser4 is no different than say the current criteria in the kernal for real-time linux. Okay, sure a major kernel developer is involved (perhaps that is what reiser4 lacks) in the real-time changes but not all the real-time changes have been merged, some were rejected. Often these changes involved major rewrites (like making the changes less invasive etc).
Reiser4 and kernel inclusion Posted Sep 22, 2005 15:56 UTC (Thu) by sbergman27 (subscriber, #10767) [Link] My take is that much of Reiser4 is just hype. The performance is spotty. (I hear this from actual users of reiser4.) It brings in much excess and redundant baggage that will, if history is any guide, have to be supported by non-Namesys personell in the not too distant future. It can cause kernel deadlock. And those features of the filesystem which do turn out to be useful should be available to all filesystems in a generic way and not exclusive to Namesys' own little corner of the kernel.
I am in favor of its inclusion *AFTER* the very valid issues raised on lkml have been addressed. Just like *anything else* that goes into the kernel.
Reiser4 and kernel inclusion Posted Sep 22, 2005 17:47 UTC (Thu) by iabervon (subscriber, #722) [Link] I think the real issue is that Hans is trying to write a new filesystem to revolutionize how we use files, but that just isn't exclusively the jurisdiction of a filesystem. In the Linux scheme, there is a restricted set of things that the filesystem does, and then there are other things that get done by the VFS. All of the really interesting stuff that Hans is doing belongs in the VFS, and shouldn't care too much about the filesystem in use. It would be nice to cat a file that's inside a tar file, but if that's done in a filesystem, you'll be able to do it on your hard drive, but doing it on a CD or a ramdisk won't work. Obviously, this is going to confuse users, who don't expect ramdisks to work differently from hard drives.
The problem is really that Hans didn't start by understanding how the division of labor among kernel subsystems worked and why it was set up that way, so the parts where he has really good ideas aren't the parts that are in a state suitable for submission, and so he doesn't have anything better to take up his attention than flaming people.
Reiser4 and kernel inclusion Posted Sep 22, 2005 18:13 UTC (Thu) by sbergman27 (subscriber, #10767) [Link] Namesys has a business interest in keeping the new features exclusive to its own filesystem.
Reiser4 and kernel inclusion Posted Sep 22, 2005 19:30 UTC (Thu) by cventers (guest, #31465) [Link] I'm a merge-Reiser4 supporter, and I'm not sure that your comment impliesyour opinion on the issue this reply addresses, but "business interests" do not belong in the Linux kernel.
Reiser4 and kernel inclusion Posted Sep 22, 2005 22:25 UTC (Thu) by sbergman27 (subscriber, #10767) [Link] I've expressed my opinion elsewhere in this thread. Here I am just stating a fact. Namesys has an interest in seeing that Reiser4 is the only filesystem with the additional features. Namesys also has a business interest in being the sole entity up to the task of supporting and extending Reiser4. Remember, they make their money from supporting and extending their code privately via paid contract. And while there is nothing wrong with that, one must keep in mind that Namesys' interests do not necessarily coincide with that of the rest of the community.
That is the reason Namesys is so resistant to the fixes that the other kernel developers have proposed. Hans' bad attitude is just a secondary effect and has diverted attention from the real issues.
Bottom line: I am not familiar enough with Christoph Hellwig's reputation to comment. However, I do trust Andrew Morton and Alan Cox to remain level headed and act in the best long term interests of the project. I trust them not to get carried away by Namesys' extensive hype. Likewise, I trust them not to speak against the merging of a patch simply out of spite.
BTW, if people want to use reiser4 without having to mess with the external patch, I'm pretty sure that the latest SUSE versions include it. Which is the way it should be. The vendors are free to add whatever feature patches they want; their trees are short-term. The vanilla kernel is all about long-term maintainability. Linux's future depends on it.
Reiser4 and kernel inclusion Posted Sep 25, 2005 15:36 UTC (Sun) by IkeTo (subscriber, #2122) [Link] > Namesys has a business interest in keeping the new features> exclusive to its own filesystem.
Um... does the developers of the Internet (or actually, ARPANET) has any "business interest" in keeping the new features they design exclusive to the Internet? It seems to me that much of namesys funding comes from the same source as the initial development of the Internet (The Defense Advanced Research Projects Agency)...
Reiser4 and kernel inclusion Posted Sep 24, 2005 18:01 UTC (Sat) by giraffedata (subscriber, #1954) [Link] The problem is really that Hans didn't start by understanding how the division of labor among kernel subsystems worked and why it was set up that way He understood. He was trying to balance a grand vision with practicality. He has said he thinks his work should replace the VFS layer; i.e. he doesn't think it belongs in a normal filesystem driver. But he figured it would be easier to ship separately, and get included, as a loadable kernel module. I think Reiser4 would look a lot different if Hans Reiser owned the Linux kernel.
Moot point - he doesn't. Posted Sep 27, 2005 2:49 UTC (Tue) by kirkengaard (subscriber, #15022) [Link] And it's so good that he doesn't, and indeed, that no-one/everyone owns the linux kernel, and that it is dedicated to reproducing POSIX-standard functionality. See the Massachusetts OpenDocument debate for one example of why standardized functionality is ultimately optimal, and manufacturer-specific, non-standard implementations are less than optimal, even if they provide useful functionality. Grand visions are nice, but world-reproducible, world-repeatable, world-implementable specifications win. Hans didn't create an open specification, nor did he create a systematic approach to anything above filesystem data management. Look at his documentation. He starts, not with Linux integration at all, but with "What is a file?". His grand vision is generated a priori, ex nihil. He picks Linux as the recipient of his gift by default. His other options are Microsoft or the graveyard of niche OSes. He is doomed to see his beautiful whole-cloth implementations broken down into usable functions and adapted for use by people who don't have his ends in mind (with the possible exception of whatever DARPA uses it for). It's the commoditization problem.
Moot point - he doesn't. Posted Sep 27, 2005 14:49 UTC (Tue) by IkeTo (subscriber, #2122) [Link] Standardized functionalities might be "ultimately" optimal, but there are many points in the time when we haven't reach that "ultimate" state. At such times, standardization does far more bad than good.
Standardization requires that there is a group of people who are enthusiastic about the field, knowledgable with the field, and have implementation experience with the field. Without that, it is moot to even talk about standardization. Given that "viewing a filesystem as a file and directory at the same time, and nothing else, in order to implement a superset of POSIX functionality" is such a new vision, I don't think it to be a candidate for standardization or formal specification. At times, it is best to have no specification and no standards to obstruct experiments and developments.
Filesystems are dead, or should die Posted Sep 22, 2005 18:43 UTC (Thu) by eff-tech (subscriber, #32417) [Link] Feature-rich filesystems are complex, and complexity is the last thing you want in such a crucial component. Even a relatively simple filesystem like ext3 went bad a few versions ago when you had an SMP system. (They fixed it, but still.) Furthermore, spiffy filesystem features like versioning (VMS), multiple streams (NTFS, HPFS, HFS(+)), encryption (EFS, cryptoloop), extended attributes (ext2/3, BFS), ACLs, and so on, are not portable and are often unused because of that non-portability and/or because of the unnecessary development and operational complexity they introduce.
For example, does anyone use NTFS' multiple streams feature, except to hide malware? HFS+'s multiple streams just cause pain when e.g. backing a Mac up to a Linux server. Metadata indexing such as in BFS was nice, but insufficient -- we want full-text indexing now, and the kernel is no place for such complicated code. Extended metadata is a losing strategy anyway.
Reiser is supposedly fast, and so is ext2/3, but it doesn't matter. There is no universally fast filesystem; a speed gain for one task is often offset by a speed loss for another task. If you need speed, you need many fast spindles. The noatime mount option is good (that noatime is an option at all is another argument against features!). If you have a trustworthy UPS, you can consider the async option.
The thing to do, if you insist on having a filesystem at all, is to make it as simple and safe as possible, and to provide a mechanism for user code to be called when certain filesystem events happen -- push the complexity out to userland. Some Linux systems come with a daemon that monitors filesystem updates, so that it can keep its own data structures updated (e.g. a full-text index).
Even better, though, is to just get rid of filesystems. See EROS and Coyotos.
I'm not so much against ReiserFS per se, as I am against creeping featurism.
Filesystems are dead, or should die Posted Sep 23, 2005 11:39 UTC (Fri) by job (guest, #670) [Link] "Feature rich" is not a one dimentional variable. One might consider the Plan 9 file system to be very feature rich, i.e. you can do a lot of stuff with it that you can't do with POSIX file systems. But it achieves this in the fine tradition of being composed of very well defined extremely simple components.
I don't think portability is a goal in itself. It is good to achieve whenever possible, but must not reduce systems to their least common denominator. There must also be room for new developments. On the other hand, if you think of pure research systems such as EROS as even better, we probably agree on this point anyway.
Filesystems are dead, or should die Posted Sep 24, 2005 1:49 UTC (Sat) by abartlet (subscriber, #3928) [Link] > For example, does anyone use NTFS' multiple streams feature, except to hide malware?
Unfortunetly yes, and this is becoming a bigger and bigger problem for Samba. As Win9X moves out of production, and applications begin to depend on NTFS, the named streams are 'just too useful'. They currently include biographical details in MS office, and as faw as we can tell will include even more in future.
The main things stopping this is traditions with e-mail and the FTP backup example described earlier. However, we expect this to be a genuine requirement from Windows clients in the future.
Andrew Bartlett
Filesystems are dead, or should die Posted Oct 1, 2005 21:01 UTC (Sat) by renox (subscriber, #23785) [Link] > The thing to do, if you insist on having a filesystem at all, is to make it as simple and safe as possible and to provide a mechanism for user code to be called when certain filesystem events happen
Funny advice that, we all know that journaling adds complexity to a FS, so safe and simple are conflicting options.
>Even better, though, is to just get rid of filesystems. See EROS and Coyotos.
Better in which sense?
Linux sets the standard. Posted Sep 22, 2005 19:25 UTC (Thu) by cventers (guest, #31465) [Link] I certainly don't "own Linux" or have the opportunity to contribute asmuch as I'd like to, which means my opinion carries no actual weight. However, I object to the statement "Linux is not an experimental or research-oriented system". True - the quality of the system as a whole is not experimental, and great emphasis is put on making the most robust production system possible. But Linux is a UNIX *clone*, and its vast and multiplying market share have given it the opportunity to set the standard for the future. Backwards compatibility is absolutely essential, but as long as we're not abandoning the core tenants of UNIX that have carried it so many years, why not utilize our market share and the excellent pool of available genius to advance computing as a whole? Eric Raymond once said that Microsoft's shoddy products and corrupt business practices set the computing industry back ten years. I couldn't agree more. But I think Linux and the rest of Open Source are in the process of pushing it forward twenty years. Hans has made some mistakes, and he doesn't share Linus's amazing talent to peacefully lead a mob. Many of the objections raised by LKML need to be addressed for Reiser4 to be merged. But objecting merely because of political reasons, a dislike of Hans/Namesys, or the idea that Linux isn't here to innovate, is totally retarded.
Linux sets the standard. Posted Sep 23, 2005 23:05 UTC (Fri) by amikins (guest, #451) [Link] An argument could be made that Linux is not there to innovate, but instead it's there to be as correct as possible.Sometimes that means doing something new, but more often that means doing something old but with refinement.
In particular, as a 'UNIX clone', there's a lot of 'tried and true' concepts that largely get upheld in designing aspects of Linux. There's a consistent logic to what came before, and sometimes deviating from that is only inviting failure.. Not because the deviation in concept is itself bad, but because it doesn't mesh with everything else in place. Might as well start over at that point.
Besides.. Isn't it the job of the people who are responsible for its current and future state to decide what Linux is here to do?
And if you don't agree.. Well, download a tarball and either apply some patches from other folks or start coding yourself. Just remember that while the result may suit your desires and needs better that way, it's not "Linux" unless the folks currently in charge of Linux decide it is, at least until someone else takes over.
Welcome to open source.
Linux sets the standard. Posted Sep 24, 2005 3:20 UTC (Sat) by cventers (guest, #31465) [Link] Believe me, I understand open source well. I also trust in LKML. The mainpoint of my comment was an objection to the idea that Linux isn't here to innovate. Being 'correct as possible' is certainly very important, and I believe that the UNIX model has obviously proven itself with its longevity. I do think some of the Reiser4 ideas (like files-as-directories) do vary from this model in potentially damaging ways. I'm not sure anyone can say for certain *what* Linux is here to do - remember, in 1992, it was nothing more than a college student's pet project. But now look at it - it's actively developed on by people and companies around the world, achieves O(1) scalability in numerous categories (managing to blow much of its competition out of the water in many areas), boasts good stability, and runs on everything from set-top DVD players to supercomputers. Whether or not Linux is here specifically to innovate, Linux *has* innovated in vast and amazing ways. It is a shining example of some of the earliest UNIX ideas - portability, simplicity, robustness. So we, the great Linux masses, and more specifically, the brains on LKML, have been given the opportunity to define the future of computing. Whatever we do, let's not be afraid to innovate.
adventurousness of kernel development Posted Sep 24, 2005 4:25 UTC (Sat) by pengo (guest, #7787) [Link] The kernel folk have changed to having faster release cycles, with shorter windows for new features, and as a result they're getting smaller things done with greater momentum. However, This is impacting the adventurousness of development.
Andrew Morton recently suggested that number of major features lined up for the kernel have been slowing down. But instead of seeing this as a symptom of a release cycle speed that doesn't allow for big new features, he sees it as a sign that the kernel is nearly "finished", whatever that means.
The core developers see it as their job to put up barriers for new ideas, shouting down anything new or different. Only the elite are good enough to prove themselves and get the ultimate "mainline merge". Sure, it works, but it's a negative and destructive culture to be creating.
So now where is the direction for development coming from? The push for Reiser4's features have come from Hans Reiser himself, and through his costant dedication he might get a subset of it might be merged.. eventually. One person slogging away to make change. What is this? government bureaucracy or something? I thought this was meant to be a community.
Of course new features are going to slow down when the barriers to new feature inclusion are so high that you need to spend years working on something, and possibly get yourself into hundreds of thousands of dollars of debt, just to create a gift that is so easily wasted.
Features need to be brainstormed, pondered, considered, played with, listed, given scenarios to work through, and allowed to grow and develop. Not just constantly shouted down. There needs to be someone in charge of managing ideas and potential new features for the linux kernel, not only a hord of hackers who know better.
I'm not saying it's wrong to criticise an idea or a patch or whatever, and certainly Linux wouldn't what it is today without Linus taking the role of chief of "saying no" to patches. But where's the other side that makes new things? It rarely seems to come from within, and worse, is rarely encouraged.
</armchair developer rant>
Now for my proposed solutions:
* How about Linus starts a 2.7 branch to encourage some new major features. This would be concurrent with the current development style, but give a place for some major changes to simmer. And would encourage developers to think about major features again.
* Reiser4 folk and people looking for then next new thing start their own distribution that only runs with a Reiser4 fs, and makes its feature visible to users as much as possible, and exploits its features as much as possible. For example, having pretty displays of file metadata in Nautilus, and creating intuitive interfaces for opening folders that are also files, or having scripts that dont care if they create a lot of little files in a directory (when it makes sense to do so). Of course it's all just a simple matter of programming. (i.e. a lot of work), but would give interested users a way to use Reiser4 and explore its features, without needing mainline inclusion.
* Kernel development should try working on wikis more. In my experience, wikis allow ideas to grow and develop better than mailing lists, which are more geared towards people making points, and are so conducive to flamewars.
</armchair developer proposals>
Pengo.
Reiser4 and kernel inclusion Posted Sep 26, 2005 17:11 UTC (Mon) by zooko (subscriber, #2589) [Link] By the way, The Economist is currently running a story about how Microsoft, Apple, and Google are working on this general idea:
http://www.economist.com/science/tq/displayStory.cfm?stor...
Hans Reiser is certainly right about the historical trend here.
Reiser4 and kernel inclusion Posted Sep 29, 2005 12:51 UTC (Thu) by leandro (guest, #1460) [Link] In the Reiser view of the future, applications like relational database managers need not exist; such tasks will be handled in the filesystem itself. This just shows why Reiser’s work will never fly. While he looks great in technical and managerial abilities, he (just like most people, including people commenting here like Wol) has got the fundamentals all wrong. The relational model prescribes a far simpler, more powerful logical interface to data than the one proposed by Reiser. Just as most people, he has believed SQL vendors’ claim of being relational and thus has thrown out the concept without ever having understood it. The situation here is somewhat similar to, but worse still than, the one in programming models: people bypass functional programming and formal methods because they think them too hard, without ever having ever tried; lack of understanding and familiarity in the beginning eventually makes for much more and worse work in the long run, and thus Java on Oracle instead of functional programming on a relational DBMS.
Reiser4 and kernel inclusion Posted Oct 4, 2005 14:09 UTC (Tue) by joelm (guest, #32437) [Link] At the end of the day...
Choice divides a path of sanity.
|
|||||||||||||