Debating reiser4 - again [LWN.net]

Hans Reiser is nothing if not persistent. Back in October, 2002, he requested that his new reiser4 filesystem be included into the 2.5 development kernel before it went into the pre-2.6 stabilization mode. Nearly four years have passed, during which reiser4 has been through endless linux-kernel debates, numerous changes to fix problems found by reviewers, the removal of core features, and a long wait in the -mm kernel. Despite all of this, reiser4 is still not in the mainline - but Hans has not given up.

There have been a number of obstacles to overcome so far. The "files as directories" feature tweaked POSIX semantics in a way that disturbed some people, and, more importantly, had crucial locking problems; that feature has been removed. The posted benchmarks have not been entirely credible to all observers. There is concern about how committed the reiser4 developers are to ongoing support of the filesystem, once it is merged. Hans tends to have difficult relations with other kernel developers, and does not always respond entirely gracefully to (often not entirely graceful) review comments. The end result has been a difficult path toward inclusion for a filesystem which truly does offer some interesting ideas and the potential for top-level performance.

Partially as a result of a feeling that the reiser4 process has gone on for too long, the debate has returned to linux-kernel. Hans and company would like to see reiser4 put into 2.6.19, and it seems that they might just succeed.

Some outstanding issues remain, though some of them may not be as problematic as some people think. The biggest of those, probably, is the reiser4 plugin concept. Plugins allow the filesystem to behave differently for every file stored there; they can add features like compression, encryption, or many of the more esoteric things currently done with FUSE. Plugins raise all kinds of red flags in the development community. So, for example, Linus states:

As long you call them "plugins" and treat them as such, I (and I suspect a lot of other people) are totally uninterested, and in fact, a lot of people will suspect that the primary aim is to either subvert the kernel copyright rules, or at best to create a mess of incompatible semantics with no sane overlying rules for locking etc.

Jeff Garzik has concerns as well:

I don't want to be the distro support person trying to fix a crash in "reiser4", where the customer has secretly replaced the standard inode data structure with a plugin written by an intern, and secretly replaced the directory algorithm with a closed source plugin from PickYourVendor. Trying picking through that mess with a filesystem debugger.

The message for the reiser4 developers over the last few years is that any such mechanism, if it makes sense at all, should be implemented within the VFS level, rather than within any specific filesystem. Reiser4 plugins are seen as a separate, private VFS with a long potential for problems.

What a number of people have not realized, perhaps, is that the plugin issue is much smaller than it once might have been. They cannot be loaded at run time, so there should not be copyright issues like those that accompany closed-source kernel modules. And most of the plugin functionality has been removed in response to past comments. Andrew Morton, who has recently reviewed the code himself, comments:

The plugins appear to be wildly misnamed - they're just an internal abstraction layer which permits later feature additions to be added in a clean and safe manner. Certainly not worth all this fuss.

From Andrew's point of view, the biggest problems would appear to be the lack of direct I/O and extended attribute support. Direct I/O looks like it might not be too far in the future, but it does not appear that there is any immediate prospect of extended attributes. That means that, among other things, a reiser4 filesystem cannot support SELinux. That limitation may cause some distributors to leave reiser4 support out, even after reiser4 has finally been merged into the mainline kernel.

The remaining objections may be enough to dissuade some users or distributors from working with reiser4, but it would seem that they should not be enough to block the merging of reiser4 into the mainline. A new filesystem does not affect anybody who does not use it, and the bad pitfalls for reiser4 users (deadlocks, for example) should be long gone. So it may just be that Hans Reiser's long wait is nearing its end.

Index entries for this article
Kernel	Filesystems/Reiser4

to post comments

Reiser4 inclusion and social aspects

Posted Aug 3, 2006 10:14 UTC (Thu) by Jorgen.Fjeld (guest, #1038) [Link] (2 responses)

A nice and technologically oriented article, however, the lack of
inclusion also appears to have less-technical reasons.

There has been a long standing problem that the reiser4 code has not
received a full review by a kernel hacker. This is mainly due to the
complexity of the reiser4 code, although the complexity mainly stems from
the new ideas of reiser4, the kernel hackers have been very reluctant to
accept code that they, apparently, have difficulties hacking
understanding, and therefore are unable to hack efficiently on. Much of
the debate has therefore focused on whether Hans and his team are
dedicated to the task of fixing and improving reiser4, instead of starting
on a new project, such as, reiser5.
Many of the topics discussed seems to boil down to the same problem,
reluctance to adopt a beast that is very different and very large.

Hans and his team has shown incredible patience with regard to kernel
inclusion, and they have been working on reiser4 for a long time. This is
a clear sign of their commitment. With such commitment I find almost
inevitable that they will continue to actively support reiser4 for quite
some time.

The goal of Hans Reiser is after all to further file system technology,
and so far he has been very hard working on reaching that goal. I tend to
agree with Hans that it is difficult to further technology without
creating a large and different beast.

In light of this it is very promising that Andrew Morton is reviewing the
reiser4 code, both for his comments that may improve reiser4, and because
he reveals more of reiser4 to other kernel hackers, which I believe is a
social prerequisite for kernel inclusion.

Reiser4 inclusion and social aspects

Posted Aug 3, 2006 12:45 UTC (Thu) by smitty_one_each (subscriber, #28989) [Link]

This may be of interest:
http://groups.google.com/group/linux.kernel/browse_thread...

incredible patience?

Posted Aug 3, 2006 15:36 UTC (Thu) by southey (guest, #9466) [Link]

Thanks for the laugh! Perhaps you should also read 'Filesystems, Politics and the Kernel' on kerneltrap: http://kerneltrap.org/node/6876 and the linked comments.

There has been a long standing problem that the reiser4 code has not received a full review by a kernel hacker. This is mainly due to the complexity of the reiser4 code, although the complexity mainly stems from the new ideas of reiser4, the kernel hackers have been very reluctant to accept code that they, apparently, have difficulties hacking understanding, and therefore are unable to hack efficiently on.
If you read the threads, you will see that this is incorrect. It has had reviews and it is not the complexity but rather the simple things of conforming to the kernel style as well as just the sheer size of the code that appears to be the main sources of you 'long standing problem'.

Sorry, while I respect the efforts of his team, you still have to play nice with others especially when you need their help and that includes 'jumping through the hoops' as needed.

Debating reiser4 - again

Posted Aug 3, 2006 11:08 UTC (Thu) by ken (subscriber, #625) [Link] (3 responses)

This is starting to be really silly. Lets look at another filesystem like NTFS how many years did that exist in the kernel in a severly broken state.

Can't be that hard to mark reiser4 EXPERIMENTAL.

And this notion that stuff should be integrated at the vfs layer is really silly I really do not want people to put things into that layer unless the feature has been used for years and proved its usefullness.

And it's not like it's impossible to remove it if it trylly turns out to be a big messy misstake (DEVFS anyone ?)

Debating reiser4 - again

Posted Aug 3, 2006 11:17 UTC (Thu) by nix (subscriber, #2304) [Link] (2 responses)

All your points are fallacious.

NTFS inclusion is quite different from reiser4, because NTFS can be read by drivers on other OSes (notably Windows). As such, removal or BROKEN-marking of the NTFS driver was acceptable because users can still reach their data by using the other OS. This is not true of reiser4: once it's in, it can never be removed without essentially vaporizing users' data for them (or at least placing them in a quandary: change your FS (really annoying), don't upgrade and possibly get exposed to security problems, or maintain reiser4 yourself).

As for not merging stuff at the VFS layer, well, if something changes filesystem semantics and is intended for wide use it *must* go into the VFS: only that way can userspace communicate with it, and only that way can other filesystems stand a chance of using whateveritis. Whether it goes in as a library that other filesystems can use (like JBD), or as a change to the VFS layers above the filesystem is a different matter: some stuff (format-changing plugins and so on) can probably be a libfsplugin/, while files-as-directories would require upper-level VFS changes and very possibly changes to glibc (so that apps could tell whether this file can be viewed as a directory without having to regard every file as a directory and breaking all existing code that uses the f_type to determine whether to do an open() or an opendir().)

xattrs, acls, and several other such things have gone into the VFS layer, and IIRC they went in really rather early, when the first internal user went in. You can't really put this stuff off.

Debating reiser4 - again

Posted Aug 3, 2006 11:55 UTC (Thu) by ken (subscriber, #625) [Link] (1 responses)

Well xatts acls had existed for years on other platforms and had posix documents on how it should work before it was added to linux this was exactly what I meant by proven usefull.

And I really do not see any problem WHATSOEVER of removing reiser4 anythime in the future as long as it has the experimental status.

We have other filesystems that has uniq features like XFS that has a special project quota that does not exist in any other filessytem and nobody has forced them to move it into the common vfs(quota) layer.

But leaving the implementation details aside the standards for reiser4 just looks to be set so much higer than anything else thats whats looks silly to me. And I have read enough about this that no amount of argumeting is going to change my view on that so you can just save yourself some time by not even trying.

Debating reiser4 - again

Posted Aug 6, 2006 15:11 UTC (Sun) by andreashappe (subscriber, #4810) [Link]

*plonk*

a shame that this functionality is not implemented in web forums.

Renaming Plugins

Posted Aug 3, 2006 13:31 UTC (Thu) by PaulDickson (subscriber, #478) [Link]

Instead of plugins, how about "Feature Shims".

Debating reiser4 - again

Posted Aug 3, 2006 13:56 UTC (Thu) by sbergman27 (guest, #10767) [Link] (1 responses)

> From Andrew's point of view, the biggest problems would appear to be the lack of direct I/O and extended attribute support. Direct I/O looks like it might not be too far in the future, but it does not appear that there is any immediate prospect of extended attributes. That means that, among other things, a reiser4 filesystem cannot support SELinux.

I am, perhaps, beginning to change my mind on this. Perhaps Resier4 should go into mainline, even if it is not as featureful as other Linux filesystems.

Debating reiser4 - again

Posted Aug 3, 2006 19:57 UTC (Thu) by nix (subscriber, #2304) [Link]

Lots of filesystems don't support xattrs, and we have *lots* of examples of xattrs being added to filesystems after initial deployment. (Fundamentally the fs can just model them as a sort of limited per-file directory, if it needs to.)

Reiser4 - Evolution or Intelligent Design?

Posted Aug 5, 2006 5:11 UTC (Sat) by csawtell (guest, #986) [Link] (4 responses)

So it may just be that Hans Reiser's long wait is nearing its end.

I sincerely hope so. I have been using Reiser4 daily for well over a year on my ThinkPad and its USB attached external drive. To put it simply: The Reiser4 filesystem works very well indeed. Perhaps its code is not as polished as some of the other Linux Kernel packages may be, but that polishing will happen as soon as it is allowed into the evolutionary cauldron which is the Linux Kernel. Not only does Reiser4 deserve to be released from the Purgatory of Andrew Morton's tree where it's proven itself to be harmless if nothing else, but also the world deserves to have unfettered, if experimental, access to it should they so desire. Any remaining problems will either get fixed by evolution, or Reiser4 will fall by the wayside as a bad mutation, as has happened to devfs.

To play God with Reiser4 now completely belies the claim that Linux is "Evolution, not Intelligent Design". Exactly the same could be said for the suspend2 system which also works very well indeed.

Reiser4 - Evolution or Intelligent Design?

Posted Aug 6, 2006 1:28 UTC (Sun) by dlang (guest, #313) [Link] (3 responses)

umm, the world has access to reiser4

what's being asked for is for the kernel maintaineers to accept responsibility for maintaining it forever (well, 10 years or so, effectively forever)

there's no right to have anything accepted into the kernel, especially when it carries a substantial maintinance burden with it.

Reiser4 - Evolution or Intelligent Design?

Posted Aug 7, 2006 5:51 UTC (Mon) by csawtell (guest, #986) [Link] (2 responses)

umm, the world has access to reiser4
In the strict sense of the word that's true, but in practise it's not true. To the best my knowledge none of the major distributions have a module for it in the install CDs.

what's being asked for is for the kernel maintaineers to accept responsibility for maintaining it forever (well, 10 years or so, effectively forever)
You mean like devfs was?
If Reiser4 turns out to be a bad-mutation just like devfs was, it can be removed, especially if it's marked as experimental until it's more proven.

there's no right to have anything accepted into the kernel,
True.

especially when it carries a substantial maintinance burden with it.
Does it, really? I don't think Hans Reiser and his team, have plans to walk off this mortal coil as soon Reiser4 is accepted into the mainline kernel. Note that Reiser4 works exceptionally well _right now_.

When making any technical decision, it's important to keep the mind clear of influences caused by personality differences, nationalistic feelings, or earlier misunderstandings.

Removing coe from mainline

Posted Aug 7, 2006 15:58 UTC (Mon) by kpower (guest, #37136) [Link] (1 responses)

what's being asked for is for the kernel maintaineers to accept responsibility for maintaining it forever (well, 10 years or so, effectively forever)

You mean like devfs was? If Reiser4 turns out to be a bad-mutation just like devfs was, it can be removed, especially if it's marked as experimental until it's more proven.

When it was time to remove devfs, users were given two different upgrade paths: return to a static /dev, or use sysfs/udev In part this was possible because the kernel developers had the technical knowledge, expertise and experience with the code and problem in order to provide a solution. This despite the disappearance of the originator of devfs.

If it came time to remove reiserfs4 from the kernel, could the current or future kernel developers provide the same service to users of reiserfs4? That appears to be the motivation to have the reiserfs code comply with kernel code guidelines.

Labeling new code as experiemental obviously helps, as that indicates DO NOT USE IN PRODUCTION, but there comes a point when that label goes away. Sometimes it's not even applied initially. That is what the kernel developers reviews are attempting to address, just like most other projects have to.

Removing coe from mainline

Posted Aug 7, 2006 17:44 UTC (Mon) by job (guest, #670) [Link]

Backup, re-mkfs, restore. It's even easier than moving from devfs to
static /dev, where you have to make sure manually that all your nodes are
in place as all may be not be present at any given moment.

Debating reiser4 - again

Posted Aug 6, 2006 5:17 UTC (Sun) by Lovechild (guest, #3592) [Link]

The fun thing is that right before this whole xattr thing hit LKML I actually approached Hans about implementing it as a reiser4 plugin since they already have all the intrastructure needed. Nate Diller promptly jumped on the task and we should hopefully see this implemented soon.

The Reiser4 developers are incredibly responsive to such requests as they serve a clear purpose of getting into the kernel and further into distributions. It's useless to them just to be in the kernel unless vendors also buy into adding support and that won't happen unless you can have SELinux, Beagle and all those nice features that require xattr.

I, for one, welcome Reiser4 on my system once I can install Fedora on it (and I started work on this support myself) just like I would ext3.