WAFL != ZFS

Posted Jun 12, 2007 21:56 UTC (Tue) by qu1j0t3 (guest, #25786)
Parent article: Linus on GPLv3 and ZFS

I have to assume Linus knows that. Sigh. If not, like another poster here, he should Google... I'm tired of posting ZFS linkfests ;-)

He's treading close to the FUD-line with this one. There's also a hidden assumption here that Jonathan Schwartz is being disingenuous with his massively revamped corporate strategy.

Sun's a hardware company. They're happy for you to run Linux on your Sun gear if you prefer - it's a supported option - heck, they even support Windows.

to post comments

WAFL != ZFS

Posted Jun 12, 2007 23:03 UTC (Tue) by allesfresser (guest, #216) [Link] (16 responses)

I don't think it's anywhere near FUD, personally. It sounded simply like classic Linus--he's being very transparently honest. He hopes and wishes that Schwartz and company are being as open and forthright as they claim to be, but knowing human nature and the temptations that beset us, he is keeping his powder dry and his head down, so to speak.

forthright != Linus

Posted Jun 12, 2007 23:43 UTC (Tue) by qu1j0t3 (guest, #25786) [Link] (15 responses)

It's not very "forthright" to inject snide Linusisms such as "the only really interesting thing they have is ZFS, and even there, I suspect we'd be better off talking to NetApp"! XFS is already integrated, and that has about as much in common with ZFS as WAFL does.

It comes across as sour grapes about the license, and even some N-I-H ("core Solaris: who are you kidding - Linux code is _better_"). Btw, there is as much spurious rancor of the opposite polarity from the Sun camp, as recent zfs-discuss flamefests can attest.

Why can't we all just get along? - Admit that some people like BSD license, some people like GPL, Sun likes CDDL for now, and ZFS plain rocks... :)

Linux devs ignore it at their peril; Linus, being an engineer of Sun's calibre, could do a much more helpful job of deconstructing the issue.

forthright != Linus

Posted Jun 13, 2007 0:03 UTC (Wed) by JoeBuck (guest, #2330) [Link] (5 responses)

The issue with Sun is not that they prefer a particular license, but that they are choosing to license patents only to code that uses their particular license, while IBM, Red Hat, Novell, and others are licensing a number of patents (or in Red Hat's case, all their patents) to developers who use a much larger set of open source licenses.

okay but,

Posted Jun 13, 2007 0:21 UTC (Wed) by qu1j0t3 (guest, #25786) [Link] (4 responses)

This still does not fully explain to me why, to date, kernel devs aren't looking dispassionately as the affordances of ZFS and how they might have them without stepping on anyone's patent*. Max V. Yudin recently asked on zfs-discuss,

... is it legal to write ZFS clone from scratch while maintaining binary compatibility with original?
Jeff mentioned in his blog that Sun filled 56 patents on ZFS related technologies. Can anybody from the company provide me with more information about this?
If porting ZFS to Linux kernel is not an option and I were to implement different file system with ZFS ideas in mind how can I be safe and not break any Sun patents?

There has been no meaningful resolution of his questions. At least it may prove that, thanks to software patents, interesting development is now impossible. So much for stimulating innovation...

* - I suppose NetApp has patents too, but perhaps Linus wishes to imply that they would be more tractable to deal with than Sun (maybe he actually knows somebody @ NetApp). Let's dream for a moment, and imagine that Linus and Jonathan, over a piña colada one Sunday, work out a magical way to free ZFS for kernel inclusion. That would be a P/R coup for Sun an order of magnitude greater than even the Apple buzz. Since Solaris 10 famously runs on all varieties of hardware (IBM, HP, Dell, even Macs), I don't seriously think Jonathan believes this would damage hardware sales. Then again, I only have the ponytail, not an MBA, and my bonuses are a few zeroes short of his. ;-)

okay but,

Posted Jun 13, 2007 7:57 UTC (Wed) by TRS-80 (guest, #1804) [Link] (2 responses)

One could always start from Sun's GPLv2 ZFS code in GRUB. And Jonathon Schwartz has just posted saying Linux ZFS would have full patent indemnity.

okay but,

Posted Jun 13, 2007 9:38 UTC (Wed) by mjthayer (guest, #39183) [Link]

That said "GPL v2 or later" if I read correctly. I didn't take the time to read the code, but presumably that is only code for reading and would not affect potential patents on writing parts.

On another note, if Sun make Solaris GPL3 and accept external contributions, it might get tricky to keep parts (i.e. ZFS) under another licence.

okay but,

Posted Jun 13, 2007 19:51 UTC (Wed) by dlang (guest, #313) [Link]

note that the zfs code released for grub is not enough to actually be able to write to the filesystem, just enough for grub to be able to find the files that it needs.

Netapp and patents

Posted Jun 22, 2007 7:10 UTC (Fri) by anton (subscriber, #25547) [Link]

I suppose NetApp has patents too, but perhaps Linus wishes to imply that they would be more tractable to deal with than Sun

Yes, Netapp has patents, and they caused Daniel Phillips to stop working on the tux2 filesystem; I have not followed the story enough to know if Netapp did anything other than file the patents to achieve this result.

Netapp's WAFL is not very interesting for Linux anyway, because it requires special NVRAM hardware to buffer writes during some of the more time-consuming operations (e.g., snapshot creation). I don't think that this hardware dependence can be eliminated without major changes to the WAFL code.

Concerning not breaking Sun patents, you can look for older sources where similar ideas have been described, e.g., various papers on log-structured file systems, e.g., our Freenix 2000 paper, or (maybe too young) my file system ideas.

ZFS and WAFL

Posted Jun 15, 2007 0:02 UTC (Fri) by joern (guest, #22392) [Link] (8 responses)

> XFS is already integrated, and that has about as much in common with ZFS as WAFL does.

That is plain wrong. XFS is in the huge class of traditional filesystems with a static mapping between file offsets and device offsets. ZFS is in the somewhat smaller (ignoring reseach projects) class of COW filesystems, just like WAFL. Anyone unable to see the similarities is well advised to read more and write less. ;)

ZFS and WAFL

Posted Jun 17, 2007 15:30 UTC (Sun) by qu1j0t3 (guest, #25786) [Link] (7 responses)

As you see I haven't studied XFS in depth, but I was under the impression it was COW like ZFS. AFAIK, WAFL also lacks the more interesting features of ZFS (foremost being end-to-end checksumming).

ZFS and WAFL

Posted Jun 17, 2007 17:29 UTC (Sun) by joern (guest, #22392) [Link] (6 responses)

Checksums are easy to add once you have a COW format. Either you add them to the block pointers, as ZFS did, or you add them to the objects themselves, as JFFS2 and LogFS did.

Either way you have an incompatible format change. But the amount of code affected if rather small. Took about 1-2% of the effort to design a new filesystem in the LogFS case.

btrfs?

Posted Jun 17, 2007 18:06 UTC (Sun) by qu1j0t3 (guest, #25786) [Link] (5 responses)

Maybe so, but there's quite a lot of catch-up to do. Once you have COW, transactions, and checksums, then you want self-healing; then snapshots; pools; quotas; compression; and so on, until you eventually have something like ZFS. :)

Linus' grandstanding aside, it's possible there is quiet work going on to improve the situation, as David Magda commented on zfs-discuss:

Somewhat off topic, but it seems that someone released a COW file system for Linux (currently in 'alpha'):

	* Extent based file storage (2^64 max file size)
	* Space efficient packing of small files
	* Space efficient indexed directories
	* Dynamic inode allocation
	* Writable snapshots
	* Subvolumes (separate internal filesystem roots)
	- Object level mirroring and striping
	* Checksums on data and metadata (multiple algorithms available)
	- Strong integration with device mapper for multiple device support
	- Online filesystem check
	* Very fast offline filesystem check
	- Efficient incremental backup and FS mirroring

	http://lkml.org/lkml/2007/6/12/242
	http://oss.oracle.com/~mason/btrfs/

Via Storage Mojo

btrfs?

Posted Jun 17, 2007 19:05 UTC (Sun) by joern (guest, #22392) [Link] (4 responses)

> Maybe so, but there's quite a lot of catch-up to do. Once you have COW, transactions, and checksums, then you want self-healing; then snapshots; pools; quotas; compression; and so on, until you eventually have something like ZFS. :)

Sure, ZFS has an impressive set of features. If nothing else, it has showed how things can be done. And I have little doubt that btrfs, which you quoted, will end up having most of those features relatively soon. And even if Chris dies tomorrow, I'll keep working on LogFS.

The only question is when, not if. :)

good to hear

Posted Jun 17, 2007 19:15 UTC (Sun) by qu1j0t3 (guest, #25786) [Link] (3 responses)

The only question is when, not if. :)

From comments I've heard, and the buzz around Linus' and Jon's posts, there seems to be considerable community interest around ZFS (I don't want to use anything else, or wait, so I switched to Solaris 10 some time ago).

Look forward to more news on this front. Did you follow-up on LKML? (I have not checked :)

good to hear

Posted Jun 17, 2007 21:43 UTC (Sun) by joern (guest, #22392) [Link] (2 responses)

> From comments I've heard, and the buzz around Linus' and Jon's posts, there seems to be considerable community interest around ZFS (I don't want to use anything else, or wait, so I switched to Solaris 10 some time ago).

> Look forward to more news on this front. Did you follow-up on LKML? (I have not checked :)

My personal interest is in flash, not hard disks. Therefore ZFS is impressive technology, but solving someone else's problem. It is not the last word in filesystems either, as the fsck will run for hours or days if it ever becomes necessary. So there remain valid reasons to work on different filesystems.

Impressive technology none the less.

COW for Flash?

Posted Jun 18, 2007 17:26 UTC (Mon) by qu1j0t3 (guest, #25786) [Link] (1 responses)

It has been said that COW is ideal for Flash. Can you explain why ZFS isn't relevant here?

There is no fsck; ZFS is "always consistent on disk" (through COW+atomic transactions). It seems to me this is a necessary invariant to achieve its other features (such as snapshots). Debate flares up (occasionally) as to whether a scavenger will be necessary. If so, it won't much resemble 'fsck' - and certainly won't be run in normal operation or after reset/powerfail/etc (ZFS behaviour under impromptu reset is extremely well tested).

I suspect, but correct me if I'm wrong, once you "know" you've lost data in ZFS (through exhausting redundancy or ditto blocks), it's actually gone by definition, and unrecoverable by re-creating links. No doubt Bonwick et all have explained it better somewhere...

COW for Flash?

Posted Jun 19, 2007 8:17 UTC (Tue) by joern (guest, #22392) [Link]

> It has been said that COW is ideal for Flash. Can you explain why ZFS isn't relevant here?

Raw flash behaves sufficiently different to hard disks that some ZFS design assumptions become untrue. Flash has large erase blocks. Within erase blocks, data must be written from front to back. Writing the block again requires erasing all of it. So the filesystem block size either has to be equal to the erase block size, or you need garbage collection. And with garbage collection comes a nasty deadlock problem most people don't even realize exists. :)

Next comes wear out and protection against it. Afaics, ZFS has several hot zones that receive significantly more writes than others.

I guess those two are the big reasons.