XFS2?

Posted Jan 23, 2012 19:07 UTC (Mon) by martinfick (subscriber, #4455)
Parent article: XFS: the filesystem of the future?

> That implies an on-disk format change. The plan, according to Dave, is to not provide any sort of forward or backward format compatibility; the format change will be a true flag day. This is being done to allow complete freedom in designing a new format that will serve XFS users for a long time.

Why not take a leaf out of the EXT book, and simply fork the filesystem at this point and call it XFS2?

XFS2?

Posted Jan 23, 2012 21:16 UTC (Mon) by dgc (subscriber, #6611) [Link] (5 responses)

No fork is needed because 99% of the code that does all the work will be common to both formats. The majority of change is in the routines that read and write the disk format, and that's a very small amount of code that is mostly isolated.

Dave.

XFS2?

Posted Jan 24, 2012 4:36 UTC (Tue) by raven667 (subscriber, #5198) [Link] (4 responses)

There is a lot of sharing between ext* as well and even compatibility of disk structures but they are still differently named.

XFS2?

Posted Jan 24, 2012 10:18 UTC (Tue) by dgm (subscriber, #49227) [Link] (3 responses)

Also, it will make the life of everybody easier, specially kernel developers trying to bisect a kernel (look at what happened when btrfs changed disk format).

You can always deprecate XFS1 format in the future if you wish.

XFS2?

Posted Jan 26, 2012 4:59 UTC (Thu) by sandeen (guest, #42852) [Link] (2 responses)

> You can always deprecate XFS1 format in the future if you wish.

Just like we successfully deprecated ext2 and ext3 - right? ;)

There is no reason to fork XFS, IMHO.

Get your facts straight

Posted Jan 26, 2012 8:27 UTC (Thu) by khim (subscriber, #9252) [Link] (1 responses)

Well, original extfs was eventually deprecated and we had no way to deprecate ext2 and/or ext3 till ext4 was mature enough. In fact ext2/ext3 switch shows perfectly why such changes are better to do that way: when you introduce disk format change you usually do that to provide some new features and if people don't need these features they can continue to use old format and old, stable, codebase.

Get your facts straight

Posted Jan 27, 2012 1:07 UTC (Fri) by dgc (subscriber, #6611) [Link]

> In fact ext2/ext3 switch shows perfectly why such changes are better
> to do that way: when you introduce disk format change you usually do
> that to provide some new features and if people don't need these
> features they can continue to use old format and old, stable, codebase.

Actually, the ext2/3/4 splits show exactly why this model doesn't work. Instead of having a single code base to maintain, you have independent code bases that have to be maintained and fixes ported across all trees. What has really happened is that the "old stable" code bases have become "old stale" code bases as ext4 has moved on.

IOWs, history has already shown that we (developers) are poor at pushing fixes made in the ext4 code base back to the ext3 and ext2 code bases,and that's often because the person that made the fix is completely unaware that the problem also exists in ext2/3. There's a reason that the "use ext4 for ext2/3" config option exists - so that one code base can be used to support all three different filesystem types.

That's a big reason for not forking or renaming XFS just for changing 1% of the code base - ongoing maintenance is far simpler and less burdensome when only one code base is used for all different versions of the filesystem....

Dave.

XFS2?

Posted Jan 23, 2012 21:26 UTC (Mon) by sandeen (guest, #42852) [Link] (5 responses)

Note also that this is not the first XFS disk format change; XFS can handle such changes gracefully. But not all changes are compatible in both directions; trying to maintain forward & backwards compatibility across all features is a trail of woe.

However the statement "no forward or backward compatibility" sounds a bit extreme; old filesystems can still be mounted & run by the newer code, the new features just won't be there, and there won't be a facility to "upgrade" an older filesystem.

Newer filesystems with newer features won't, however, be mountable by older kernels. Again, not the first time XFS has gone down this path, no forks required.

Extreme?

Posted Jan 23, 2012 22:08 UTC (Mon) by corbet (editor, #1) [Link] (4 responses)

His slide read "No attempt to provide backwards or forwards compatibility for format changes." The next one read "Flag day!". So it may be extreme, but I don't think the extremeness came from me...:)

Not Extreme

Posted Jan 23, 2012 22:11 UTC (Mon) by sandeen (guest, #42852) [Link]

I didn't say it came from you, Jonathan ;)

I just wanted to make sure people knew that old xfs filesystems were not being left in the dust (or, left only to old kernels).

Extreme?

Posted Jan 23, 2012 23:26 UTC (Mon) by dgc (subscriber, #6611) [Link] (2 responses)

I guess I didn't explain that particularly well :/

By "flag day" I mean this is a complete superblock version bump, not a new set of feature bits that some subset of the new features can be selected at mkfs time. IOWs, the on-disk format changes are an all-or-nothing change that will all land in a single release, so the functionality is either supported or it isn't....

By "No attempt to provide backwards or forwards compatibility for format changes" I mean that there is no attempt to:

- restrain the change of format in a way that older kernels can still read the metadata even in read only mode (no backwards compatibility)
- locate (and potentially limit) the new metadata fields in a way that an old format can be upgraded in place just via an on-the-fly RMW cycle (no forwards compatibility)

Hence we can simply modify the metadata block headers or records to add the new information rather than have to try to find random unused holes or padding in the metadata to locate everything. And because all XFS metadata has magic numbers in it, we can easily tell the difference between metadta formats by changing the magic numbers - that's another reason why self-describing metadata is important...

FWIW, the "find and use holes/padding" is the approach ext4 is taking to adding CRCs, so in some places they are having to use a weaker CRC (CRC16 instead of CRC32c) because there is only 16 bits of unused space in the metadata block. This is being done because the ext4 developers want to support offline addition of CRCs to a filesystem.

This, however, is exactly the sort of "compromised on-disk format" we want to avoid because it makes the implementation more complex and it doesn't provide all the information we need to detect errors that we know can happen and need to protect against. That's not an acceptable trade-off for the storage capacities we expect to be supporting in 5 years time....

I hope it's a bit clearer what I meant now ;)

Dave.

Extreme?

Posted Jan 23, 2012 23:39 UTC (Mon) by dlang (guest, #313) [Link] (1 responses)

so restating what I think you are saying

old kernels will not be able to use the new format (even in read-only mode)

new kernels will be able to use both the old and new format

once a filesystem is converted, there will be no way to 'unconvert' it.

This sounds reasonable (although may still be reason enough to call it XFS2 even if it's the same codebase supporting both on-disk formats)

what you initially said sounded more like

old kernels will not be able to use the new format

new kernels will not be able to use the old format

when you upgrade the kernel you will be forced to convert the filesystem, and there is no way to unconvert it.

This is not acceptable and is the reason people were concerned.

Extreme?

Posted Jan 24, 2012 0:18 UTC (Tue) by dgc (subscriber, #6611) [Link]

> old kernels will not be able to use the new format (even in read-only mode)

Correct.

> new kernels will be able to use both the old and new format

Correct.

> once a filesystem is converted, there will be no way to 'unconvert' it.

There is no "convert" or "unconvert" operation - the format is selected at mkfs time and it is fixed for the life of the filesystem. That's an explicit design decision (as I've previously described)....

> This sounds reasonable (although may still be reason enough to call it
> XFS2 even if it's the same codebase supporting both on-disk formats)

A name change would only cause confusion. XFS has a history of mkfs-only selectable format changes over time and this change is being handled in exactly the same manner as all the previous ones that have been made. If XFS was renamed every time the on-disk format changed then it would be XFS 17 by now. ;)

Dave.