When and why to deprecate filesystems [LWN.net]

When and why to deprecate filesystems

Posted Mar 8, 2022 2:10 UTC (Tue) by NYKevin (subscriber, #129325)
In reply to: When and why to deprecate filesystems by pabs
Parent article: When and why to deprecate filesystems

Not all filesystems can support the same metadata in the first place. Old-school FAT, for example, rounds timestamps to a resolution of two seconds, and I'm pretty sure it entirely lacks support for POSIX file modes, much less extended nonsense like ACLs. NTFS has "alternate data streams," which I have never heard of existing under any other filesystem. There was allegedly some version of HFS+ which supported directory hard links. And so on...

This only has the slightest chance of working if both the source and the destination FS's are effectively equivalent (e.g. both ext4 with the same set of mount options). Even then, what happens if you try to copy a file (not directory) with a link count greater than one? Does it just break the link, and make a brand-new file on the destination? Or does the kernel somehow know that you were also planning to copy another link to the same file, and therefore it needs to recreate the hard link on the other side when you get around to copying that second link? rsync has enough information to figure that out, but the kernel probably does not.

to post comments

When and why to deprecate filesystems

Posted Mar 8, 2022 2:18 UTC (Tue) by pabs (subscriber, #43278) [Link] (2 responses)

Yeah, definitely a complicated feature, but the issues you bring up don't seem unsurmountable. Options to ignore certain features could be added. A recursive option could let the kernel know that yes you want that link.

When and why to deprecate filesystems

Posted Mar 8, 2022 22:42 UTC (Tue) by NYKevin (subscriber, #129325) [Link]

> Options to ignore certain features could be added.

The problem is that, in practice, either it errors out every time a feature can't be copied and wasn't explicitly excluded, or else it silently swallows the error and loses your data. That might not sound so bad, until you realize that you're going to lose a lot of rather basic information in some cases; FAT in particular is extremely lossy, and will lose symlinks (and everything else that is not a "regular file" or directory), (probably) simple permissions, and almost certainly all forms of "extended" attributes such as ACLs. I also don't believe it supports hard links, and some particularly old versions don't even support subdirectories.

> A recursive option could let the kernel know that yes you want that link.

This opens up an enormous can of worms. What happens if the process receives a signal while it's inside a recursive kernelspace copy operation? Does the whole operation fail with EINTR, and you have to start over from scratch? What happens if file #2,758 can't be copied, do you just return EIO and make the application deal with it? More generally, how do you handle exceptional circumstances while the copy is taking place?

When and why to deprecate filesystems

Posted Mar 10, 2022 2:23 UTC (Thu) by bartoc (guest, #124262) [Link]

I seem to recall some additions to `copy_file_range` to allow this in some situations, perhaps only between two different filesystems of the same type (btrfs to btrfs or so on).

When and why to deprecate filesystems

Posted Mar 8, 2022 16:38 UTC (Tue) by stevie-oh (subscriber, #130795) [Link] (4 responses)

> NTFS has "alternate data streams," which I have never heard of existing under any other filesystem

Mac OS' HFS had them first, there they are called "resource forks".

In fact, the entire reason NTFS has Alternate Data Streams is that Microsoft wanted to make it possible to use a Windows server as a file server for a group of Macs.

https://en.wikipedia.org/wiki/NTFS#Alternate_data_stream_(ADS)

When and why to deprecate filesystems

Posted Mar 9, 2022 7:44 UTC (Wed) by eru (subscriber, #2753) [Link] (3 responses)

Judging by that Wikipedia article, alternate data streams are now used for similar purposes as Linux uses extended attributes. So not an entirely useless feature.
Copying from NTFS to a modern Linux file system could use an extended attribute to simulate a stream, unless it contains too much data.

When and why to deprecate filesystems

Posted Mar 10, 2022 2:26 UTC (Thu) by bartoc (guest, #124262) [Link]

yes, however really you might want some translation (in particular for mark-of-the-web).

NTFS supports actual extended attributes too, so you have some risk of collision if you try and merge ADS and ntfs xattr.

When and why to deprecate filesystems

Posted Mar 18, 2022 19:55 UTC (Fri) by cypherpunks2 (guest, #152408) [Link] (1 responses)

ADS and xattrs aren't entirely equivalent. One important aspect of ADS is that the stream cannot be accessed without knowing the name, whereas you can list all existing xattrs for a file without knowing them.

When and why to deprecate filesystems

Posted Mar 19, 2022 23:34 UTC (Sat) by smurf (subscriber, #17840) [Link]

I find that very hard to believe, given that (a) you couldn't backup your system without that function and (b) you get pointers to numerous utilities as well as the relevant Windows kernel functions when you do a simple web search for "enumerate alternate data streams".