LWN: Comments on "Change notifications for network filesystems"

Change notifications for network filesystems

bonassis — Sat, 09 Dec 2023 14:25:52 +0000

Hi.

Good to see there is attention for fsnotify methods (inotify, fanotify) and network filesystems. Earlier I've been pretty busy why this does not work on Linux, and wrote this about it (only FUSE):

https://github.com/libfuse/libfuse/wiki/Fsnotify-and-FUSE

One citation about why it does not work: "In short: the individual filesystems do not "know" a watch has been set, and thus cannot react on that."
I've written some patches back then, which made the fsnotify subsystem in Linux informs the VFS fuse kernel module when a watch has been set (or changed or removed). That worked, but later I came to the conviction that handling of this is far better of in userspace, than in the kernel.

In network filesystems, you are working in an network enviroment right? And when you want fsnotify to work in an network environment, it's because you want that applications like a filemanager, but also officesuites for example are informed about changes made by others, on other hosts (because changes made on the same host you are working on does work ...).
Now I think that users are not only interested in seeing a simple event like a file is created in a watched folder, but also by who (a user in username@domain.org notation) from what host. You can do a lot in the kernel, but you can never let a network filesystem in the kernel pass through this information. A better way to do this is doing this in userspace.

Some time ago there was a special service for that (FAM = File Alteration Monitor) but that is not used anymore.

The way it should work in my opinion:
- there is a dedicated service which offers a fs change notify service to applications
- applications can ask (via mask) what info they want: apart from pretty standard events like a file is added, removed or changed/modified, also by who, from what host and time.
- this service checks the filesystem the watch has been set upon: if its a not a networkfs and not a fuse fs use the native fs method, which is fanotify at this moment for Linux. This method provides ways to determine who (via pid) caused the event.
- when dealing with a FUSE fs, the daemon is running in userspace, then it should be not so hard to forward the watch request to this daemon. This daemon can than (if it supports this, otherwise fallback to default which is fanotify) reply what info it can handle, and possibly provide the information when an event on the backend occurs.

I'm working in a set of software (OSNS, https://github.com/stefbon/OSNS) based upon the SSH protocol, MDNS and FUSE and this is doable.
With a network filesystem in the kernel (cifs) this is harder.

I'm very interested in what you think,

best regards,

S. Bon
the Netherlands

Change notifications for network filesystems

mtodorov — Mon, 30 May 2022 11:07:23 +0000

IMHO, from the security point of view, it would be very useful i.e. to know which user is trying to modify /bin/bash on local filesystem. If this is a user named jdoe@localhost, and he is not one of the admins, then Huston we have a problem!

IMNSHO, the network file system's integrity should be the responsibility of the NSF, SMB or other server ...

A process could request (for example) IN_EVENT_UID in the list of events listened to, and the fs driver could reply with EINVAL or perhaps even more distinctive EREMOTE (Object is remote).

Change notifications for network filesystems

taladar — Mon, 30 May 2022 09:15:22 +0000

Wouldn't that be difficult for network filesystems in particular. You don't really have uids that are the same across the whole network filesystem scope (server + all clients).

Change notifications for network filesystems

mtodorov — Sun, 29 May 2022 21:58:06 +0000

P.S.

Please pardon my typo and imprecision, this should say:

"It doesn't seem to have to break anything, since programs rely on event_len rather than sizeof (struct fanotify_event_metadata) to get data."

Change notifications for network filesystems

mtodorov — Sun, 29 May 2022 21:37:56 +0000

It is however disabling not to know which user attempted to open or access a file, or caused file event.

The prudent approach may be to add uid, gid, real and effective user ids to the structure:

           struct fanotify_event_metadata {
               __u32 event_len;
               __u8 vers;
               __u8 reserved;
               __u16 metadata_len;
               __aligned_u64 mask;
               __s32 fd;
               __s32 pid;
           };

It doesn't seem to have to break anything, since programs rely on even_len rather than sizeof (struct fanotify) to get data.

Rationale: it is possible to lookup which user is the owner of the PID, however, while that information is being searched for, the process may exit already.

It also involves a race condition. And it may not be the same which user we give an access to file to. Lookup in /proc file system is expensive and inefficient. :-(

My $0.02.

Change notifications for network filesystems

Fowl — Fri, 27 May 2022 09:05:32 +0000

It seems remarkable to me that there doesn’t seem to be an abstraction on the user space APIs in the VFS layer? Or is the overlap in functionality so small that it wouldn’t be worth it?

Change notifications for network filesystems

Kamilion — Thu, 26 May 2022 20:53:04 +0000

Wait a second, I thought we 'got rid' of inotify *years* ago by stubbing it out to call fanotify?