What goes into default Debian?

Posted Feb 19, 2021 6:11 UTC (Fri) by pabs (subscriber, #43278)
In reply to: What goes into default Debian? by chris.sykes
Parent article: What goes into default Debian?

The Linux kernel's fanotify interface should be enough for that, although it does require all those things to have a daemon running constantly storing the filesystem events that happen.

What goes into default Debian?

Posted Feb 19, 2021 11:44 UTC (Fri) by chris.sykes (subscriber, #54374) [Link] (4 responses)

Yeah, the multiple independent daemons all sat in *notify was what I thought it might be nice to avoid.

An analogy might be something like crond, but for file-system changes. It would maintain persistent state between boots, and could implement fs specific optimisations.

But as I say, I haven't really thought this through :-)

What goes into default Debian?

Posted Feb 19, 2021 14:17 UTC (Fri) by mathstuf (subscriber, #69389) [Link] (1 responses)

> But as I say, I haven't really thought this through :-)

The easy hole is "the filesystem can be mounted outside of your machine". Unless you're doing Secure Boot chaining to the unlocking of your disk or something, guaranteeing the filesystem is untouched while your daemon is not running is probably…very difficult.

What goes into default Debian?

Posted Feb 20, 2021 7:22 UTC (Sat) by pabs (subscriber, #43278) [Link]

You would just have to re-scan the full system on every daemon start.

What goes into default Debian?

Posted Feb 20, 2021 7:25 UTC (Sat) by pabs (subscriber, #43278) [Link] (1 responses)

There are various inotify-based cron-like daemons, but your idea is beginning to sound like it should become a core systemd service.

What goes into default Debian?

Posted Feb 20, 2021 14:51 UTC (Sat) by mathstuf (subscriber, #69389) [Link]

There are .path units what can trigger other units on various state changes already. They don't work with directory trees so well though IIRC.

What goes into default Debian?

Posted Feb 20, 2021 5:20 UTC (Sat) by zblaxell (subscriber, #26385) [Link]

fanotify receivers can easily fall behind if there are too many events.

I once tried to make a build system with inferred dependency tracking using fanotify. On paper it's straightforward: fanotify gives a stream of events like "pid P opened file F for reading and G for writing", and from that we infer G depends on F. So we run the build and log all the file accesses and then process the log later and we have at least a rough idea of what file outputs use what other files as inputs. Easy.

Now try this with a build that has 4000 processes opening a million files on a dozen CPU cores. In nonblocking mode, the events can't be dequeued fast enough (there certainly isn't time to resolve filenames), and in blocking mode it dramatically slows down the build.

Same problem for incremental backups: the fanotify monitor plays along for a while, then a lot of files get updated at once and the monitor has to say "nope, can't do it, you'll have to do a full scan to find out what I missed."