Metrics missing

Posted Jun 5, 2024 14:12 UTC (Wed) by farnz (subscriber, #17727)
In reply to: Metrics missing by koh
Parent article: Debian's /tmpest in a teapot

The big use case for /tmp that's always been OK is "write out a small file that another program will immediately take in as input". If the dirent gets deleted not long after creation, that's A-OK, because the reader has already opened it and has a lock on the inode as a result (the file contents are therefore safe until the file is closed).

Basically for the case where you can't pipe output to the input of the other program, because it needs to seek or otherwise do file-like things on it, not pipe-like things.

Metrics missing

Posted Jun 5, 2024 14:48 UTC (Wed) by koh (subscriber, #101482) [Link] (8 responses)

> "write out a small file that another program will immediately take in as input" [...] lock on the inode

I suppose I should be glad that we finally have the following sequence atomic and uninterruptible:

1. spawn child process
2. wait until child is initialized and starts running
3. wait until child decides to open *the right file*

All this time I was under the impression that it's not, but now appearently we can be sure that noone presses ctrl-z or puts the computer to sleep while compiling something...

Uses for /tmp

Posted Jun 5, 2024 15:29 UTC (Wed) by farnz (subscriber, #17727) [Link] (7 responses)

We don't have that sequence; but you need some form of error handling for that case, anyway, since it's possible that you write out the file to a drive that physically fails between write and read - and there's never been a guarantee that the failure of the drive that contains /tmp will take down the system as a whole.

Uses for /tmp

Posted Jun 5, 2024 15:37 UTC (Wed) by mb (subscriber, #50428) [Link] (6 responses)

There is a huge difference between hardware failures and "cleanup" daemons deleting files.

Drive failure: The error reaction is: Throw an error and notify the user. Change the driver and the next run will succeed.

A "cleanup" daemon deleted my files: A sane error handling strategy does not exist. Next time the "cleanup" will delete my files again. The "cleanup" is the root cause of failure.

Uses for /tmp

Posted Jun 5, 2024 16:48 UTC (Wed) by farnz (subscriber, #17727) [Link] (5 responses)

In the case where the user paused your program for multiple days, then got surprised, it's "re-run without stopping at a critical moment, and the next run will succeed". The cleanup won't delete your files again unless you have that long pause, which in the case I was responding to was introduced with Ctrl-Z or system suspend.

The underlying issue is the expectation that data in a temporary location is safe for an arbitrary amount of time; I could have replaced "Ctrl-Z" and wait with "Ctrl-Z, then rm -fr /tmp/*" for the same effect. You should already be handling "the system-wide shared temporary location might get erased while in use", because there's already lots of ways in which it could get erased behind your back; making it a tmpfs, or adding a clean-up daemon, is just one more way on the existing list of ways you could lose data in /tmp.

Uses for /tmp

Posted Jun 5, 2024 21:00 UTC (Wed) by koh (subscriber, #101482) [Link] (4 responses)

Are you saying that any process relying on writing to and reading from /tmp is unreliable (as suspend/sigstop/etc. can happen to any process)? If so, what purpose does /tmp have in case I want my process to produce a reliable result? Is there any spec stating this unreliability? So far I've also not been aware of any restriction on reliability based on the time processes (or entire machines) can work or be suspended for unless their tasks involve network or other transient resources. Basically, IIUC, you're claiming that /tmp is a transient resource during process execution. Why?

The FHS-3.0 section 3.18 just says this about /tmp:
> The /tmp directory must be made available for programs that require temporary files.
> Programs must not assume that any files or directories in /tmp are preserved between invocations of the program.

There is no wording indicating that processes can't rely, during their execution, on the integrity of the files they themselves have written (modulo system bugs and hardware errors, as usual). That indeed would be contradictory to the file system's only stated purpose: storing temporary files *required* by programs.

If my understanding is correct, periodic "cleaning" reduces reliability on purpose. tmpfs is fine, though, as long as resuming a suspend-to-disk doesn't touch it.

Uses for /tmp

Posted Jun 5, 2024 21:14 UTC (Wed) by bluca (subscriber, #118303) [Link] (3 responses)

Yes, processes using /tmp to store precious, unreplaceable data are buggy, and have always been buggy, on Linux and on Unix before it. With tmpfiles.d we are actually a bit nicer, and give you a way out: take an flock, and the file will be left alone. And of course there's the usual filesystem semantics where, as long as a FD is open, the file content is still fully accessible to the owner of the FD, even if the dentry is gone.

But for precious, precious data a state directory in /var/lib needs to be used instead. Using /tmp means tacit acknowledgement that "yes this might go away at any time, and it's fine, I'll just detect it and recreate it if needed". And that's not even getting into the issue of the various filename hijacking vulneratbilities, that are non-obvious to handle if one isn't aware of them.

Uses for /tmp

Posted Jun 5, 2024 23:18 UTC (Wed) by koh (subscriber, #101482) [Link] (2 responses)

> Yes, processes using /tmp to store precious, unreplaceable data are buggy, and have always been buggy, on Linux and on Unix before it.

Citation needed.

> Using /tmp means tacit acknowledgement that "yes this might go away at any time, and it's fine, I'll just detect it and recreate it if needed".

A fractal of bad design: how can you "detect it" in particular when (rightly so) the advice has always been to use mkstemp(3) or similar?

Uses for /tmp

Posted Jun 6, 2024 0:25 UTC (Thu) by pizza (subscriber, #46) [Link] (1 responses)

>> Yes, processes using /tmp to store precious, unreplaceable data are buggy, and have always been buggy, on Linux and on Unix before it.
> Citation needed.

For Linux: Linux FHS 2.2, section 3.15 and 3.18, and before that, section 3.11 of the FSSTND, dating all the way back to Feburary 1994 [1], which was largely derived from standard Unix conventions.

Meanwhile, for Unix I can't be bothered to look it up, but every Unix system I had access to in the mid-late 90s [2] had an aggressively-reaped /tmp. Solaris and other Unix support forums show similar answers to [1].

[1] http://www.ibiblio.org/pub/Linux/docs/fsstnd/old/fsstnd-1.0/
[2] Solaris, SunOS, and HP-UX.

Uses for /tmp

Posted Jun 6, 2024 6:38 UTC (Thu) by koh (subscriber, #101482) [Link]

> For Linux: Linux FHS 2.2, section 3.15 and 3.18, and before that, section 3.11 of the FSSTND, dating all the way back to Feburary 1994 [1], which was largely derived from standard Unix conventions.

Thanks, I see.

Nitpick: Unlike [1], the FHS 2.2 and current 3.0 do no longer mention that dentrys in /tmp are unreliable by default. In fact, they state that programs may *require* temporary files stored therein. Appearently reliably only for WORN usage, since "detection" is racey with periodic cleanups.