Uses for /tmp

Posted Jun 5, 2024 15:29 UTC (Wed) by farnz (subscriber, #17727)
In reply to: Metrics missing by koh
Parent article: Debian's /tmpest in a teapot

We don't have that sequence; but you need some form of error handling for that case, anyway, since it's possible that you write out the file to a drive that physically fails between write and read - and there's never been a guarantee that the failure of the drive that contains /tmp will take down the system as a whole.

Uses for /tmp

Posted Jun 5, 2024 15:37 UTC (Wed) by mb (subscriber, #50428) [Link] (6 responses)

There is a huge difference between hardware failures and "cleanup" daemons deleting files.

Drive failure: The error reaction is: Throw an error and notify the user. Change the driver and the next run will succeed.

A "cleanup" daemon deleted my files: A sane error handling strategy does not exist. Next time the "cleanup" will delete my files again. The "cleanup" is the root cause of failure.

Uses for /tmp

Posted Jun 5, 2024 16:48 UTC (Wed) by farnz (subscriber, #17727) [Link] (5 responses)

In the case where the user paused your program for multiple days, then got surprised, it's "re-run without stopping at a critical moment, and the next run will succeed". The cleanup won't delete your files again unless you have that long pause, which in the case I was responding to was introduced with Ctrl-Z or system suspend.

The underlying issue is the expectation that data in a temporary location is safe for an arbitrary amount of time; I could have replaced "Ctrl-Z" and wait with "Ctrl-Z, then rm -fr /tmp/*" for the same effect. You should already be handling "the system-wide shared temporary location might get erased while in use", because there's already lots of ways in which it could get erased behind your back; making it a tmpfs, or adding a clean-up daemon, is just one more way on the existing list of ways you could lose data in /tmp.

Uses for /tmp

Posted Jun 5, 2024 21:00 UTC (Wed) by koh (subscriber, #101482) [Link] (4 responses)

Are you saying that any process relying on writing to and reading from /tmp is unreliable (as suspend/sigstop/etc. can happen to any process)? If so, what purpose does /tmp have in case I want my process to produce a reliable result? Is there any spec stating this unreliability? So far I've also not been aware of any restriction on reliability based on the time processes (or entire machines) can work or be suspended for unless their tasks involve network or other transient resources. Basically, IIUC, you're claiming that /tmp is a transient resource during process execution. Why?

The FHS-3.0 section 3.18 just says this about /tmp:
> The /tmp directory must be made available for programs that require temporary files.
> Programs must not assume that any files or directories in /tmp are preserved between invocations of the program.

There is no wording indicating that processes can't rely, during their execution, on the integrity of the files they themselves have written (modulo system bugs and hardware errors, as usual). That indeed would be contradictory to the file system's only stated purpose: storing temporary files *required* by programs.

If my understanding is correct, periodic "cleaning" reduces reliability on purpose. tmpfs is fine, though, as long as resuming a suspend-to-disk doesn't touch it.

Uses for /tmp

Posted Jun 5, 2024 21:14 UTC (Wed) by bluca (subscriber, #118303) [Link] (3 responses)

Yes, processes using /tmp to store precious, unreplaceable data are buggy, and have always been buggy, on Linux and on Unix before it. With tmpfiles.d we are actually a bit nicer, and give you a way out: take an flock, and the file will be left alone. And of course there's the usual filesystem semantics where, as long as a FD is open, the file content is still fully accessible to the owner of the FD, even if the dentry is gone.

But for precious, precious data a state directory in /var/lib needs to be used instead. Using /tmp means tacit acknowledgement that "yes this might go away at any time, and it's fine, I'll just detect it and recreate it if needed". And that's not even getting into the issue of the various filename hijacking vulneratbilities, that are non-obvious to handle if one isn't aware of them.

Uses for /tmp

Posted Jun 5, 2024 23:18 UTC (Wed) by koh (subscriber, #101482) [Link] (2 responses)

> Yes, processes using /tmp to store precious, unreplaceable data are buggy, and have always been buggy, on Linux and on Unix before it.

Citation needed.

> Using /tmp means tacit acknowledgement that "yes this might go away at any time, and it's fine, I'll just detect it and recreate it if needed".

A fractal of bad design: how can you "detect it" in particular when (rightly so) the advice has always been to use mkstemp(3) or similar?

Uses for /tmp

Posted Jun 6, 2024 0:25 UTC (Thu) by pizza (subscriber, #46) [Link] (1 responses)

>> Yes, processes using /tmp to store precious, unreplaceable data are buggy, and have always been buggy, on Linux and on Unix before it.
> Citation needed.

For Linux: Linux FHS 2.2, section 3.15 and 3.18, and before that, section 3.11 of the FSSTND, dating all the way back to Feburary 1994 [1], which was largely derived from standard Unix conventions.

Meanwhile, for Unix I can't be bothered to look it up, but every Unix system I had access to in the mid-late 90s [2] had an aggressively-reaped /tmp. Solaris and other Unix support forums show similar answers to [1].

[1] http://www.ibiblio.org/pub/Linux/docs/fsstnd/old/fsstnd-1.0/
[2] Solaris, SunOS, and HP-UX.

Uses for /tmp

Posted Jun 6, 2024 6:38 UTC (Thu) by koh (subscriber, #101482) [Link]

> For Linux: Linux FHS 2.2, section 3.15 and 3.18, and before that, section 3.11 of the FSSTND, dating all the way back to Feburary 1994 [1], which was largely derived from standard Unix conventions.

Thanks, I see.

Nitpick: Unlike [1], the FHS 2.2 and current 3.0 do no longer mention that dentrys in /tmp are unreliable by default. In fact, they state that programs may *require* temporary files stored therein. Appearently reliably only for WORN usage, since "detection" is racey with periodic cleanups.