O_DIRECTORY|O_TMPFILE [LWN.net]

O_DIRECTORY|O_TMPFILE

Posted Aug 2, 2026 3:01 UTC (Sun) by koverstreet (subscriber, #4296)
In reply to: O_DIRECTORY|O_TMPFILE by donald.buczek
Parent article: Reconsidering O_CREAT|O_DIRECTORY

Given how we've moved towards referencing directories by file handle, it's not a crazy ask, and it would solve real problems.

It'd create others, though: unlinked files are a problem for observability; if you forget to kill something, or a program is doing something unexpected, it makes "where'd my disk space go" into a real game.

Private mount namespaces can create similar problems.

So, as the bcachefs maintainer: I'd be happy to take patches or implement O_DIRECTORY|O_TMPFILE in bcachefs - after we have a solution to that one.

It'd need major cooperation between the VFS and the filesystem. The filesystem would have to implement rm -rf in .evict(), and that means on disk format changes because unlinked files have to be processed in recovery, and an unlinked file is not quite the same thing as an unlinked directory.

You'd need the VFS to track when the tree has gone away, and the dcache would have to be able to operate on disconnected subtrees. The VFS does this sort of tracking today only in the context of a mount, not a random disconnected directory, to my knowledge.

to post comments

O_DIRECTORY|O_TMPFILE

Posted Aug 2, 2026 5:37 UTC (Sun) by donald.buczek (subscriber, #112892) [Link] (1 responses)

It'd create others, though: unlinked files are a problem for observability; if you forget to kill something, or a program is doing something unexpected, it makes "where'd my disk space go" into a real game.

Yes, but we already have that with unlinked regular files.

So, as the bcachefs maintainer: I'd be happy to take patches or implement O_DIRECTORY|O_TMPFILE in bcachefs - after we have a solution to that one.

Would anyone other than you be able to do that? There is no point even asking my employer to release me from work for the last few years leading up to retirement. :-)

You'd need the VFS to track when the tree has gone away, and the dcache would have to be able to operate on disconnected subtrees.

So I think an implementation for tmpfs, which already has its own independent value, would indeed be a good first target and an implementation on a persistent filesystem might come after that.

O_DIRECTORY|O_TMPFILE

Posted Aug 2, 2026 7:33 UTC (Sun) by koverstreet (subscriber, #4296) [Link]

> Would anyone other than you be able to do that? There is no point even asking my employer to release me from work for the last few years leading up to retirement. :-)

Your estimate is a couple years for figuring out an API for tracking and viewing unlinked-but-open files? Yeesh.

I get pretty scathing when people pile on complexity without thinking about how it'll be debugged. There's too many opaque black boxes already in the kernel.

My rule is, before you pile something on, you need to make sure what you already have is understandable.

O_DIRECTORY|O_TMPFILE

Posted Aug 3, 2026 6:48 UTC (Mon) by neilbrown (subscriber, #359) [Link] (6 responses)

The filesystem would have to implement rm -rf in .evict(), and that means on disk format changes because unlinked files have to be processed in recovery, and an unlinked file is not quite the same thing as an unlinked directory.

The "rm -rf' in .evict() would be lazy I think and could proceed breadth-first rather than the depth-first that "rm -r" does. It would do a "readdir" and attach each directory to the list of unattached directories, and schedule a truncate of each file however that normally happens. The unattached directories would be recorded however unattached files are (and orphan list?) and on recovery you wouldn't need to process them immediately, but simply add them to the list of unattached directories. Then let the lazy cleanup continue.

I think this might be sufficiently generic that the VFS could provide code to handle it.

Then we could add at AT_RECURSIVE flag to be used with AT_REMOVEDIR in unlinkat(). Wouldn't that be fun!

O_DIRECTORY|O_TMPFILE

Posted Aug 4, 2026 16:54 UTC (Tue) by koverstreet (subscriber, #4296) [Link] (5 responses)

Well, if you've got an unlinked directory with files in it, the filesystem has to handle that in recovery. But breadth first is a good point, that does simplify.

And I like your AT_RECURSIVE|AT_REMOVEDIR idea. I think there's actually no need for unlinked-but-open directories, it's the rm -rf on close that we want; that means we don't have open but invisible files and it might even be possible to use in bash scripts. Done that way, maybe it could be a purely VFS thing.

O_DIRECTORY|O_TMPFILE

Posted Aug 4, 2026 23:16 UTC (Tue) by neilbrown (subscriber, #359) [Link] (4 responses)

I think there's actually no need for unlinked-but-open directories, it's the rm -rf on close that we want

"man 2 open" identifies 4 important aspects of O_TMPFILE:

are automatically deleted when closed;
can never be reached via any pathname;
are not subject to symlink attacks; and
do not require the caller to devise unique names.

I think these are all important. However this is the VFS view, not the internal FS view. For example, I imagine a filesystem could implement these just like regular directories which are subdirs of the root, but with names that contain a "/'. .iterate_shared() would skip over these. This would answer 2 and 3. The fs would choose a unique name (sprintf(s, "/%d", i->i_ino) which would answer 4. Then you just need the "rm -rf" on close which the VFS could handle most of.

Maybe the fs would provide:

.tmpdir() which does .mkdir() in the root directory with a synthesized name
.mvtmp() which take a dentry and moves it to the root with a sythesized name
.iterate_tmp() which is like iterate_shared() but returns dentries for the hidden objects, which can be passed to .unlink() or .rmdir()

The VFS could have code that uses iterate_tmp() and everything that it finds which isn't open is unlink()ed if a file, rmdir()ed if an empty directory, or iterate_shared() with everything passed to .mvtmp() (or maybe unlink()). This code would be called at mount time and whenever a tmp file/dir is closed for the last time.

So yes, the FS doesn't need to have a concept of unlinked-but-open, but the VFS must have access to "open, but no link that is visible/accessable".

O_DIRECTORY|O_TMPFILE

Posted Aug 5, 2026 2:34 UTC (Wed) by koverstreet (subscriber, #4296) [Link] (3 responses)

Why is "can not be reached via any pathname" important, though?

It breaks the recursively enumerable property of filesystems; i.e. I can recursively enumerate to find all the consumed resources. I'm not a fan of private mount namespaces for the same reason :)

And considering that the implementation you're proposing has them having dirents anyways - to be that's another reason to consider just skipping the "unlinked directories idea".

> For example, I imagine a filesystem could implement these just like regular directories which are subdirs of the root

Not of the root, they'd need to be subdirectories of the directory that userspace specified. Project quota in practice already inherits from the parent directory on other filesystems, and on bcachefs IO path options inherit from the parent directory.

O_DIRECTORY|O_TMPFILE

Posted Aug 5, 2026 3:34 UTC (Wed) by neilbrown (subscriber, #359) [Link] (2 responses)

Why is "can not be reached via any pathname" important, though?

I suspect that is primarily about supporting "are not subject to symlink attacks", though that ends up meaning a bit more with directories.

I think the key idea is that no other process has any write capability to anything in the "TMP" file or tree, so that it is easier to write safe code that cannot be subverted. I suspect read access is not so important - normal permissions should be enough. Not giving a name to something is a simple way to avoid anyone getting any capability at all.

If we were to add an FMODE flag which denies write access and is inherited through openat() and friends, then I think it would be safe to allow these TMP dirs to have a name as long as the new FMODE flag was imposed whenever a path lookup crossed into a TMP dir via the name. It isn't a obviously-secure as not having a name, but maybe it is close enough.

Of course unlinked things have names via /proc/$PID/fd/$FD so maybe having a name in the fs doesn't reduce access.

Not of the root, they'd need to be subdirectories of the directory that userspace specified.

Sure, put them wherever you like as long as they can be found with .iterate_tmp(). I was just presenting a proof-of-concept approach.

Project quota in practice already inherits from the parent directory on other filesystems, and on bcachefs IO path options inherit from the parent directory.

When you create the tmp directory inode it would certainly make sense to inherit from the given parent. Whether it then needs to be linked from that parent or not would depend on internal implementation details of the FS.

O_DIRECTORY|O_TMPFILE

Posted Aug 5, 2026 18:18 UTC (Wed) by koverstreet (subscriber, #4296) [Link] (1 responses)

> I think the key idea is that no other process has any write capability to anything in the "TMP" file or tree, so that it is easier to write safe code that cannot be subverted. I suspect read access is not so important - normal permissions should be enough. Not giving a name to something is a simple way to avoid anyone getting any capability at all.

Well, I think you described the real issue right there, a shared world writeable /tmp was never a good idea :) We've been going through contortions and pointing at symlinks, but really that was the biggest footgun. It boggles my mind that a world writeable /tmp is still the default for most code, that one is plenty fixable.

Similarly with programs invoked by sudo and unprepared for all the ways that they can be maliciously pointed at something they shouldn't write to. There's a better solution to that one, but we've been stuck with the Unix security model for too long and it's become too ingrained.

The question is if it's even possible to bring a better security model to Linux, and I'm not sure about that one. The biggest thing that needs fixing is that users and groups are integer namespaces when they really should be heirarchical; Plan 9 representing users and groups as strings everywhere was a step in the right direction, and extending that and making users paths gets you something much saner than Linux namespaces. And there's been a lot of development on capability based systems, but it's hard to see doing that well in a system where APIs are still C.

Just consider this article, where a lot of bits and brain cycles are being devoted to "can we extend bitflags to represent a new mode without screwing something else up". That sort of thing is a complete non issue when you've got algebraic data types - this clearly wants to be a Rust style enum.

We're in an awkward moment in the history of systems engineering... things should be getting a whole lot better, cleaner, easier to reason about over the next decade, if we can put in the work to clean stuff up and make the Rust transition happen (among other things). But it's going to be a lot of tedious grindy work and messy transitions before we get there and can start doing the genuinely fun and interesting stuff.

This thread is of real interest to me because Valve keeps nagging me about "can we get better filesystem interfaces to solve stuff that's been a giant pain in the backside of userspace for decades?", and I see this thread and O_DIRECTORY|O_TMPFILE and my first reaction is "OF COURSE WE WANT THAT! WE'VE WANTED THAT FOR YEARS!".

But then by the time monthly meetings roll around I have to put on my hard nosed realist hat and tell them "yes, that's a really good idea and something we have all the pieces to give you, but right now I have to focus on getting in-kernel Rust out the door and getting that transition underway - what you want is going to be much cleaner and more straightforward once that's done".

Anyways, filing this away for who knows when :)

O_DIRECTORY|O_TMPFILE

Posted Aug 17, 2026 9:27 UTC (Mon) by taladar (subscriber, #68407) [Link]

I am not sure introducing yet another hierarchy is such a great idea. Hierarchies tend to be bad for composition. We have seen that with inheritance-based interface systems in languages like C++. Something like Rust's traits compose much better and in the security domain in particular we have use cases for that too.

One simple use case that hierarchies are bad at comes up when you need to look at the same objects from different angles, e.g. organisational structure in one case, physical limits in another. Say you want to give everyone who works under manager x certain permissions, everyone with a certain seniority certain permissions and everyone who works in a specific building certain permissions. Those could all include the same person but you can't have them in one hierarchy.

In practice I have encountered this e.g. with cgroups when trying to setup shared memory limits for systemd-oomd for systemd --user units owned by different users (the sum of memory used by the processes of a specific unit across all users shouldn't exceed a certain limit, e.g. PHP-FPM pools for all vhosts).