Wheeler: Fixing Unix/Linux/POSIX Filenames
Posted Mar 26, 2009 22:12 UTC (Thu) by nix
In reply to: Wheeler: Fixing Unix/Linux/POSIX Filenames
Parent article: Wheeler: Fixing Unix/Linux/POSIX Filenames
David, thanks for responding.
Such a key-value storage will have trouble with "/" in the key, since it's
the directory separator. So if you truly need arbitrary keys, you already
have to do some encoding anyway - so why not encode to something more
convenient? If you don't need arbitrary encoding, then let's find some
reasonable limits that stop the worst of the bleeding. Also, there's no
need to have all those weird filenames merged with other stuff in the same
directory; you could create a single directory with "." as the first
character in the name, and create the key-value store in that
I claim mental block: this solution became obvious to me a day or so back.
(Rather, since I already use . as a metacharacter to mean 'private',
use .. to mean 'extra-private: metadata'. Yes, this too is bizarre, but at
least it's not dash-prepended.)
But I have seen a system in production use at Big Banks (first saw
it yesterday, or first noticed it, probably thanks to this conversation)
that uses the filesystem as a base-254-of-key to value store. It's gross
but it's sometimes done.
But then, we know how competent Big Banks are. (Especially this one, did
you but know who it was.)
As far as "forever" goes, the program "convmv" does mass file renames for
encoding; you can use it to convert a given filesystem from whatever
encoding you've been using to UTF-8 (problem solved).
Yes, but this only works if you can mandate a no-encoding transparent view
of filenames! As soon as you start to automatically encode them, this sort
of transcoding is impossible.
I have no objection to making the things you propose options. What
I object to is making them mandatory, because this would make some
things impossible. (Strange things, but still.)
In fact, there's already a mechanism in the Linux kernel that might do
this job already: getfattr(1)/setfattr(1). If it were implemented this
way, I'd suggest that by default directories would "prevent bad filenames"
(e.g., control chars and leading "-"); you could then use "setfattr" on
directories to permit badness. New directories could then inherit the
state of their parent. I would make those "trusted extended attributes" -
you'd have to be CAP_SYS_ADMIN (typically superuser) to be able to create
It depends how harsh the limits are. I'd say that 'no control characters'
is certainly reasonable to have only the superuser lift. Perhaps a less
harsh constraint to impose is that regular users cannot set this attribute
on directories readable by 'other', and that chmodding a directory after
the fact strips this attribute off it. Now users cannot dump landmines in
that directory for users outside their group (root is assumed to know what
I'd say that setting this attribute flips a pathconf-viewable attribute as
well, so that other POSIX-compliant systems can adopt the same approach
and applications can portably query it without needing to implement/depend
on all of the ACL machinery.
to post comments)