Filesystems and case-insensitivity [LWN.net]

Filesystems and case-insensitivity

Posted Nov 28, 2018 17:54 UTC (Wed) by tnoo (subscriber, #20427)
Parent article: Filesystems and case-insensitivity

What a weird idea! How will a case-insensitive filesystem deal with filenames generated by a hash algorithm (as used e.g. by git and many other programs). If this is really needed, it should be a user-space layer, but file systems should handle unique byte sequences uniquely.

to post comments

Filesystems and case-insensitivity

Posted Nov 28, 2018 20:42 UTC (Wed) by saffroy (guest, #43999) [Link] (2 responses)

Consider hash values 0xabcdef and 0xABCDEF: well, they are the same value. :) Hash-based names in hex actually don't care about case.

Besides, see my other comment about when and why it is needed.

Filesystems and case-insensitivity

Posted Nov 29, 2018 17:05 UTC (Thu) by cesarb (subscriber, #6266) [Link] (1 responses)

Not if they're base64: abcdef and ABCDEF are different values in base64. Git can get away with base16 because it uses a short 160-bit hash, but other hash algorithms have much longer outputs (256 or even 512 bits).

Filesystems and case-insensitivity

Posted Nov 29, 2018 17:29 UTC (Thu) by bfields (subscriber, #19510) [Link]

What's an example of an application that does that?

Filesystems and case-insensitivity

Posted Nov 28, 2018 21:00 UTC (Wed) by smurf (subscriber, #17840) [Link] (3 responses)

No, you can't do it in userspace. The kernel does not have directory locking, which you'd need for an atomic "create a file while making sure that there is no other file with the same name (case-insensitive)" operation. Thus if you create "makefile" you need to scan the whole directory for any other matching filename, and that *still* doesn't stop anybody from introducing consistency errors (you can race to create "Makefile" and "MakeFile" at the same time).

Filesystems and case-insensitivity

Posted Nov 29, 2018 9:06 UTC (Thu) by dgm (subscriber, #49227) [Link] (2 responses)

> No, you can't do it in userspace. The kernel does not have directory locking, which you'd need for an atomic "create a file while making sure that there is no other file with the same name (case-insensitive)" operation.

Yes, you can. You only need to use the canonical representation of the file name (the case-folded one) and check that file name.

Filesystems and case-insensitivity

Posted Nov 29, 2018 12:20 UTC (Thu) by smurf (subscriber, #17840) [Link] (1 responses)

You presume that there *is* a canonical representation of a file name. For case-preserving file systems, there is no such thing.

Filesystems and case-insensitivity

Posted Nov 29, 2018 15:23 UTC (Thu) by dgm (subscriber, #49227) [Link]

Well, nobody was talking about case preservation, but now that you mention it, you have three options:
- use the filesystem as case preserving (and give up case insensitivity)
- use the filesystem as canse-insensitive (and forget about case preservation)
- use xattrs
And all that, without touching a single line of kernel code.

Filesystems and case-insensitivity

Posted Nov 29, 2018 1:02 UTC (Thu) by bfields (subscriber, #19510) [Link] (2 responses)

The filenames generated by git just use 0-9 and a-f, I don't see how case insensitivity would cause any problems.

Git, like most applications, already has to be prepared to work on case-insensitive filesystems.

Filesystems and case-insensitivity

Posted Nov 29, 2018 4:06 UTC (Thu) by smurf (subscriber, #17840) [Link] (1 responses)

The problem is tags, which once ended up as file and directory names.
Git has a "packed-refs" file these days, so the problem *should* be solved, but I haven't checked.

Filesystems and case-insensitivity

Posted Nov 29, 2018 15:31 UTC (Thu) by bfields (subscriber, #19510) [Link]

I haven't looked in a while either, but I think like packed objects it's just an optimization, and refs can also be created in unpacked formats.

File and directory names are a problem too, of course. If a directory you're storing in git includes both foo and FOO, then you'll have a problem when you try to check it out on a case-insensitive filesystem.

I don't think that's really fixable; some people actually do have such content which they need to track in git, others can't deal with it, it's up to the user to decide what they care about.

But git works on case-insensitive filesystems if the stuff you put into it does.