Filesystems and case-insensitivity

Posted Nov 28, 2018 14:07 UTC (Wed) by sorokin (guest, #88478)
Parent article: Filesystems and case-insensitivity

Do anyone have any measurements how much faster/slower case insensitive file systems are?

The problem I see is that filesystems serve two purposes:
1. They are a place where user store his files.
2. They are a place where programs store some internal data. Kind of (key, value) storage with hierarchical key.

For the first usage I see the merit of having case-insensitive filesystem. It depends on personal preference though.

For the second usage case-insensitiveness is a downside. When program lookup some its internal file/resource, case-insensitive comparison is both unnecessary and potentially incorrect. When I scan a directory with readdir/statat I don't want statat to be case-insensitive.

to post comments

Filesystems and case-insensitivity

Posted Nov 28, 2018 14:28 UTC (Wed) by bandrami (guest, #94229) [Link] (3 responses)

statat(2) doesn't interact with the filename at all, does it? You need to have already opened the file, but it doesn't know or care by what name you opened it, or even if that name still exists.

Filesystems and case-insensitivity

Posted Nov 28, 2018 14:43 UTC (Wed) by sorokin (guest, #88478) [Link] (1 responses)

> statat(2) doesn't interact with the filename at all, does it?

Unfortunately it does. See "pathname" parameter: int fstatat(int dirfd, const char *pathname, struct stat *statbuf, int flags);

It stats the file "pathname" relative to directory "dirfd". Normally when readdir returns DT_UNKNOWN one has to statat the filename relative to the directory to figure out the real d_type.

Filesystems and case-insensitivity

Posted Nov 28, 2018 15:42 UTC (Wed) by bandrami (guest, #94229) [Link]

Gah, thanks. Looks like I was thinking of BSD.

Filesystems and case-insensitivity

Posted Nov 28, 2018 21:05 UTC (Wed) by madscientist (subscriber, #16861) [Link]

Are you thinking of fstat(2)?

Filesystems and case-insensitivity

Posted Nov 28, 2018 16:09 UTC (Wed) by sorokin (guest, #88478) [Link] (1 responses)

BTW, even if user want to work with files in case insensitive manner, that does not mean that underlying filesystem must be case insensitive.

For example I can imagine that save file dialog can ask the following question: "You are trying to create file Foo.txt while foo.txt already exists. Do you want to create another file that differs only in
letter case."

Correspondingly open file dialog first look for exact match and if no file is found search for file case insensitively. I would like having convenience feature like this even now.

My point is that this should be done only in limited number of user facing dialogs. Doing this for most existing system calls would be inefficient and can be incorrect if filenames are used as opaque keys.

Filesystems and case-insensitivity

Posted Nov 28, 2018 20:34 UTC (Wed) by saffroy (guest, #43999) [Link]

Doing this for most existing system calls would be inefficient

Well, it really depends on the use case (pun intended). Once I added case-insensitivity support to a proprietary filesystem specifically to improve performance, with great success.

Consider a case-sensitive folder with 10.000 files (this is not rare at all), shared over Samba. Every time a Samba client requests creation of a new file, and because the client requires case insensitivity, Samba has to scan the entire folder to check if the new name collides with an existing name. Yes, that's for every new file.

If the filesystem is actually case-insensitive, Samba can skip these scans, which is a huge performance boost.

Filesystems and case-insensitivity

Posted Nov 28, 2018 16:43 UTC (Wed) by smcv (subscriber, #53363) [Link] (2 responses)

> They are a place where programs store some internal data. Kind of (key, value) storage with hierarchical key.

For the SteamOS use case, it's desirable that this lookup can be case-insensitive: game developers typically do most of their testing and development on Windows, where opening "level3.MAP" will successfully find a file named "Level3.map". If the obvious port of that game fails to work on Linux, it makes Linux gaming look bad, and makes porting games to Linux less appealing.

Emulations of case-insensitive enviroments, like Wine and Samba, also need to match the case-insensitive behaviour of the environment they're emulating.

Filesystems and case-insensitivity

Posted Nov 28, 2018 20:28 UTC (Wed) by HenrikH (subscriber, #31152) [Link]

And it can be pointed out that this have actually happened (the Linux version does not start due to case inconsistencies) quite a few times on Steam so it's not just a theoretical problem.

Filesystems and case-insensitivity

Posted Nov 28, 2018 21:12 UTC (Wed) by madscientist (subscriber, #16861) [Link]

It can easily go the other way too: for example I've seen cases where someone created Git branches on Linux which differed only by case. That worked fine for them, but people on MacOS or Windows had difficult-to-understand problems (because Git branches exist as directories/files).