Working with UTF-8 in the kernel [LWN.net]

Working with UTF-8 in the kernel

Posted Mar 30, 2019 18:19 UTC (Sat) by nybble41 (subscriber, #55106)
In reply to: Working with UTF-8 in the kernel by foom
Parent article: Working with UTF-8 in the kernel

The fact that case folding is broken everywhere else it's been implemented offers a good argument against implementing it in Linux.

to post comments

Working with UTF-8 in the kernel

Posted Mar 30, 2019 21:41 UTC (Sat) by mirabilos (subscriber, #84359) [Link] (3 responses)

case-insensitive filename comparison belongs into userspace, because there, if you setlocale(3) correctly, things like toupper(3), tolower(3) and strcoll(3) work properly even for Turkish.

So, the kernel should have nothing to do with this *at all*.

Working with UTF-8 in the kernel

Posted Apr 1, 2019 9:45 UTC (Mon) by nim-nim (subscriber, #34454) [Link] (2 responses)

The kernel needs to provide the encoding and normalisation userspace can work from. You can’t push those to userspace, because userspace then ends up guessing (badly) the filesystem encoding.

Working with UTF-8 in the kernel

Posted Apr 5, 2019 1:24 UTC (Fri) by xtifr (guest, #143) [Link] (1 responses)

If the case-insensitive FS is a user-space overlay on an existing filesystem, then you don't *need* to guess anything. It will do case-folding based on the locale of the user who mounted the overlay.

This means *all* the overheads will truly need to be present only for those who actively *use* the system.

I honestly don't know how this is all going forward without *at least* a user-space proof-of-concept system.

Working with UTF-8 in the kernel

Posted Apr 6, 2019 21:56 UTC (Sat) by foom (subscriber, #14868) [Link]

> It will do case-folding based on the locale of the user who mounted the overlay.

But that behavior would be pretty awful -- which files you can access depending upon your current locale? There's a reason filesystems (including this ext4 proposal) store the mapping used when creating the filesystem...

> without *at least* a user-space proof-of-concept system.

Two have been mentioned already. Android has an overlay filesystem for local access, and samba implements it when exporting the filesystem over the network.