Filesystem-oriented flags: sad, messy and not going away

Posted Mar 17, 2020 6:00 UTC (Tue) by areilly (subscriber, #87829)
In reply to: Filesystem-oriented flags: sad, messy and not going away by Cyberax
Parent article: Filesystem-oriented flags: sad, messy and not going away

Yes, at the moment, but API versioning is the tool that you need in order to rationalize and tidy-up down the track, without breaking the code that will inevitably have been built against the current, messy set.

Filesystem-oriented flags: sad, messy and not going away

Posted Mar 17, 2020 6:18 UTC (Tue) by Cyberax (✭ supporter ✭, #52523) [Link] (11 responses)

This is doable in Linux.

You can create a set of new syscalls with sane flags: openat_rational, link_rational, open_I_really_mean_it's_not_broken_this_time and so on. You then expose these new syscalls with their wonderful flags through libc, libc can also provide their emulation for the older kernels that lack the new syscalls.

Then after 20 years or so you can remove the old flags from libc, so that new code will be able to use the new flags. Then after another 10 years or so, the old syscalls can be removed from the kernel.

Filesystem-oriented flags: sad, messy and not going away

Posted Mar 17, 2020 6:29 UTC (Tue) by areilly (subscriber, #87829) [Link] (10 responses)

Yes, but the Solaris and BSD versioned syscall approach allows you to remove the old syscalls and their messy flags immediately, reducing complexity, size and technical debt in the kernel. For the extra win, the piece of libc code that has to provide the old API in terms of the new becomes a nice modular piece of API history that documents the change.

Filesystem-oriented flags: sad, messy and not going away

Posted Mar 17, 2020 6:33 UTC (Tue) by Cyberax (✭ supporter ✭, #52523) [Link] (9 responses)

> Yes, but the Solaris and BSD versioned syscall approach allows you to remove the old syscalls and their messy flags immediately
How? There will still be software that uses old flags, for the foreseeable future. You'll have to provide their emulation _somewhere_.

Filesystem-oriented flags: sad, messy and not going away

Posted Mar 17, 2020 6:37 UTC (Tue) by josh (subscriber, #17465) [Link] (4 responses)

On BSD, the syscall interface is subject to change, while the libc interface is stable. So it'd be libc's job to implement the old interface in terms of the new one.

Filesystem-oriented flags: sad, messy and not going away

Posted Mar 17, 2020 9:51 UTC (Tue) by smurf (subscriber, #17840) [Link] (3 responses)

This means that BSD's libc is tied to the kernel. Linux' libc is not.

A corollary is that statically-linked programs may or may not continue to work when you update your kernel, a notion which Linus emphatically rejects.

Filesystem-oriented flags: sad, messy and not going away

Posted Mar 17, 2020 10:25 UTC (Tue) by josh (subscriber, #17465) [Link]

I agree, and I prefer the Linux approach.

That said, it'd be interesting if we had a slightly more extensible syscall layer that could tell when an argument was passed or not passed, which would allow existing existing syscalls without having to create new ones.

It's looking increasingly like io_uring might be that extensible syscall layer.

Filesystem-oriented flags: sad, messy and not going away

Posted Mar 19, 2020 20:55 UTC (Thu) by BenHutchings (subscriber, #37955) [Link] (1 responses)

Indeed, there is no such thing as "Linux's libc". There's glibc, bionic, uclibc, musl, klibc, and at least one language run-time (Go) that doesn't depend on a C library.

Filesystem-oriented flags: sad, messy and not going away

Posted Mar 23, 2020 15:27 UTC (Mon) by gray_-_wolf (subscriber, #131074) [Link]

> least one language run-time (Go) that doesn't depend on a C library.

sometimes... would be nice if it never did but that is sadly not the case :/

Filesystem-oriented flags: sad, messy and not going away

Posted Mar 17, 2020 6:47 UTC (Tue) by areilly (subscriber, #87829) [Link] (3 responses)

The solaris model is that the syscall API is a shared object (along with the rest of libc). User-space code doesn't get to make syscalls at all. The libc shared library can be versioned and indeed multiple, so you can (theoretically) keep older cruftier ones only as long as you have any executables that need them, on an install-by-install basis. I believe that OpenBSD is considering a similar scheme in order to have some sort of protection about where syscalls can come from, to prevent trampoline and gadget-style malware, perhaps.

The BSD versioned syscalls are in the kernel (so you can still have static executables), but they can be supplied by loadable kernel modules (as the linux and SCO syscalls are/were), which can eventually be deprecated or not loaded as suits the use-case, without getting (too much) in the way of the "fresh" syscall API.

Filesystem-oriented flags: sad, messy and not going away

Posted Mar 17, 2020 6:53 UTC (Tue) by josh (subscriber, #17465) [Link] (2 responses)

> The solaris model is that the syscall API is a shared object

How does Solaris provide that to userspace? Similar to the VDSO, or via a library provided on the filesystem that calls an unstable kernel interface?

Filesystem-oriented flags: sad, messy and not going away

Posted Mar 17, 2020 7:41 UTC (Tue) by areilly (subscriber, #87829) [Link] (1 responses)

I'm afraid that I don't know. I had always assumed that it was a specially-blessed user-space library provided by the filesystem. I'm sure there are readers who know more about Solaris than I do (it wouldn't be hard).

Filesystem-oriented flags: sad, messy and not going away

Posted Mar 18, 2020 21:37 UTC (Wed) by justincormack (subscriber, #70439) [Link]

Yes it was just a normal library - you could make syscalls elsewhere but they were neither documented or stable.

OpenBSD has been taking this model to a more modern design, where libc is blessed, and only it can make syscalls, by having a special attribute set. This is designed as a security measure, to stop arbitrary code using syscalls.