Race-free process creation in the GNU C Library
Race-free process creation in the GNU C Library
Posted Sep 5, 2023 16:01 UTC (Tue) by Cyberax (✭ supporter ✭, #52523)In reply to: Race-free process creation in the GNU C Library by wtarreau
Parent article: Race-free process creation in the GNU C Library
Can you find a single library that starts subprocesses, that has hooks for these kinds of locks? That's what I mean by "not composable".
Also, having to do such lock dances is an indication of a bad API in itself.
Posted Sep 6, 2023 14:02 UTC (Wed)
by wtarreau (subscriber, #51152)
[Link] (8 responses)
I don't know, since I don't know what such libs currently do. But it would seem like the correct thing to do if they claim to be thread-compatible.
> Also, having to do such lock dances is an indication of a bad API in itself.
If necessary it could be wrapped into a simpler API. But the locks are precisely due to a race which is inherent to process reaping/signaling that can be happening in parallel and that one needs to serialize. I don't see why one must suddenly start to make an exception for this specific case and say "let's pretend there is no race here so that we can save one lock" nor "let's assume programmers creating threads don't understand the limits of threads". I would, however, clearly welcome an in-libc pair of wrappers that just adds these locks around wait() and kill() such as locked_wait() and locked_kill() to be more friendly to the user and to lib developers. But my feeling is that if it's just for this, it's becoming overkill, and the fact that it started a discussion seems to indicate others have the same feeling.
Posted Sep 6, 2023 15:22 UTC (Wed)
by Cyberax (✭ supporter ✭, #52523)
[Link] (7 responses)
No. You need to wrap _user_ code in locks. It's not just functions themselves. I.e.:
You have to write ALL process-related code in this manner:
1. get_lock
This can't be wrapped into simple locked functions, unless you want to have a closure-based API. And even then you'll have all the locking-related issues, like deadlocks.
In short, the classic process API is inherently broken in the presence of threads. It can't be sanely fixed.
Posted Sep 6, 2023 20:53 UTC (Wed)
by wtarreau (subscriber, #51152)
[Link] (6 responses)
Posted Sep 7, 2023 0:13 UTC (Thu)
by Cyberax (✭ supporter ✭, #52523)
[Link] (5 responses)
I consider having to do locks in sometimes inconvenient places to block "spooky action at a distance" the very definition of brokenness.
Also, there's still a case where you might need to do operations (e.g. send signals) to processes that are not your children.
Posted Sep 7, 2023 4:00 UTC (Thu)
by wtarreau (subscriber, #51152)
[Link] (4 responses)
If so, absolutely everything involving threads or communication with other processes is broken. I'm sorry but I disagree with this definition.
> Also, there's still a case where you might need to do operations (e.g. send signals) to processes that are not your children.
Yes, and this has always shown a moderate reliability only. That's the classical "ps auxw" then "kill $pid". I don't see what in the proposed API could improve this situation at all.
Posted Sep 7, 2023 4:05 UTC (Thu)
by Cyberax (✭ supporter ✭, #52523)
[Link] (3 responses)
Imagine that memory allocations worked the same way. Instead of getting a pointer, you get a "zone ID" with the same semantics as PIDs.
Sorry, but plenty of interfaces are well-designed and work just fine with threads. In libc: memory allocations, file operations, IPC primitives, etc.
To be fair, libc also has a plenty of other broken interfaces: the notion of the current directory, non-reentrable functions, the whole mess with locales and timezones.
> I don't see what in the proposed API could improve this situation at all.
Uhm... You get a pidfd and you can use it to make sure that the PID won't be reused while at least one pidfd descriptor is open. This makes it possible to do race-free process manipulation.
Posted Sep 7, 2023 18:26 UTC (Thu)
by wtarreau (subscriber, #51152)
[Link] (2 responses)
Actually you gave a pretty good example, because memory allocations *do* work the same way. If one thread tries to access a memory location while another one is freeing it and without coordinating together, you'll pretty quickly see either a use-after-free bug or a basic segfault.
> You get a pidfd and you can use it to make sure that the PID won't be reused while at least one pidfd descriptor is open. This makes it possible to do race-free process manipulation.
I'm just seeing it as convenience at the expense of extra FDs, which may in some cases result in new classes of bugs such as leaks if some FDs are passed by accident or just lost without being closed or stuck into a UNIX socket but closed so that nobody sees it, and even possibly vulnerabilities later if accessing such an FD is possible and is sufficient to send a signal over it despite the processes not being supposed to be able to interact.
Don't get me wrong, I'm not saying it's bad, we all love when some APIs are made easier to use or open new possibilities. It's just that I don't feel like this was that difficult to use correctly and that the small extra efforts probably did not warrant the possible classes of issues that will inevitably come with it. Time will tell.
Posted Sep 7, 2023 18:39 UTC (Thu)
by bluca (subscriber, #118303)
[Link]
Then you still haven't quite grasped what the actual problems being solved here are, and it might be time to go look at the sources linked in the article before further commenting, especially the cover letters and the linked bugzillas
Posted Sep 7, 2023 19:31 UTC (Thu)
by Cyberax (✭ supporter ✭, #52523)
[Link]
Now imagine that allocation can be freed at any moment.
> I'm just seeing it as convenience at the expense of extra FDs,
FDs are not a scarce resource.
> which may in some cases result in new classes of bugs such as leaks if some FDs are passed by accident
How is that different from any other FDs?
> or just lost without being closed
Don't lose resources.
> or stuck into a UNIX socket but closed so that nobody sees it
And?
Race-free process creation in the GNU C Library
Race-free process creation in the GNU C Library
2. pid = create_process()
3. verify_pid_is_correct(pid)
4. kill(pid, 9)
5. release_lock()
Race-free process creation in the GNU C Library
Race-free process creation in the GNU C Library
Race-free process creation in the GNU C Library
Race-free process creation in the GNU C Library
Race-free process creation in the GNU C Library
Race-free process creation in the GNU C Library
Race-free process creation in the GNU C Library