User-space interrupts
User-space interrupts
Posted Sep 30, 2021 20:11 UTC (Thu) by NYKevin (subscriber, #129325)Parent article: User-space interrupts
IMHO it depends on multiple factors:
* EINTR is annoying to handle from userspace, so much so that some languages (Python at least) just transparently handle it for you. But you already have to handle it anyway, unless you're exclusively doing SIG_IGN/SIG_DFL for all signals.
* The "wait for something to happen" syscalls (nanosleep, epoll_wait, etc.) should probably be interrupted, regardless of what is decided for other syscalls.
* Since this is supposed to be a "low latency" mechanism, for when signals aren't fast enough, it would be very odd if you had to wait around for the kernel before you could get the interrupt, given that signals actually do interrupt the kernel...
Posted Oct 1, 2021 12:10 UTC (Fri)
by droundy (subscriber, #4559)
[Link] (2 responses)
Posted Oct 1, 2021 13:46 UTC (Fri)
by mathstuf (subscriber, #69389)
[Link] (1 responses)
Posted Oct 1, 2021 16:30 UTC (Fri)
by nybble41 (subscriber, #55106)
[Link]
Posted Oct 3, 2021 2:25 UTC (Sun)
by kepstin (subscriber, #72391)
[Link] (1 responses)
Or... Is this primarily for real time processes maybe? If the processes are carefully scheduled and known to be running concurrently on different cpus, I guess a direct userspace interrupt would be a win for latency.
Posted Oct 3, 2021 18:35 UTC (Sun)
by Bigos (subscriber, #96807)
[Link]
User mode scheduling (aka "green threads") was given as an example use case. If I understand correctly, one thread can preempt the other using a userspace interrupt. The interrupt handler can then modify the state so that on return something else is called, similar to how kernel scheduler works, but without any context switches. However, green threads are usually used when scheduling small short-living (or often-waiting) tasks, so cooperative preemption is enough. And when forceful preemption is necessary, it happens seldom enough for it not to be a bottleneck.
Jens Axboe mentioned io_uring cq notification [1], though that is about kernel -> userspace which has not been implemented yet.
There might be a better use case example that I am not thinking about. In fact, this has been stated as one of the issues on the mailing list [2].
At first, I wanted to point out that userspace RCU (URCU) could be a possible use case as well, but that was already resolved by membarrier() syscall years ago [3]. However, userspace interrupts might improve the performance of this even without broadcast support.
[1] https://lwn.net/ml/linux-kernel/ecf3cf2e-685d-afb9-1a5d-1...
Posted Oct 12, 2021 21:57 UTC (Tue)
by anton (subscriber, #25547)
[Link] (10 responses)
Based on this understanding, I would expect the system-call wrapper to deal with EINTR, i.e., higher-level user programs should never see it. But strangely, AFAIK the system-call wrappers just deliver EINTR to the higher-level user code. Why? Is my understanding of EINTR wrong?
Posted Oct 12, 2021 22:54 UTC (Tue)
by mpr22 (subscriber, #60784)
[Link] (1 responses)
Imagine further that you send that daemon SIGHUP, telling it to reload its configuration files.
Do you really want it to wait until it receives its next packet before reloading its configuration files? After all, your config change might mean it needs to close one or more of the sockets it's currently waiting on and/or open new sockets.
I don't.
I'd much rather that its signal handler for SIGHUP just sets a flag saying "config must be reloaded", and that the system call wrapper then returns -1 and sets errno to EINTR, so that my daemon can check its config-reload flag and go "oh hey I've been reconfigured".
Posted Oct 13, 2021 5:51 UTC (Wed)
by anton (subscriber, #25547)
[Link]
Posted Oct 12, 2021 23:45 UTC (Tue)
by neilbrown (subscriber, #359)
[Link] (7 responses)
In many cases a syscall that returns EINTR should *not* be restarted. A blocking read is an obvious case. The application should be given the opportunity to choose whether to restart.
Posted Oct 13, 2021 6:37 UTC (Wed)
by anton (subscriber, #25547)
[Link] (6 responses)
Why do you think that a blocking read() should not just continue (by restarting the system call) if the signal handler just returns after doing whatever it was doing? Why should every user of read() have to deal with EINTR?
Posted Oct 13, 2021 13:19 UTC (Wed)
by madscientist (subscriber, #16861)
[Link]
Posted Oct 13, 2021 22:47 UTC (Wed)
by neilbrown (subscriber, #359)
[Link] (4 responses)
It's not "every user of read()", but only every user of read() reading from a file descriptor on which reads can block.
You certainly *could* have a platform where read() always retries EINTR, and signal handlers have to use longjmp if they want to abort a system call. But I don't think that would be *clearly* better than the current situation. Maybe marginally better - I don't know.
Posted Oct 13, 2021 23:23 UTC (Wed)
by mpr22 (subscriber, #60784)
[Link] (1 responses)
When an idea involves normalizing the use of longjmp() in code written by mere mortals, that creates in my mind the (theoretically, but probably not practically, rebuttable) presumption that the idea is absolutely terrible.
Posted Oct 14, 2021 17:18 UTC (Thu)
by anton (subscriber, #25547)
[Link]
Posted Oct 14, 2021 17:13 UTC (Thu)
by anton (subscriber, #25547)
[Link] (1 responses)
In the present case, requiring the general routine to know whether it is used on a file that can result in EINTR, and how to behave in that case (which is very likely application-dependent) breaks modularity.
The application and its signal handler know how to deal with the situation, and the longjmp() approach is a good one in that sense. Of course, leaving an asynchronous signal with longjmp() has its dangers, but that's still the way we chose in Gforth (where asynchronous signals are rare).
Posted Oct 14, 2021 19:51 UTC (Thu)
by nybble41 (subscriber, #55106)
[Link]
Users generally expect to be able to use sockets and pipes in place of regular files, e.g. using process substitution in Bash or named FIFOs or Unix-domain sockets in the filesystem, or arbitrary paths under /proc/$PID/fd/. Unless there is a good reason to require capabilities specific to regular files, for example lseek() or mmap()—or the application creates the file itself with O_EXCL—then applications ought to expect that read() and write() may process less data than requested even if the normal case involves regular files.
As for the longjmp() approach, that only works because the kernel backs out of the blocking call before invoking the signal handler. (A longjmp() call from a signal handler can't perform a non-local return out of arbitrary *kernel* stack frames.) At that point it's mostly a matter of policy whether the kernel restarts the system call after the handler returns or just returns EINTR to the caller—either always restarting or always returning EINTR would not simplify the kernel signficantly—and in general matters of policy are best left to application or library code rather than the kernel. Wrapping every non-interruptable read() in a loop to restart it until you get all the data you wanted is not substantially more code, or more *complex* code, than wrapping every read() which you might want to interrupt in a call to setjmp() and communicating that fact to the signal handler so it can decide whether to call longjmp().
POSIX also has these caveats regarding longjmp() from a signal handler:
> It is recommended that applications do not call longjmp() or siglongjmp() from signal handlers. To avoid undefined behavior when calling these functions from a signal handler, the application needs to ensure one of the following two things: … After the call to longjmp() or siglongjmp() the process only calls async-signal-safe functions and does not return from the initial call to main(). … Any signal whose handler calls longjmp() or siglongjmp() is blocked during *every* call to a non-async-signal-safe function, and no such calls are made after returning from the initial call to main().
It would be difficult to guarantee either of these restrictions are met in a complex application with many library dependencies. For example, if you return from a signal handler with longjmp() and then call printf() without masking every signal whose handler could call longjmp() then you've already broken both of those rules and invoked undefined behavior.
Posted Sep 18, 2024 6:23 UTC (Wed)
by renox (guest, #23785)
[Link] (2 responses)
Posted Sep 18, 2024 6:43 UTC (Wed)
by Cyberax (✭ supporter ✭, #52523)
[Link]
Posted Sep 18, 2024 7:09 UTC (Wed)
by intelfx (subscriber, #130118)
[Link]
User-space interrupts
User-space interrupts
User-space interrupts
User-space interrupts
User-space interrupts
[2] https://lwn.net/ml/linux-kernel/456bf9cf-87b8-4c3d-ac0c-7...
[3] https://lwn.net/Articles/369567/
User-space interrupts
EINTR is annoying to handle from userspace, so much so that some languages (Python at least) just transparently handle it for you.
My understanding of EINTR (especially after reading Gabriel's Worse-is-better paper) is that it's too complicated to deal with the situation (dealing with a signal) in the kernel, so the kernel returns to user space with EINTR, the signal can be delivered, and afterwards user space sees the EINTR and restarts the system call.
User-space interrupts
In this scenario the signal will be handled right after the system call (not the wrapper) returns, i.e., immediately. The default for SIGHUP is to terminate the process, so the parent process can reread the configuration file and restart the process (just one example on how to deal with that).
User-space interrupts
User-space interrupts
It *might* be possible for a platform (such as python) to require that syscalls which cannot be restarted don't get used. e.g. reads must be non-blocking. But that is a higher-level design choice than the libc syscall wrapper.
Looking at the wrapper of read() (one of the system calls that , it does not handle ERESTART, and ERESTART is not documented as an error returned from read(), while EINTR is.
User-space interrupts
User-space interrupts
User-space interrupts
That does NOT include regular files - mainly char devices and sockets.
When you read from something that can block, you usually want more than you can be sure of getting in a single read. You'll usually need to be prepared to read some more anyway (not always, but often).
So you need to be prepared for a short read, and handling EINTR as well is not a whole lot more effort.
User-space interrupts
I found writing a signal handler more of a challenge than performing the longjmp() (and I don't remember ever having a bug due to the longjmp()). The problem is that asynchronous signals can be invoked anywhere, including in the middle of updating some data structure, so you have a chance of corrupting the data structure if you longjmp() out of a signal handler for an asynchronous signal.
User-space interrupts
Unix has the very good idea that everything is a file. One of the benefits is that you can write some general routine on top of the system calls, and it is useful for all kinds of things. This admittedly does not work all the time, but we should strive for it.
User-space interrupts
User-space interrupts
User-space interrupts
User-space interrupts
User-space interrupts