| |||||||||||||
|
| |||||||||||||
timerfd() and system call reviewtimerfd() and system call reviewPosted Aug 15, 2007 19:01 UTC (Wed) by njs (subscriber, #40338)In reply to: timerfd() and system call review by clugstj Parent article: timerfd() and system call review
You, uh, want to have one thread per pending timer?
The real solution avoiding timerfd is to write a proper main loop like the ones in glib, Qt, Twisted Python, libevent, etc., that puts timers on a heap and uses the delay from timer at the head of the heap to set the timeout on one's blocking syscall (select, epoll, kqueue, whatever).
This is all a truly fantastic pain in the butt, though, esp. once you bring in other events like signals, process handling (waitpid), etc. Even worse, it's not composable -- if you have libraries that need to do IO, getting them integrated with each other and with your main loop is almost impossible. Again there are pure-userspace solutions possible in principle (e.g. abstracted wrappers over multiple event loops like liboop), but in practice it remains a huge issue. This is one of the reasons we still don't have a really decent async dns resolver library, for instance.
I don't know how much of a help these Linux-specific solutions will be in the long run, but being able to wrap all event sources into fds via timerfd and signalfd and so on, and combine multiple fds into a single fd (for purposes of event selection) via epoll, certainly has the *potential* to simplify all these messes significantly. Maybe we'll even figure out eventually whether this or kqueue is better.
(Log in to post comments)
timerfd() and system call review Posted Aug 15, 2007 22:34 UTC (Wed) by pphaneuf (subscriber, #23480) [Link] My thoughts exactly, except that I'd point out that a "timer signaled over a pipe library" would probably have a single thread, which would use a heap for the timers. Kind of silly, having a thread whose goal is to sleep all the time, basically, but hey, you've got to do what you've got to do...
timerfd() and system call review Posted Aug 16, 2007 10:18 UTC (Thu) by IkeTo (subscriber, #2122) [Link] I think the "one thread per timer" might be part of the problem, but not the whole problem. The bigger part is that to allow programming is such a style, all your events (i.e., wait for input) must signal that condition when the input comes. What it means is that every fd you wait for must be in its own thread (creating a lot of headaches to prevent race conditions and deadlocks), or else you must use pselect() or epoll() or whatever which subsumes the need for the timer thread anyway. On the other hand, I still don't quite know what is the best use case of timerfd(). It clearly is geared towards pselect() or epoll(). But that simply replaces the user-mode code to manage the timers by some kernel-mode code to manage the timers via file descriptors. I can see the kernel-mode approach more wasteful: it requires a timer structure and a file structure for each timer, a userland approach would require just an entry in a priority queue.
timerfd() and system call review Posted Aug 16, 2007 11:48 UTC (Thu) by nix (subscriber, #2304) [Link] It might be more wasteful, but it's a lot easier to write the userspacecode. Memory is cheap in the quantities we're talking about here (apps that use millions of timers simultaneously are going to be very rare).
And having *everything* be an fd would finally realize one of the goals of
timerfd() and system call review Posted Aug 16, 2007 12:43 UTC (Thu) by IkeTo (subscriber, #2122) [Link] > but it's a lot easier to write the userspace code
But when "userspace code" means library code, this is going to be hard to sell. After all the application developer see none of those. Can you imagine a version of, say, GLib implements its event loop using the timerfd() interface? Personally I can't.
timerfd() and system call review Posted Aug 30, 2007 23:19 UTC (Thu) by nix (subscriber, #2304) [Link] Actually I'd expect this to be mostly used by libraries. Currentlylibraries have the problem that signal disposition is process-global and can't be reset without interfering with other libraries, which is ameliorated by signalfd. Also, major libraries like glib *can* realistically include system-dependent code without being too annoying: it only has to go into glib, rather than into all its users (and glib already supports some Linux-specific interfaces anyway: indeed in a sense that's part of its raison d'etre).
timerfd() and system call review Posted Aug 28, 2007 21:58 UTC (Tue) by renox (guest, #23785) [Link] >>And having *everything* be an fd would finally realize one of the goals of the Unix world since its creation :) <<
Given that Plan9 has been much more thorough than Unix in the 'everything is a file' way, I wonder how they solved this issue?
timerfd() and system call review Posted Aug 30, 2007 23:23 UTC (Thu) by nix (subscriber, #2304) [Link] Plan 9 has `notes' instead of signals, but it looks like they too were`call this function automatically' things rather than being reified into fds. Surprising. (However, notes are plan9ish in another way: they're strings, not integers.)
timerfd() and system call review Posted Aug 16, 2007 13:20 UTC (Thu) by pphaneuf (subscriber, #23480) [Link] The use for timerfd is one of integration, I think. It's now possible to make an epoll fd, put all of your things in it as well as your timers, and just give back that single fd to an application, telling it to just call a specific function whenever it becomes readable. Note that in those integration situations (a classical example of which being an asynchronous DNS resolver), you don't get to specify the expiration of the select (or similar) call, or at least, not without complicating the interface. Of course, it can currently be faked with a thread, either entirely (use a single pipe, and put your whole select loop in a thread, managing the timers as well) or in part (if you have epoll, you can put the fds in it, and use a thread just for the timers). You avoid the race conditions and deadlocks by only doing the minimal amount in the thread, just enough to simulate epoll or timerfd. What's nice is that with each improvement (epoll and then timerfd), you can make that integration simpler and less complicated (running a thread in the background involves having to deal with untimely termination, making sure to block all signals and other such details). It's also possible to implement the same interface for the application whether you're faking it or not, so you can have some autoconf tests for epoll and timerfd, and then apply the right amount of emulation. As for the wastefulness, the timer structure is also more or less the same as that "entry" you'd put in a priority queue, and if you use pipes to fake things, you end up having not one, but two file structures, not to mention an almost entirely useless buffer.
|
|
||||||||||||