| |||||||||||||
Thread-based or event-based?Thread-based or event-based?Posted Mar 1, 2007 15:16 UTC (Thu) by zooko (subscriber, #2589)Parent article: Thread-based or event-based?
A lot of people (including myself) have decided to write event-based code in user space.
One notable success in that area is BitTorrent. You might be interested in Mark Miller's dissertation, http://erights.org/talks/thesis/index.html, which gives a principled account of why event-based programming can be strictly safer and strictly more secure than other concurrency paradigms.
You might also be interested in the latest salvo in an academic contest to write the fastest web server. The current winner, according to this paper, is event-based http://bcr2.uwaterloo.ca/~brecht/papers/getpaper.php?file... also the Twisted Python project http://twistedmatrix.com.
If the kernel hackers could make user-space code like Twisted Python faster or simpler (at the level that it interfaces with the system, I mean), then I would be pleased. Likewise if the kernel could facilitate user space code such as Twisted to use asynchronous file I/O then that would be nice.
(Log in to post comments)
Thread-based or event-based? Posted Mar 1, 2007 18:22 UTC (Thu) by bronson (subscriber, #4806) [Link] That's a fascinating paper. Good find! Here's a link that worked for me: http://bcr2.uwaterloo.ca/~brecht/papers/eurosys-2007.pdf
Personally, I've found that an event-based model is great when you're doing trivial operations (say, a transparent proxy), but tends to fall apart when you're doing more complex stuff (say, a caching proxy with arbitrary, dynamic L7 rewrites). Get that state machine right on the first try! Even a small change tends to turn into a huge rippling modification if you didn't anticipate it. In my experience, avoiding state machines tends to be a win in the long run. But that's got a serious performance cost.
A hybrid scheme would be great. I have to agree with the naysayers though: in networking, everything blocks. A threadlet scheme will just wind up running as a thread-per-connection scheme. I hope the kernel guys can find a good way to work around this.
I'm happy that kevent finally got a bit of well-deserved recognition. An article comparing kqueue, epoll, kevent, and syslets/threadlets would be fascinating. I'm afraid that would be more like an academic paper, though... Anyone interested in doing a masters dissertation? :)
Thread-based or event-based? Posted Mar 1, 2007 18:30 UTC (Thu) by zooko (subscriber, #2589) [Link] I think you are right inasmuch as you are writing in C code. If you are writing in Python it is rather easier to write your event-based code. Certainly when we rewrote Mojo Nation (a open source peer-to-peer filesharing app) from one-thread-per-connection to event-based it was easier to understand and debug afterward than before. (By the way, that's the origin of BitTorrent's use of event-based -- Bram worked at Mojo Nation and re-used the event-based architecture that we developed there when he wrote BitTorrent.)
If you are writing in Twisted Python it is easier still, and if you are writing in E it is easier still.
C++ and Java fall right between C and Python in terms of the difficulty of doing event-based -- I have written multithreaded code and event-based code in each of them.
By the way, almost all GUI toolkits use the event-based paradigm, so many programmers are already familiar with the paradigm in at least one domain.
Regards,
Zooko
Thread-based or event-based? Posted Mar 1, 2007 20:38 UTC (Thu) by bronson (subscriber, #4806) [Link] What did you use to do event-based in C++? Did you like it?
It seems like many modern GUI toolkits are using closures and continuations to hide the event loop. So, while many programmers are familiar with it, I'm not sure they really like it. :)
Thread-based or event-based? Posted Mar 12, 2007 17:19 UTC (Mon) by zooko (subscriber, #2589) [Link] I did event-based code in C++ in a few ways -- non-blocking I/O with my own event framework atop it, various GUI toolkits, and in a sense also using DEC Message Queue middleware.
It was satisfying enough. Certainly the freedom from deadlock was worth it.
Thread-based or event-based? Posted Mar 2, 2007 17:34 UTC (Fri) by krasic (subscriber, #4782) [Link] I know a bit about twisted, but I don't know about "E". Could someone give a reference or pointer?
Thread-based or event-based? Posted Mar 2, 2007 18:06 UTC (Fri) by markh (subscriber, #33984) [Link] > I know a bit about twisted, but I don't know about "E". Could someone> give a reference or pointer?
Thread-based or event-based? Posted Mar 2, 2007 18:17 UTC (Fri) by shane (subscriber, #3335) [Link] If you are writing in Python it is rather easier to write your event-based code.Of course, it's also easier to write your threaded code.:)
Thread-based or event-based? Posted Mar 4, 2007 18:55 UTC (Sun) by rwmj (subscriber, #5474) [Link] Personally, I've found that an event-based model is great when you're doing trivial operations (say, a transparent proxy), but tends to fall apart when you're doing more complex stuff (say, a caching proxy with arbitrary, dynamic L7 rewrites). Get that state machine right on the first try! Even a small change tends to turn into a huge rippling modification if you didn't anticipate it. It's actually possible to implement threads on top of events. See for example my implementation in pthrlib. At the syscall level, this is using poll(2) to poll for events. However the programming model is cooperative threads. It's also possible to turn threads back into events, but (I'm guessing) this won't be very efficient because threads are certainly heavier than events. So this is one contribution to the argument that the kernel should just deliver up events, and allow the userspace to deal with those directly, or turn them into threads a la pthrlib. Rich.
|
|||||||||||||