| |||||||||||||
CoroutinesCoroutinesPosted May 8, 2012 17:26 UTC (Tue) by geertj (subscriber, #4116)Parent article: nPth - The new GNU portable threads library
Coroutines are great. The offer the speed of event based programming, combined with the ease of sequential programming with lockless, blocking I/O.
Threads on the other hand offer neither. Locking requirements makes them hard to get right, and they eat up a lot of resources making e.g. DoS attacks on thread-based web servers easy.
If you combine coroutines with multiple worker processes, it also allows you to use multiple cores.
I recently did a MIT licensed co-routine library for C/C++:
(Log in to post comments)
Coroutines Posted May 8, 2012 17:39 UTC (Tue) by endecotp (guest, #36428) [Link]
> I recently did a MIT licensed co-routine library for C/C++:
Boost has a proposal for a coroutine library by Oliver Kowalke which should be reviewed sometime in the next few months. See e.g.
http://ok73.ok.funpic.de/boost/libs/coroutine/doc/html/in...
It would be great if you could write a review when the time comes - getting the view of someone with relevant experience is obviously very useful.
Coroutines Posted May 9, 2012 20:42 UTC (Wed) by kjp (subscriber, #39639) [Link]
woah. it saves full stack and local variables? The last time I used coroutines, all I had were a few macros that just saved the line number (using a switch statement to take you there on resume). I was always afraid that I had an uninitialized local variable problem hiding somewhere, no matter what the testing showed.... of course, it was REALLY fast :)
Coroutines Posted May 8, 2012 17:46 UTC (Tue) by juliank (subscriber, #45896) [Link]
Threads don't need locking if you only use message passing between threads, and this should be the primary way to do such things.
Coroutines Posted May 8, 2012 17:59 UTC (Tue) by fuhchee (subscriber, #40059) [Link]
"don't need locking if you only use message passing between"
Despite that advantage, pure message-passing-based concurrency hasn't taken the CS world by storm.
Coroutines Posted May 8, 2012 18:50 UTC (Tue) by juliank (subscriber, #45896) [Link]
Not yet, but work is happening, for example in Go (Go actually uses a hybrid co-routine/thread architecture, but this does not matter); and other stuff continuing those Plan 9 concepts.
Coroutines Posted May 8, 2012 19:07 UTC (Tue) by wahern (subscriber, #37304) [Link]
Go is based on Limbo, not Plan 9. It's like the difference between Awk and Unix. I mention it because Plan 9 was influential is so many other ways. The recent C11 standard has many features which originated (I believe) with Plan 9's C extensions. UTF-8 was first used in Plan 9. Modern /proc, I believe, is descended from Plan 9. And nothing about Plan 9 dictates or even strongly suggests a message-based multi-threaded language design.
Coroutines Posted May 8, 2012 19:37 UTC (Tue) by juliank (subscriber, #45896) [Link]
I'd say that Go is actually closer to Alef then Limbo. And it really is Plan 9 derived in some way. Not only the languages such as Alef, and later Limbo, but there's also a C library called libtask that implements the channel/thread mechanism (using user threads, Plan 9 doesn't seem to do kernel threads).
Coroutines Posted May 8, 2012 20:18 UTC (Tue) by wahern (subscriber, #37304) [Link]
The design behind Plan 9 predates both Alef and the threading library. Lots of fruitful things were developed on Plan 9, and some folded back into the low-level systems. libtask was developed for Alef and only made available to run-of-the-mill C application later. According to Wikipedia, Plan 9 began in the mid '80s, and libtask was folded into the mix in 2000, two years before the project officially ended.
Limbo succeeded Alef, and Go seems to have succeeded Limbo. But I'll admit this is all just largely opinion.
But more to the point, Plan 9 never touted intraprocess message passing. If you read any of the papers, when they talked about parallelism they spoke about their rfork() system call (which may have inspired Linux' clone syscall; OpenBSD's rfork is clearly derivative of Plan 9). Alef, and later Limbo, was where all the experimentation into messaging passing went, and I think my comparison to Awk v. Unix is apt. To see where I'm coming from, just read their own description and emphasis: http://www.cs.bell-labs.com/sys/doc/9.html
Coroutines Posted May 9, 2012 16:39 UTC (Wed) by hanwen (subscriber, #4329) [Link]
According to the spec Go, but you still need locking between goroutines; an implementation which uses 1 thread per go-routine is conforming.
Coroutines Posted May 9, 2012 18:51 UTC (Wed) by juliank (subscriber, #45896) [Link]
Goroutines are transparently moved across threads, and work correctly, as long as you do not access shared memory (which you should not do).
Coroutines Posted May 11, 2012 13:43 UTC (Fri) by hanwen (subscriber, #4329) [Link]
well, you need synchronization. Either channels or locks. A coroutine which does not synchronize at all may be optimized to not run at all, according to the spec.
Coroutines Posted May 8, 2012 20:54 UTC (Tue) by Cyberax (✭ supporter ✭, #52523) [Link]
Not a lot of applications require massively parallel computations.
In niche areas Erlang (built on message passing) is quite popular.
Coroutines Posted May 8, 2012 19:03 UTC (Tue) by endecotp (guest, #36428) [Link]
> Threads don't need locking if you only use message
> passing between threads
How do you implement your message passing? Most likely it needs some form of locking internally.
I would express it this way: don't try to implement your multi-threaded application using ad-hoc locking / synchronisation. Instead put all of the locking into some concurrency building blocks and restrict your inter-thread communication to only those building blocks. Message passing is one possible set of building blocks, but there are others.
Coroutines Posted May 8, 2012 19:19 UTC (Tue) by robert_s (subscriber, #42402) [Link]
You clearly didn't get enough of cooperative multitasking the first time around.
Coroutines Posted May 9, 2012 2:12 UTC (Wed) by jamesh (guest, #1159) [Link]
Cooperative multi-tasking at the operating system level between independent applications has obvious problems with one application starving the others for CPU time if it doesn't yield.
Things are a bit different if you're talking about cooperative multi-tasking within a single application though: if the application is controlling all the co-routines, then those sort of bugs should be predictable and fixable. Co-routines can greatly simplify some programming tasks: letting you use the stack to manage state, while not having to deal with the complexities of concurrent execution in the same address space.
Coroutines Posted May 9, 2012 12:37 UTC (Wed) by geertj (subscriber, #4116) [Link]
> Things are a bit different if you're talking about cooperative multi-tasking within a single application though: if the application is controlling all the co-routines, then those sort of bugs should be predictable and fixable.
Exactly - this is the key difference. And it's not like the different threads in a multi-threaded program don't need to cooperate. They can just as easy starve each other if e.g. they do not correctly cooperate in locking.
Coroutines Posted May 8, 2012 20:11 UTC (Tue) by valyala (guest, #41196) [Link]
Greenlet API looks too complex comparing to my own 'cross-platform' user-space threads' library - http://code.google.com/p/fiber-framework/ . It tries hiding cooperative multitasking complexities under easy-to-use API.
Coroutines Posted May 8, 2012 20:31 UTC (Tue) by wahern (subscriber, #37304) [Link]
The problem with using getcontext/setcontext is that those APIs are no longer maintained, and mixing them with pthreads causes trouble. For example, on some systems thread-local-storage breaks when code trying to access a TLS object is run on a stack created with makecontext. I ran into this issue just the other day when I linked in libffi, which pulled in pthreads. (Odd, I know; but even libffi is using pthreads these days; to make its closure framework thread-safe, I believe.)
I've been writing event-oriented and thread-oriented C apps for over 13 years. Using makecontext was useful then, but not so much today.
Re: getcontext/setcontext alternatives Posted May 9, 2012 2:53 UTC (Wed) by scottt (subscriber, #5028) [Link] A few questions:
Re: getcontext/setcontext alternatives Posted May 9, 2012 11:20 UTC (Wed) by geertj (subscriber, #4116) [Link]
> What's the alternative to getcontext() and friends? Going back to manually
> writing per arch "stack switching" code like cgreenlet/greenlet-asm.S?
One trick that is sometimes used is sigaltstack() + setjmp(). However i would argue this is actually less portable than writing assembly (the assembly you refer to above is not OS specific, it is only architecture specific and the same function it works on Linux, Mac OSX and Windows).
In the best case, we would get a working and not-deprecated makecontext() function from libc at some point. Also we would need to get an extended longjmp() that allows for code injection in the target co-routine (for propagating exceptions in C++).
Re: getcontext/setcontext alternatives Posted May 11, 2012 23:53 UTC (Fri) by jwakely (subscriber, #60262) [Link]
Can std::current_exception and std::rethrow_exception be used for that exception propagation or are they insufficient?
|
|||||||||||||