LWN: Comments on "nPth - The new GNU portable threads library"

Re: getcontext/setcontext alternatives

jwakely — Fri, 11 May 2012 23:53:59 +0000

Can std::current_exception and std::rethrow_exception be used for that exception propagation or are they insufficient?

Coroutines

hanwen — Fri, 11 May 2012 13:43:09 +0000

well, you need synchronization. Either channels or locks. A coroutine which does not synchronize at all may be optimized to not run at all, according to the spec.

nPth - The new GNU portable threads library

kugel — Wed, 09 May 2012 21:01:24 +0000

This uses the same mechanism (pthreads locked by a single global semaphore so only one is runnable at a time) we use in Rockbox to emulate non-preemptive environment on normal PCs (the simulators using SDL).

We actually got rid of this mechanism because is much more performance demanding than real cooperative usermode threads implemented with set/longjmp (pth works like that). Context switch overhead is huge compared to plain longjmp().

One advantage of the pthread-approach is you can temporarily enable preemption during blocking I/O calls so that not the whole program is blocked. Oh, and valgrind/gdb debugability (we actually keep the emulation around for this purpose). But that's about it. For the most part, real cooperative threads perform better. That's our experience.

Coroutines

kjp — Wed, 09 May 2012 20:42:05 +0000

woah. it saves full stack and local variables? The last time I used coroutines, all I had were a few macros that just saved the line number (using a switch statement to take you there on resume). I was always afraid that I had an uninitialized local variable problem hiding somewhere, no matter what the testing showed.... of course, it was REALLY fast :)

Coroutines

juliank — Wed, 09 May 2012 18:51:40 +0000

Goroutines are transparently moved across threads, and work correctly, as long as you do not access shared memory (which you should not do).

Coroutines

hanwen — Wed, 09 May 2012 16:39:31 +0000

According to the spec Go, but you still need locking between goroutines; an implementation which uses 1 thread per go-routine is conforming.

Coroutines

geertj — Wed, 09 May 2012 12:37:16 +0000

> Things are a bit different if you're talking about cooperative multi-tasking within a single application though: if the application is controlling all the co-routines, then those sort of bugs should be predictable and fixable.

Exactly - this is the key difference. And it's not like the different threads in a multi-threaded program don't need to cooperate. They can just as easy starve each other if e.g. they do not correctly cooperate in locking.

Re: getcontext/setcontext alternatives

geertj — Wed, 09 May 2012 11:20:12 +0000

> What's the alternative to getcontext() and friends? Going back to manually
> writing per arch "stack switching" code like cgreenlet/greenlet-asm.S?

One trick that is sometimes used is sigaltstack() + setjmp(). However i would argue this is actually less portable than writing assembly (the assembly you refer to above is not OS specific, it is only architecture specific and the same function it works on Linux, Mac OSX and Windows).

In the best case, we would get a working and not-deprecated makecontext() function from libc at some point. Also we would need to get an extended longjmp() that allows for code injection in the target co-routine (for propagating exceptions in C++).

Re: getcontext/setcontext alternatives

scottt — Wed, 09 May 2012 02:53:43 +0000

A few questions:

On which architecture did you run into the "thread-local-storage breaks when code trying to access a TLS object is run on a stack created with makecontext" problem?
What's the alternative to getcontext() and friends? Going back to manually writing per arch "stack switching" code like cgreenlet/greenlet-asm.S?

I'm aware that {make,get,set}context() are considered deprecated and were only recently implemented in the glibc ARM port but they're still in use in some reasonably popular apps like the VNC server implementation in qemu etc.

Coroutines

jamesh — Wed, 09 May 2012 02:12:39 +0000

Cooperative multi-tasking at the operating system level between independent applications has obvious problems with one application starving the others for CPU time if it doesn't yield.

Things are a bit different if you're talking about cooperative multi-tasking within a single application though: if the application is controlling all the co-routines, then those sort of bugs should be predictable and fixable. Co-routines can greatly simplify some programming tasks: letting you use the stack to manage state, while not having to deal with the complexities of concurrent execution in the same address space.

Why Stop At Coroutines?

ldo — Tue, 08 May 2012 23:02:13 +0000

The next logical step is to implement full continuations.

Coroutines

Cyberax — Tue, 08 May 2012 20:54:11 +0000

Not a lot of applications require massively parallel computations.

In niche areas Erlang (built on message passing) is quite popular.

Coroutines

wahern — Tue, 08 May 2012 20:31:44 +0000

The problem with using getcontext/setcontext is that those APIs are no longer maintained, and mixing them with pthreads causes trouble. For example, on some systems thread-local-storage breaks when code trying to access a TLS object is run on a stack created with makecontext. I ran into this issue just the other day when I linked in libffi, which pulled in pthreads. (Odd, I know; but even libffi is using pthreads these days; to make its closure framework thread-safe, I believe.)

I've been writing event-oriented and thread-oriented C apps for over 13 years. Using makecontext was useful then, but not so much today.

Coroutines

wahern — Tue, 08 May 2012 20:18:46 +0000

The design behind Plan 9 predates both Alef and the threading library. Lots of fruitful things were developed on Plan 9, and some folded back into the low-level systems. libtask was developed for Alef and only made available to run-of-the-mill C application later. According to Wikipedia, Plan 9 began in the mid '80s, and libtask was folded into the mix in 2000, two years before the project officially ended.

Limbo succeeded Alef, and Go seems to have succeeded Limbo. But I'll admit this is all just largely opinion.

But more to the point, Plan 9 never touted intraprocess message passing. If you read any of the papers, when they talked about parallelism they spoke about their rfork() system call (which may have inspired Linux' clone syscall; OpenBSD's rfork is clearly derivative of Plan 9). Alef, and later Limbo, was where all the experimentation into messaging passing went, and I think my comparison to Awk v. Unix is apt. To see where I'm coming from, just read their own description and emphasis: http://www.cs.bell-labs.com/sys/doc/9.html

Coroutines

valyala — Tue, 08 May 2012 20:11:28 +0000

Greenlet API looks too complex comparing to my own 'cross-platform' user-space threads' library - http://code.google.com/p/fiber-framework/ . It tries hiding cooperative multitasking complexities under easy-to-use API.

Coroutines

juliank — Tue, 08 May 2012 19:37:01 +0000

I'd say that Go is actually closer to Alef then Limbo. And it really is Plan 9 derived in some way. Not only the languages such as Alef, and later Limbo, but there's also a C library called libtask that implements the channel/thread mechanism (using user threads, Plan 9 doesn't seem to do kernel threads).

Coroutines

robert_s — Tue, 08 May 2012 19:19:50 +0000

You clearly didn't get enough of cooperative multitasking the first time around.

Coroutines

wahern — Tue, 08 May 2012 19:07:52 +0000

Go is based on Limbo, not Plan 9. It's like the difference between Awk and Unix. I mention it because Plan 9 was influential is so many other ways. The recent C11 standard has many features which originated (I believe) with Plan 9's C extensions. UTF-8 was first used in Plan 9. Modern /proc, I believe, is descended from Plan 9. And nothing about Plan 9 dictates or even strongly suggests a message-based multi-threaded language design.

Coroutines

endecotp — Tue, 08 May 2012 19:03:22 +0000

> Threads don't need locking if you only use message
> passing between threads

How do you implement your message passing? Most likely it needs some form of locking internally.

I would express it this way: don't try to implement your multi-threaded application using ad-hoc locking / synchronisation. Instead put all of the locking into some concurrency building blocks and restrict your inter-thread communication to only those building blocks. Message passing is one possible set of building blocks, but there are others.

Coroutines

juliank — Tue, 08 May 2012 18:50:24 +0000

Not yet, but work is happening, for example in Go (Go actually uses a hybrid co-routine/thread architecture, but this does not matter); and other stuff continuing those Plan 9 concepts.

Coroutines

fuhchee — Tue, 08 May 2012 17:59:06 +0000

"don't need locking if you only use message passing between"

Despite that advantage, pure message-passing-based concurrency hasn't taken the CS world by storm.

Coroutines

juliank — Tue, 08 May 2012 17:46:24 +0000

Threads don't need locking if you only use message passing between threads, and this should be the primary way to do such things.

Coroutines

endecotp — Tue, 08 May 2012 17:39:15 +0000

> I recently did a MIT licensed co-routine library for C/C++:

Boost has a proposal for a coroutine library by Oliver Kowalke which should be reviewed sometime in the next few months. See e.g.

http://ok73.ok.funpic.de/boost/libs/coroutine/doc/html/in...
http://thread.gmane.org/gmane.comp.lib.boost.devel/230507

It would be great if you could write a review when the time comes - getting the view of someone with relevant experience is obviously very useful.

Coroutines

geertj — Tue, 08 May 2012 17:26:27 +0000

Coroutines are great. The offer the speed of event based programming, combined with the ease of sequential programming with lockless, blocking I/O.

Threads on the other hand offer neither. Locking requirements makes them hard to get right, and they eat up a lot of resources making e.g. DoS attacks on thread-based web servers easy.

If you combine coroutines with multiple worker processes, it also allows you to use multiple cores.

I recently did a MIT licensed co-routine library for C/C++:

https://github.com/geertj/cgreenlet