De Icaza: Callbacks as our Generations' Go To Statement

I would have thought that the channels in Go would more properly be classified as coroutine support.

I think it's time to move to languages that transparently support thousands of lightweight threads with inexpensive message passing and good latency, rather than messing around with callbacks or lots of extra keywords.

But I *had* to echo your sentiment - I worked on a large Twisted program, and, yes, it was also a large twisted program. Especially when it comes to exceptions. The best part was there were parts where exceptions got swallowed, or dumped to stderr in a daemon that was only set up to log stdout.. so, yeah, you basically had no idea what was going on...

It could be possible to program neatly in Twisted, but the coroutine stuff I've seen is always much much cleaner (no experience in it though).

I have to say, though - in more cases than most people think, threads really aren't as evil as people say. Especially with some of the nicer higher-level libs that provide the thread pool and work queue components built-in. I think threads are confusing, but get beat on a little too much...

An error is an event. I.e. in the event based, callback driven programming exceptions have to handled in specific callbacks. Say for instance a connect to a TCP port fails with "connection refused." If connect() is blocking throw an exception. But with non-blocking connect() you will get the result of the connect() operation later maybe in the same callback handling a successfull connect(). So you are forced have to have error handling right there and decide what to do. You can't wrap the whole long sequence into one big try catch and handle it all in one big error handler.

To me it sounds like event-callback driven coding actually forces the coder to handle all failures intelligently. Maybe a showMessage("Connection failed: <reason>") or simply try the connect() again, or whatever is a proper failure mode.

This only strengthens by faith in single threaded, event-callback driven programs not using exceptions. And yes, as mentioned in comments above proper GUI toolkits also supports that, so it is not only for server-programming.
At me this works very well for embedded programs written in C++. I compare with Java with a lot of threads. The C++ programs are much more stable and have far fewer flaws with error handling in for instance handling of TCP connections.

When C returns the actual value is noted in the Task object B returned for later use (e.g. to update a text field in the UI).

Also note that async/await propagates up in the call hierarchy, i.e. A also called B using the await keyword and so on.

This is how I understand it and it sounds pretty nifty to me.

About as close as you can get to it in C is with setjmp()/longjmp(). You would need language support for delimited continuations (such as provided by guile's scheme implementation) to do a more wholesome emulation of it.

You can sort-of emulate C# async in effect, by having a thread which composes tasks using std::async(), blocks on the futures it obtains, and then when finished posts its result to the program's event loop (in glib, say, using g_idle_add()). With this scheme you cannot eliminate at least one blocking thread (and you may end up with more), but at least the blocking thread will be scheduled out by the OS's scheduler while it is blocking.

A more refined development of this for C/C++ is to run a thread pool set with a maximum number of threads approximating to the number of cores available, where tasks assigned to the pool increment the maximum thread count when about to block so as to keep cores active (decrementing again when coming out of the block) and deliver the composed result to the program's event loop. It avoids "callback hell" (you only have one callback), but this still puts the onus on the programmer to manage the thread count correctly. C# async hides all that.

The answer to the problem is coroutines, which allows one to flip the producer/consumer relationship between two blocks of code, and to do so in both directions. Coroutines allow you to replace a callback with a return value, while still allowing both blocks of code to push parameters using a function invocation and retrieve results with a return value, which is the most natural form.

In languages like Lua where coroutines are implemented as separate virtual stacks, allowing yielding across intermediate function invocations, coroutines can be used to hack together a kind of cooperative threading. And so when people hear coroutines they immediately jump to the idea of threading. But that entirely misses the fundamental benefit of coroutines, which is the ability to invert consumer/producer in an ad hoc manner at the language level, without resorting to message passing.

Coroutines are the final answer, and it's why my favorite language is Lua. Not only does it have flawless coroutine support, it doesn't use any tricks with the C stack, which otherwise can cause headaches when writing libraries or integrating into large applications, especially with POSIX threads.

There's no easy answer for C or C++ code. Juggling C stacks is ugly and problematic. The simplest and most straight-forward solution in C is to use GCC's computed gotos, which makes it easy to implement generators. It's _extremely_ useful, and that's why it was one of the earliest extensions. I used computed gotos all over the place in my non-blocking libraries, because I despise callbacks. Not just to restart an I/O operation, but to implement streaming-type behavior for routines which transform data in smallish chunks.