User: Password:
Subscribe / Log in / New account

De Icaza: Callbacks as our Generations' Go To Statement

De Icaza: Callbacks as our Generations' Go To Statement

Posted Aug 16, 2013 7:39 UTC (Fri) by smurf (subscriber, #17840)
Parent article: De Icaza: Callbacks as our Generations' Go To Statement

Having recently converted a largish Python program from Twisted Deferred handlers (callbacks) to Gevent (coroutines), I have to agree: callbacks are evil.

Callbacks are a twisty mess. My main problem with them is that the call graph gets lost; you get an exception somewhere and you have no idea whatsoever which code queued the callback that queued the callback that died on you.

(Log in to post comments)

De Icaza: Callbacks as our Generations' Go To Statement

Posted Aug 16, 2013 13:44 UTC (Fri) by kjp (subscriber, #39639) [Link]

Someday I'm looking forward to trying the "yield from" syntax in python 3.4 for coroutines. Seems much, much simpler.

De Icaza: Callbacks as our Generations' Go To Statement

Posted Aug 16, 2013 22:17 UTC (Fri) by ssmith32 (subscriber, #72404) [Link]

Ok, I still need to read the friendly article - and I will, Miguel's articles are usually quite good, especially when I don't know something or don't initially agree with his point...

But I *had* to echo your sentiment - I worked on a large Twisted program, and, yes, it was also a large twisted program. Especially when it comes to exceptions. The best part was there were parts where exceptions got swallowed, or dumped to stderr in a daemon that was only set up to log stdout.. so, yeah, you basically had no idea what was going on...

It could be possible to program neatly in Twisted, but the coroutine stuff I've seen is always much much cleaner (no experience in it though).

I have to say, though - in more cases than most people think, threads really aren't as evil as people say. Especially with some of the nicer higher-level libs that provide the thread pool and work queue components built-in. I think threads are confusing, but get beat on a little too much...

De Icaza: Callbacks as our Generations' Go To Statement

Posted Aug 16, 2013 22:48 UTC (Fri) by ssmith32 (subscriber, #72404) [Link]

Ok, read it,, and hrmm.. seems like, at least in his example, threads would be simplest.

He had a method that took and picture and posted it.
Each part of the method that did any I/O, he had "await" in it, so the program could continue.

If he just wrapped the method in a thread, he'd have it execute asynchronously AND the method would be even simpler, AND you would know when it was executing, like so:

Thread thr = new Thread( TakePhotoAndPost );

What confuses me about coroutines and callbacks both, is that you have even less of an idea when things execute - you have no idea when it's taking a picture and posting it... and these things usually matter. I mean you need to know when it takes a picture, at least, so you take the right one...

De Icaza: Callbacks as our Generations' Go To Statement

Posted Aug 21, 2013 11:13 UTC (Wed) by alankila (guest, #47141) [Link]

The general problem here is that thr.join() is not generally permissible. You aren't allowed to wait for results of another thread in the first thread, which is usually dedicated for UI event processing. You can only join if you know it has quit already, because then you know that the join() will return without a delay.

I've had to solve these kind of problems and I typically startup a thread pool with number of threads equal to CPU cores of system, and then start pushing work items into them. If multiple work items are dependent on each other before a further work item can be scheduled, then the work items have a counter which decrements by 1 every time the item is done and when that counter reaches 0, then another work item is pushed into the thread queue. There's also typically a way to schedule work items to be run in the UI event loop, so the work can continue in context of that thread later on.

The problem here is that it's all kinda confusing-looking. Hence the motivation for something like async/await. It does look pretty nifty but I haven't really studied it in detail so far.

De Icaza: Callbacks as our Generations' Go To Statement

Posted Aug 20, 2013 22:19 UTC (Tue) by simlo (guest, #10866) [Link]

From this comment as well as the main article the main problem of callbacks seems to be error handling using exceptions.

An error is an event. I.e. in the event based, callback driven programming exceptions have to handled in specific callbacks. Say for instance a connect to a TCP port fails with "connection refused." If connect() is blocking throw an exception. But with non-blocking connect() you will get the result of the connect() operation later maybe in the same callback handling a successfull connect(). So you are forced have to have error handling right there and decide what to do. You can't wrap the whole long sequence into one big try catch and handle it all in one big error handler.

To me it sounds like event-callback driven coding actually forces the coder to handle all failures intelligently. Maybe a showMessage("Connection failed: <reason>") or simply try the connect() again, or whatever is a proper failure mode.

This only strengthens by faith in single threaded, event-callback driven programs not using exceptions. And yes, as mentioned in comments above proper GUI toolkits also supports that, so it is not only for server-programming.
At me this works very well for embedded programs written in C++. I compare with Java with a lot of threads. The C++ programs are much more stable and have far fewer flaws with error handling in for instance handling of TCP connections.

Copyright © 2017, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds