A proposal for "rebooted" Python asynchronous I/O support

Posted Dec 30, 2012 3:35 UTC (Sun) by wahern (subscriber, #37304)
In reply to: A proposal for "rebooted" Python asynchronous I/O support by tack
Parent article: A proposal for "rebooted" Python asynchronous I/O support

From the PEP Guido says they're basically just generators underneath, which means these aren't like more modern coroutine implementations which can yield from sub-subroutine invocations. Being restricted to the first-level invocation will hardly remove the necessity for callbacks in high-level application code, unfortunately.

This is why I much prefer Lua. It might not have the huge library, but at least the VM is done right.

A proposal for "rebooted" Python asynchronous I/O support

Posted Dec 30, 2012 4:36 UTC (Sun) by tack (guest, #12542) [Link] (2 responses)

You've misunderstood the PEP. Coroutines are implemented using generators, yes, but they can be arbitrarily nested (within runtime stack depth limits of course). In other words, coroutine A can "yield from" coroutine B which can "yield from" coroutine C and so on. If C yields a Future object, the stack unravels -- through B and A -- and the coroutine scheduler will resume C's execution when the Future it yielded is done.

A proposal for "rebooted" Python asynchronous I/O support

Posted Dec 31, 2012 23:47 UTC (Mon) by wahern (subscriber, #37304) [Link] (1 responses)

I think you misunderstood what I was saying. To understand the fundamental problem in the CPython (and Jython) VM, just think about the difference between CPython and Stackless. You can't `yield from' inside a new function invocation because of the way call frames are implemented in CPython. Only the callee of a `yield from' statement can yield. But you can explicitly stack yield froms in a call chain. Kinda messy, but it's a fundamental limitation of CPython.

Coroutines in languages like Lua are implemented more like threads, so coroutine semantics aren't limited to a single callee invocation. In Lua, invocation C in A->B->C can yield, even though coroutine resumption began in A. In CPython, when resuming A only A can yield. A can, however, _explicitly_ invoke B as a coroutine. But B (perhaps a library call) can't hide the details. A needs to adorn it's call to B in order to get coroutine semantics. Again, not so in Lua or some other languages.

A proposal for "rebooted" Python asynchronous I/O support

Posted Jan 1, 2013 0:10 UTC (Tue) by wahern (subscriber, #37304) [Link]

FWIW, I believe Python coroutines will work much like Javascript generator trampolines: http://www.neilmix.com/2007/02/07/threading-in-javascript...