An introduction to asynchronous Python

In his PyCon 2017 talk, Miguel Grinberg wanted to introduce asynchronous programming with Python to complete beginners. There is a lot of talk about asynchronous Python, especially with the advent of the asyncio module, but there are multiple ways to create asynchronous Python programs, many of which have been available for quite some time. In the talk, Grinberg took something of a step back from the intricacies of those solutions to look at what asynchronous processing means at a higher level.

He started by noting that while he does a lot of work on the Flask Python-based web microframework, this talk would not be about Flask. He did write the Flask Mega-Tutorial (and a book on Flask), but he would be trying to mention it less than ten times during the talk—a feat that he managed admirably. He has also developed a Python server for Socket.IO that started out as something for "that framework", but has since "taken on a life of its own".

He asked attendees if they had heard people say that "async makes your code go fast". If so, he said, his talk would explain why people say that. He started with a simple definition of "async" (as "asynchronous" is often shortened). It is one way of doing concurrent programming, which means doing many things at once. He is not only referring to asyncio here as there are many ways to have Python do more than one thing at once.

He then reviewed those mechanisms. First up was multiple processes, where the operating system (OS) does all the work of multi-tasking. From CPython (the reference Python implementation) that is the only way to use all the cores in the system. Another way to do more than one thing at once is by using multiple threads, which is also a way to have the OS handle the multi-tasking, but Python's Global Interpreter Lock (GIL) prevents multi-core concurrency. Asynchronous programming, on the other hand, does not require OS participation. There is a single process and thread, but the program can get multiple things done at once. He asked: "what's the trick?"

Chess

He turned to a real-world example of how this works: a chess exhibition, where a chess master takes on, say, 24 opponents simultaneously. "Before computers killed the fun out of chess", these kinds of exhibitions were done regularly, but he is not sure if they still are. If each game takes around 30 move pairs to complete, the master would require twelve hours to finish the matches if they were played consecutively (at one minute per move pair). By sequentially making moves in each game, though, the whole exercise can be completed in an hour. The master simply makes a move at a board (in, say, five seconds) and then goes on to the next, leaving the opponent lots of time to move before the master returns (after making 23 other moves). The master will "cream everyone" in that time, Grinberg said.

It is "this kind of fast" that people are talking about for async programming. The chess master is not optimized to go faster, the work is arranged so that they do not waste time waiting. "That is the complete secret" to asynchronous programming, he said, "that's how it works". In that case, the CPU is the chess master and it waits the least amount of time possible.

But attendees are probably wondering how that can be done using just one process and one thread. How is async implemented? One thing that is needed is a way for functions to suspend and resume their execution. They will suspend when they are waiting and resume when the wait is over. That sounds like a hard thing to do, but there are four ways to do that in Python without involving the OS.

The first way is with callback functions, which is "gross", he said; so gross, in fact, that he was not even going to give an example of that. Another is using generator functions, which have been a part of Python for a long time. More recent Pythons, starting with 3.5, have the async and await keywords, which can be used for async programs. There is also a third-party package, greenlet, that has a C extension to Python to support suspend and resume.

There is another piece needed to support asynchronous programming: a scheduler that keeps track of suspended functions and resumes them at the right time. In the async world, that scheduler is called an "event loop". When a function suspends, it returns control to the event loop, which finds another function to start or resume. This is not a new idea; it is effectively the same as "cooperative multi-tasking" that was used in old versions of Windows and macOS.

Examples

Grinberg created examples of a simple "hello world" program using some of the different mechanisms. He did not get to all of them in the presentation and encouraged the audience to look at the rest. He started with a simple synchronous example that had a function that slept for three seconds between printing "Hello" and "World!". If he called that in a loop ten times, it would take 30 seconds to complete since each function would run back to back.

He then showed two examples using asyncio. They were essentially the same, but one used the @coroutine decorator for the function and yield from in the body (the generator function style), while the other used async def for the function and await in the body. Both used the asyncio version of the sleep() function to sleep for three seconds between the two print() calls. Beyond those differences, and some boilerplate to set up the event loop and call the function from it, the two functions had the same core as the original example. The non-boilerplate differences are by design; asyncio makes the places where code suspends and resumes "very explicit".

The two programs are shown below:

    # async/await version
    
    import asyncio
    loop = asyncio.get_event_loop()

    async def hello():
        print('Hello')
	await asyncio.sleep(3)
	print('World!')

    if __name__ == '__main__':
	loop.run_until_complete(hello())


    # @coroutine decorator version

    import asyncio
    loop = asyncio.get_event_loop()

    @asyncio.coroutine
    def hello():
        print('Hello')
	yield from asyncio.sleep(3)
	print('World!')

    if __name__ == '__main__':
	loop.run_until_complete(hello())

Running the program gives the expected result (three seconds between the two strings), but it gets more interesting if you wrap the function call in a loop. If the loop is for ten iterations, the result will be ten "Hello" strings, a three-second wait, then ten "World!" strings.

There are other examples for mechanisms beyond asyncio, including for greenlet and Twisted. The greenlet examples look almost exactly the same as the synchronous example, just using a different sleep(). That is because greenlet tries to make asynchronous programming transparent, but hiding those differences can be a blessing and a curse, Grinberg said.

Pitfalls

There are some pitfalls in asynchronous programming and people "always trip on these things". If there is a task that requires heavy CPU use, nothing else will be done while that calculation is proceeding. In order to let other things happen, the computation needs to release the CPU periodically. That could be done by sleeping for zero seconds, for example (using await asyncio.sleep(0)).

Much of the Python standard library is written in blocking fashion, however, so the socket, subprocess, and threading modules (and other modules that use them) and even simple things like time.sleep() cannot be used in async programs. All of the asynchronous frameworks provide their own non-blocking replacements for those modules, but that means "you have to relearn how to do these things that you already know how to do", Grinberg said.

Eventlet and gevent, which are built on greenlet, both monkey patch the standard library to make it async compatible, but that is not what asyncio does. It is a framework that does not try to hide the asynchronous nature of programs. asyncio wants you to think about asynchronous programming as you design and write your code.

Comparison

He concluded his talk with a comparison of processes, threads, and async in a number of different categories. All of the techniques optimize the waiting periods; processes and threads have the OS do it for them, while async programs and frameworks do it for themselves. Only processes can use all cores of the system, however, threads and async programs do not. That leads some to write programs that combine one process per core with threads and/or async functions, which can work quite well, he said.

Scalability is "an interesting one". Running multiple processes means having multiple copies of Python, the application, and all of the resources used by both in memory, so the system will run out of memory after a fairly small number of simultaneous processes (tens of processes are a likely limit), Grinberg said. Threads are more lightweight, so there can be more of those, on the order of hundreds. But async programs are "extremely lightweight", such that thousands or tens of thousands of simultaneous tasks can be handled.

The blocking standard library functions can be used from both processes and threads, but not from async programs. The GIL only interferes with threads, processes and async can coexist with it just fine. But, he noted, there is only "some" interference from the GIL even for threads in his experience; when threads are blocked on I/O, they will not be holding the GIL, so the OS will give the CPU to another thread.

There are not many things that are better for async in that comparison. The main advantage to asynchronous programs for Python is the massive scaling they allow, Grinberg said. So if you have servers that are going to be super busy and handle lots of simultaneous clients, async may help you avoid going bankrupt from buying servers. The async programming model may also be attractive for other reasons, which is perfectly valid, but looking strictly at the processing advantages shows that scaling is where async really wins.

A YouTube video of Grinberg's talk is available; the Speaker Deck slides are similar, but not the same as what he used.

[I would like to thank The Linux Foundation for travel assistance to Portland for PyCon.]