Speeding up CPython

Posted Dec 17, 2020 9:15 UTC (Thu) by quietbritishjim (subscriber, #114117)
In reply to: Speeding up CPython by Cyberax
Parent article: Speeding up CPython

The GIL affects fewer Python programs than you might think. The GIL is released by C extension modules, so Python programs that use those can use threads properly. For example, if you multiply matrices with numpy.dot from multiple threads, those calls will genuinely execute concurrently.

In my experience, many Python programs use C extension modules heavily anyway, especially those that are CPU intensive (which are the only ones you probably multithread anyway, apart from those using blocking IO which also releases the GIL of course). Note that this includes not only third-party libraries, like numpy (as mentioned), scipy, OpenCV, etc., but also a lot of the CPU-heavy standard library modules e.g. zipfile (the compression code, not just file IO) and sqlite3 (again, the whole lot, not just IO). It might seem to defeat the point of Python a bit from the library writer's perspective, because they still need to write their code in C (or Cython) to be useful in this way, but it makes a lot of sense from the application writer's perspective: you only need to write a small amount of Python glue code to take advantage of other people's C code.

Where this falls down is in the granularity of the calls. If you're multiplying a small number of huge matrices from multiple threads then the GIL is just irrelevant. If you're multiplying a very large number of tiny matrices from multiple threads then it could become more of an issue.

Speeding up CPython

Posted Dec 17, 2020 9:44 UTC (Thu) by Cyberax (✭ supporter ✭, #52523) [Link]

The problem with Python is that you quite often have something like a web-server. It can naturally be ran as multiple processes, so that each interpreter gets its own CPU.

This works fine. But then you want to add an in-memory cache and that's where the problems start. If each process gets its own cache, then you have a massive RAM waste. And that's the only easy option (before you go off into using Redis or memcached with all the associated issues).

It's not really any better for a lot of computational code as well. In my company we have Python code that runs computations for each hour of the year, almost completely independently of each other. It was quite liberating to switch from Python to C++ and get true multi-threading, and a huge performance boost.

Speeding up CPython

Posted Dec 17, 2020 19:20 UTC (Thu) by Bluehorn (subscriber, #17484) [Link]

> The GIL affects fewer Python programs than you might think.

I have fallen for this argument. Unfortunately.

This has been the argument for the GIL for as long as I can remember: Just optimize the inner loop in something like C++. This works if you have a tool that does some costly computation (solving of linear equations, FFT, scene rendering, ray tracing) at the core with some Python code doing the stuff around that is not time critical.

That is definitely a big domain and includes a lot of scientific computing (where Python seems to shine).

Unfortunately I maintain an application written in Python that does not have such a core but deals with lots and lots of small objects representing parts of a FE-model. And it is slow. When we started, we had small datasets so we did not worry too much - we'll just rewrite the relevant implementation details in C.

Only after I hit the performance wall I noticed that I basically had to rewrite most of the classes in C to replace the slow Python code. But since everything is held together by Python collections, manipulating those objects would either require holding the GIL or doing our own locking. Also, we would lose our database integration (which is using SQLAlchemy, no idea how to adapt this to C++ objects).

So we tried to use caching to avoid repeated computations. Which means we spent a lot of development time in fixing cache invalidation problems. Still the system was slow and customers (with 24 core systems) told us that they can't understand how the software performs that bad while not even loading the system. Just parallelize...

As rewriting in another language was out of the question, we implemented multi process concurrency. At last this improved things a bit (especially the user interface stays responsive even if the system is doing something), but at a hefty price. multiprocessing was unusable because this is a GUI application. Using fork wrecks the X11 connection, and the forkserver approach was not available in Python 2 when we started. So we wrote our own multi process IPC library.

I still love Python for writing scripts, the language is extremely expressive. But I am now wary about using it for applications that have to do simple actions on many diverse and small objects.