Making CPython faster
Making CPython faster
Posted Jun 6, 2021 13:25 UTC (Sun) by gracinet (guest, #89400)In reply to: Making CPython faster by rghetta
Parent article: Making CPython faster
> But with the GIL multithreading is simply impossibile, because the GIL is not just a global lock, but a global interpreter lock, threads just serialize.
> And from a processor viewpoint each thread takes the GIL for a long time, so even if you use C modules to speed up, chances are that you don't get much real benefit.
> And from a processor viewpoint each thread takes the GIL for a long time, so even if you use C modules to speed up, chances are that you don't get much real benefit.
This is not 100% true. In fact this is only the worst case, and yes, it is common enough to be a concern.
C extensions can (and should!) release the GIL when they don't need to access Python objects (including the memory they reference).
One obvious case is waiting for I/O, another would be performing CPU-bound computations in directly allocated memory. I believe the likes of numpy do it (never checked that myself). Of course the standard library modules implemented in C are also expected to do it properly. It's not really complicated to implement either, but I hear it's often overlooked.
To be clear, I'm not saying the GIL isn't a problem, it's just not as bad as serializing the threads.