2 threads or 100 threads? [LWN.net]

2 threads or 100 threads?

Posted Aug 21, 2024 3:30 UTC (Wed) by wujj123456 (subscriber, #84680)
Parent article: Python subinterpreters and free-threading

> "if I split that in half and executed between two threads in Python 3.12, it would take twice as long"

The code and comment indicates it's splitting into 100 threads to get 2x slower. This kind of overhead is more plausible than having only 2 threads but 2x slower.

to post comments

2 threads or 100 threads?

Posted Aug 22, 2024 3:10 UTC (Thu) by kenmoffat (guest, #4807) [Link] (1 responses)

You have not clarified which of the links you are commenting on, and I find some of them hard to read. But if you split something into threads and it takes longer, then what is the point ?

I assume whatever you commented on illustrated a problem which either has now been , or needs to be, fixed. But for those of us who have little experience in an area, context is always useful.

2 threads or 100 threads?

Posted Aug 22, 2024 7:20 UTC (Thu) by wujj123456 (subscriber, #84680) [Link]

The context is in the article, and there is only one link that could plausibly be the one I'm referring to in the paragraph containing the sentence I quoted: https://gist.github.com/tonybaloney/24d545ed855a3c90f8442...

You don't need to understand the code snippet either, because it was commented with "Split array into blocks of 100 and start a thread for each", which is in conflict with the quoted sentence in the article. My question is just asking to confirm whether it's 2 thread or 100 threads causing the 2x slowdown, because that points to very different per-thread synchronization overhead from the GIL lock.

2 threads or 100 threads?

Posted Aug 22, 2024 16:11 UTC (Thu) by jake (editor, #205) [Link] (1 responses)

I think Anthony was just making a general statement about what *would* happen if he were to do that. I checked the video and the quote is correct, though I think he misstates what he means ... the part that he showed in his slides is just the upper "Crude sample" part from that link. Now that you point it out, though, I suspect he did not actually mean that it took twice as long, just that it would take the same amount of time as the simple non-threaded version because the GIL would not be released so (any number of) threads would serialize.

jake

2 threads or 100 threads?

Posted Aug 24, 2024 6:37 UTC (Sat) by wujj123456 (subscriber, #84680) [Link]

Got it. Somehow I missed the link to video. I agree with you given Anthony was just stating that the threads couldn't run in parallel. Thanks for the continued coverage on this interesting topic.