|
|
Subscribe / Log in / New account

A viable solution for Python concurrency

A viable solution for Python concurrency

Posted Oct 15, 2021 10:43 UTC (Fri) by t-v (subscriber, #112111)
In reply to: A viable solution for Python concurrency by Paf
Parent article: A viable solution for Python concurrency

Having seen Sam's stellar work on PyTorch, it would seem that he looked at quite a few of the larger libraries in the data science corner - PyTorch obviously (which uses PyBind11), but also numpy, scipy, pandas, scikit-learn, scikit-image, and JAX - to inform the discussion he offers on extensions.

I'm sure there will be corner cases that need to be fixed in a lot of places. But then it'll be only in situations where people use multiple threads before they hit the corner cases, and given the current multithreading state of things, that should be quite a filter.

P.S.: Regarding alternative approaches: In the PyTorch context, people have been exploring quite a few aspects of "how to get Python-based DL models to run efficiently" from the TorchScript JIT/interpreter for a subset of Python, to multiprocessing to subinterpreter-style multithreading, so to me this looks like part of this larger picture.


to post comments


Copyright © 2025, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds