Moving to Python 3
Posted Feb 17, 2011 8:26 UTC (Thu) by rqosa
In reply to: Moving to Python 3
Parent article: Moving to Python 3
That way can scale poorly, because there must be at least one thread per FD.
epoll-driven main loop and a pool of worker threads (with one work queue per worker thread) makes the amount of threads become independent from the amount of FDs, so you can adjust the amount of threads to whatever gives the best performance. It also has the benefit of avoiding the overhead of thread-start-on-FD-open and thread-quit-on-FD-close, since you can reuse the existing threads. (Make it so that any idle thread will wait on a semaphore until its work queue becomes non-empty. Also, rather than using
epoll directly, use libevent, so that it's portable to non-Linux systems.)
to post comments)