Moving to Python 3

In my experience, when you start sharing a lot of data between threads, you can take one of two approaches. You can have a giant lock that covers all threads. This is sort of like the old BKL or the GIL itself. One giant lock is simple, but it limits scalability a lot. Alternately, you can have many different little locks. This is great for performance and scalability, but hard on the poor programmers. Debugging becomes very difficult because runs are not repeatable. Code reuse is impaired because everything is tangled in this web of locks and you can't easily move code around.

When you have many little locks, the usual approach is to impose absolute ordering, so that if you take lock A before B at any point, you must always take A before B. That's what the kernel does. It seems to be the best strategy, but again, why impose this on yourself when you don't have to? Just don't share state unless you have to.

It's sad that programmers have been trained to think that threads are "simple" and "natural" and processes are "hard." That's the exact reverse of reality. I blame Windows and its high per-process overhead.