> Where I first started running up against problems was in a GUI that had a
> central append-only (which eased the locking constraints) data structure,
> and a corresponding tree of threads. Each thread could generate a new
> node(s) on the data structure, which might need more analysis by more
> threads. The operations on the data structure that needed to be thread
> safe had the thread safety encapsulated in the data structure. It just
> fell naturally into a threaded design.
In my experience, when you start sharing a lot of data between threads, you can take one of two approaches. You can have a giant lock that covers all threads. This is sort of like the old BKL or the GIL itself. One giant lock is simple, but it limits scalability a lot. Alternately, you can have many different little locks. This is great for performance and scalability, but hard on the poor programmers. Debugging becomes very difficult because runs are not repeatable. Code reuse is impaired because everything is tangled in this web of locks and you can't easily move code around.
When you have many little locks, the usual approach is to impose absolute ordering, so that if you take lock A before B at any point, you must always take A before B. That's what the kernel does. It seems to be the best strategy, but again, why impose this on yourself when you don't have to? Just don't share state unless you have to.
It's sad that programmers have been trained to think that threads are "simple" and "natural" and processes are "hard." That's the exact reverse of reality. I blame Windows and its high per-process overhead.