Modifying the Python object model [LWN.net]

Modifying the Python object model

Posted May 16, 2018 23:17 UTC (Wed) by roc (subscriber, #30627)
Parent article: Modifying the Python object model

> the garbage collector is reference-count based, which has poor locality

This is counter-intuitive. Generally speaking, reference counting has better data-cache locality than tracing GC, because you normally read or write an object's reference count around the same time as you touch the object for other reasons, whereas tracing GC must periodically scan all live objects whether they have recently been used or not.

Compacting or generational GC can improve locality by improving the compactness of allocations, but I would be surprised if Instagram was able to fit that into CPython.

So I wonder what's behind this statement.

to post comments

Modifying the Python object model

Posted May 16, 2018 23:39 UTC (Wed) by malefic (subscriber, #37306) [Link] (1 responses)

They probably meant increased memory fragmentation which results from keeping refcounts in every object structure.

Modifying the Python object model

Posted May 17, 2018 1:04 UTC (Thu) by roc (subscriber, #30627) [Link]

Refcounts may increase memory usage (but not by much usually), but won't increase fragmentation per se. Unless Python does something really strange.

Modifying the Python object model

Posted May 17, 2018 1:15 UTC (Thu) by Cyberax (✭ supporter ✭, #52523) [Link] (3 responses)

You need interlocked operations for refcounts if you want multithreading. This causes merry hell with cache lines bouncing around.

Modifying the Python object model

Posted May 17, 2018 10:06 UTC (Thu) by roc (subscriber, #30627) [Link] (2 responses)

Sure, but the Python interpreter can't share object between threads and the article doesn't say anything about changing that.

Modifying the Python object model

Posted May 18, 2018 14:59 UTC (Fri) by mb (subscriber, #50428) [Link] (1 responses)

Of course objects can be shared. That is the whole point of threads.

Modifying the Python object model

Posted May 18, 2018 22:00 UTC (Fri) by roc (subscriber, #30627) [Link]

OK that's true, but all access to Python objects is protected by the GIL so there is no need to use atomic operations to modify the refcount. (I should have said that objects can't be shared by concurrently executing threads.)

Modifying the Python object model

Posted May 20, 2018 5:20 UTC (Sun) by peterfeiner (subscriber, #97524) [Link]

>> the garbage collector is reference-count based, which has poor locality
> This is counter-intuitive. Generally speaking, reference counting has better data-cache locality than tracing GC, because you normally read or write an object's reference count around the same time as you touch the object for other reasons, whereas tracing GC must periodically scan all live objects whether they have recently been used or not.

I think the problem is when the last reference goes away. An object isn't being used anymore, yet its (first) cacheline is made hot. This is bad because the problem cascades when a whole object graph is freed. Lots of disparate cachelines getting fetched just to do bookkeeping.

Modifying the Python object model

Posted May 20, 2018 6:33 UTC (Sun) by njs (subscriber, #40338) [Link] (2 responses)

It may have been a garbled reference to the way refcounting causes memory write traffic all over the place. Workloads like Instagram's are often memory-bound, and they rely heavily on starting up a parent process, loading in their main code and data, and then forking it to get lots of workers that all share most of their data through CoW. But writes from refcounts and marking can break the CoW and create unnecessarily high memory usage. (They used to disable the GC entirely, and then later contributed a feature to Python to mark their shared objects as immortal so the GC would skip marking them.)

[1] https://instagram-engineering.com/dismissing-python-garba...
[2] https://bugs.python.org/issue31558

Modifying the Python object model

Posted May 20, 2018 18:21 UTC (Sun) by rweikusat2 (subscriber, #117920) [Link] (1 responses)

The extremely amusing thing here is obviously that they disabled the additional, cycle-breaking tracing collector because it was causing their problems :->.

Modifying the Python object model

Posted May 24, 2018 8:36 UTC (Thu) by njs (subscriber, #40338) [Link]

Indeed! That's part of why I'm uncertain and just guessing :-). But it is perhaps easier to fix the scattered memory writes problem with a tracing collector than with refcounting.

Or heck, maybe he meant he wanted a compacting collector... that would certainly improve memory locality in some cases.