SLQB - and then there were four [LWN.net]

SLQB - and then there were four

Posted Dec 17, 2008 9:09 UTC (Wed) by iq-0 (subscriber, #36655)
Parent article: SLQB - and then there were four

I think it's a pretty neat/clean design. Though it probably has more overhead on very large number of CPUs system than SLUB. But it's a long time ago that I read about SLUB internals to be really sure.

The big question is:
Why not allocate from 'rlist' when you're done with your own freelist? We don't have to update the metadata so it's a very cheap operation.

Pherhaps it would even be better to put items on 'rlist' to also be put on the 'freelist', so we simply allocate the least recently used item first (probably cache-hot).
Sure the remote item might be bounced back to the other CPU, but clearly the code using it doesn't seem to mind which CPU last used it and with the LRU logic it's just as likely still in our CPU cache. And if it did bounce back in te meanwhile it means we are probably dealing with a slab cache that's not that hard used (or it's usage should be optimized and this would be true for all slab allocaters).

And without having looked at the code I'd assume that only one CPU at a time is cleaning up it's 'rlist' and that usage of the 'remote_free' lock is only a "best effort" locking scheme (try_lock()-ish) because maintenance should really be a background task with minimal overhead. Though this might have problems I overlook (or is already done that way).

Just some thoughts I had reading this text, nothing really thoroughly thought through though.

to post comments

SLQB - and then there were four

Posted Dec 17, 2008 10:11 UTC (Wed) by Nick (guest, #15060) [Link]

See my above comment -- effectively we do allocate from rlist if it is from the same node. Actually, what really happens is that objects from the same node but different CPU are freed straight onto our freelist rather than our rlist -- they only get sent to rlist when our freelist is trimmed. So it's exactly as you suggest.

The issue of cleaning up rlist is interesting. There are so many ways this can be done and it is about the most difficult part of a slab allocator... No, any CPU can be cleaning its rlist at any time, and yes they might all point to a single remote CPU. That's quite unlikely and the critical section is very short, so hopefully it won't be a problem. But I don't claim to know what the best way to do it is.

Very large number of CPUs I am definitely interested in... so I'm hoping to be as good or better than the other allocators here, but we'll see.

SLQB - and then there were four

Posted Dec 17, 2008 10:59 UTC (Wed) by sdalley (subscriber, #18550) [Link] (1 responses)

> Just some thoughts I had reading this text, nothing really thoroughly thought through though.

.. and it's even harder to plough through those details with a tough cough and hiccough ...

SLQB - and then there were four

Posted Dec 17, 2008 13:05 UTC (Wed) by jengelh (subscriber, #33263) [Link]

That reminds me of the Being Bilingual sketch. And as I see it, sdalley cheated on all four /^th/ words. :-)