I'm calling you on "Caller Locks" [LWN.net]

I'm calling you on "Caller Locks"

Posted Jul 3, 2009 17:30 UTC (Fri) by prl (guest, #44893)
Parent article: Linux kernel design patterns - part 3

Neil, thanks for your articles. I've been away, hence the late comment.

But... I want to call your bluff regarding locking. In part 2 you say:

If locking is needed, then the user of the data structure is likely to know the specific needs so all locking details are left up to the caller (we call that "caller locking" as opposed to "callee locking". Caller locking is more common and generally preferred).

and in the summary you state:

When there is any doubt, choose to have the caller take locks rather than the callee. This puts more control in that hands of the client of a function.

Hmm. Can you actually back up these assertions? Anyone reading the summary might assume that you have discussed this in the previous paragraphs, but it appears not. I don't think that being "more common" is a good recommendation; could you give us even just an LKML post which supports your claim that caller locking is "generally preferred"?

[ As an aside, I find the term "locking" in this kind of article problematic. Locking is one way to perform synchronisation, a very common way, true, but this leads people to refer to "locking" when they mean is generically "being SMP safe". ]

Do you necessarily want more control in the hands of a client? Surely it depends on the client. If we are talking about utilities used by drivers where the clients are written by hardware vendors struggling to get their first Linux contributions right, then you'll have a hard time convincing me. If a subsystem can handle all the sync it needs internally, clients don't need to worry about it at all, and the library developer can keep any eye on the best way to achieve this, as there are likely to be many more different configurations and usage patterns than such a client is likely to consider.

Here's one... given that the only real way that chip designers currently have to increase processor power is putting in more cores, I'm guessing that we'll see more architectures having more support for concurrency hardware. Subsystems which deal with their own syncing are likely to cope much better with using these variants as they become available.

Can I suggest that a better statement would be that callers need to better specify their synchronisation requirements, e.g. for performance or semantics, so that callees can better accommodate them. This may include a variant which explicitly means "Don't worry about sync, I already hold a lock", but that's just one form - and not one recommended for general use unless one really knows why..

(Log in to post comments)

I'm calling you on "Caller Locks"

Posted Jul 4, 2009 0:51 UTC (Sat) by neilbrown (subscriber, #359) [Link]

OK - I concede. My bluff is called and I don't have a rabbit to pull out of a hat to back up my hasty statements.

I do have a vague memory of a well known kernel developer - with 4 initials - saying something about a preference for caller-locks, but I cannot find it and so am left without the protection of "proof by authority".

I would still maintain that in the context of generic data structures provided by a library, the evidence within the kernel is in favour of caller-locking rather than callee locking, but is an unjustifiable step to claim that it is, therefore, preferred.

In a different context, there is plenty of evidence that callee locking is preferred. There has recently been more work towards removing the BKL, and some of this has involved pushing the locking down from caller to callee, and actually changing a locking style seems to suggest a strong preference.

This is quite a different context though. The callee is not a distinct library routine that will be called from a number of places (each with different SMP-safety requirements), but is one of a number of per-device implementations of a common interface (e.g. ioctl). So instead of one function called from lots of places, you have lots of functions called from one place. Having callee locking in this case is good as it provides the callee with more control of locking, including the choice not to lock at all.

I think your example concerning multi-core devices lines up with the second scenario, and they both fit under the "midlayer mistake" pattern to some extent.

Under that pattern, the subsystem should be given as much control as possible, not having anything imposed on it by any midlayer - and in particular not having any locking imposed on it.

So maybe what I really wanted to say is that for a library interface it is usually best to choose caller-locking, while for a subsystem interface, it is usually best to choose callee-locking. At least, that is what 10 minutes of thought leads me to. I wonder if it is possible to come up with a strong definition of the difference between a "library" interface and a "subsystem" interface...

Thanks for your challenge, I really appreciate it!