Poettering: Revisiting how we put together Linux systems

Posted Sep 3, 2014 19:30 UTC (Wed) by ermo (subscriber, #86690)
In reply to: Poettering: Revisiting how we put together Linux systems by bfields
Parent article: Poettering: Revisiting how we put together Linux systems

I was going to naïvely ask why POSIX hadn't adopted the superior (in the context of e.g. NFS) BSD-style locking and then you posted that link, which contains this little gem at the very top:

"File-private POSIX locks are an attempt to take elements of both BSD-style and POSIX locks and combine them into a more threading-friendly file locking API."

Sounds like the above is just what the doctor ordered?

Poettering: Revisiting how we put together Linux systems

Posted Sep 3, 2014 23:07 UTC (Wed) by nix (subscriber, #2304) [Link] (7 responses)

It doesn't really help. The problem is that local fses have multiple locks which do not conflict with each other, but the NFS protocol has only one way to signal a lock to the remote end. So there's a trilemma: either you use that for all lock types (and suddenly they conflict remotely where they did not locally), or you don't signal one lock type at all (and suddenly you have things not locking at all remotely where they did locally), or you use a protocol extension, which has horrible compatibility problems.

I don't see a way to solve this without a new protocol revision :(

Poettering: Revisiting how we put together Linux systems

Posted Sep 4, 2014 14:03 UTC (Thu) by foom (subscriber, #14868) [Link]

It does help because it removes any reason to use the BSD lock API (at least when running on Linux, new enough kernel). Before that addition, the POSIX lock programming model was so broken, nobody sane would ever *want* to use it.

Yet, on Linux, local POSIX locks interoperate properly with POSIX locks via NFS, so, if software all switches to using POSIX locks, it'll work properly when used both locally and remotely at the same time.

Of course, very often, nothing is ever running on the NFS server that touches the exported data (or at least, nothing that needs to lock it) -- the NFS server is *just* a fileserver. In such an environment, using BSD locks over NFS on linux works properly too.

Poettering: Revisiting how we put together Linux systems

Posted Sep 5, 2014 0:44 UTC (Fri) by mezcalero (subscriber, #45103) [Link] (5 responses)

I think a big step forward would actually be if the NFS implementations were honest, and would return a clean error if they cannot actually provide correct locking. But that's not what happens, you have no way to figure what is going on a file system...

Just pretending that locking works, even if it doesn't, and returning success to apps is really the worst thing to do...

Poettering: Revisiting how we put together Linux systems

Posted Sep 8, 2014 16:12 UTC (Mon) by nix (subscriber, #2304) [Link] (4 responses)

You're suggesting erroring if a lock of one type is held on on a file when an attempt is made to take out a lock of the other type? I suspect this is the only possible fix, if you can call it a fix. Now we just have to hope that programs check for errors from the locking functions! But of course they will, everyone checks for errors religiously :P

Poettering: Revisiting how we put together Linux systems

Posted Sep 8, 2014 18:56 UTC (Mon) by bfields (subscriber, #19510) [Link] (1 responses)

That wouldn't help. I think he's suggesting just returning -ENOLCK to BSD locks unconditionally. I agree that that's cleanest but in practice I suspect it would break a lot of existing setups.

I suppose you could make it yet another mount option and then advocate making it the default. Or just add support NFS protocol support for BSD locks if it's really a priority, doesn't seem like it should be that hard.

Poettering: Revisiting how we put together Linux systems

Posted Sep 9, 2014 13:56 UTC (Tue) by nix (subscriber, #2304) [Link]

That wouldn't help. I think he's suggesting just returning -ENOLCK to BSD locks unconditionally. I agree that that's cleanest but in practice I suspect it would break a lot of existing setups.

Given how awful POSIX locks are (until you have a very recent kernel and glibc 2.20), and how sane people therefore avoided using the bloody things, I'd say it would break almost every setup relying on locking over NFS at all. A very bad idea.

Poettering: Revisiting how we put together Linux systems

Posted Sep 9, 2014 14:43 UTC (Tue) by foom (subscriber, #14868) [Link] (1 responses)

I don't think he was suggesting that, but that's actually what BSD does with BSD/POSIX locks.
A BSD lock will block a POSIX lock, and v.v.. (At least that's what happens locally; no idea what the BSD's NFS clients do.)

Linux also had that behavior a long time ago IIRC. Not sure why it changed, that was before I paid attention.

Poettering: Revisiting how we put together Linux systems

Posted Sep 9, 2014 15:27 UTC (Tue) by bfields (subscriber, #19510) [Link]

Huh. A freebsd man page agrees with you:

https://www.freebsd.org/cgi/man.cgi?query=flock&sektion=2
If a file is locked by a process through flock(), any record within the file will be seen as locked from the viewpoint of another process using fcntl(2) or lockf(3), and vice versa.

Recent linux's flock(2) suggests the Linux behavior was an attempt to match BSD behavior that has since changed?:

http://man7.org/linux/man-pages/man2/flock.2.html
Since kernel 2.0, flock() is implemented as a system call in its own right rather than being emulated in the GNU C library as a call to fcntl(2). This yields classical BSD semantics: there is no interaction between the types of lock placed by flock() and fcntl(2), and flock() does not detect deadlock. (Note, however, that on some modern BSDs, flock() and fcntl(2) locks do interact with one another.)

Strange. In any case, changing the local Linux behavior is probably out of the question at this point.