Fixing the unfixable autofs ABI

Posted May 1, 2012 2:46 UTC (Tue) by neilbrown (subscriber, #359)
In reply to: Fixing the unfixable autofs ABI by quotemstr
Parent article: Fixing the unfixable autofs ABI

Applying an old flag to a new context is always a best-effort sort of thing. It is rarely perfect.

O_DIRECT typically has significant alignment restrictions, and this new semantic has some sort of alignment restrictions (if you squint a bit).

O_SYNC is also available for pipes and the man page say "Any writes on the resulting file descriptor will block the calling process until the data has been physically written to the underlying hardware", and if we understand "the memory in the reading process" to be "the underlying hardware", then your suggested feature (which I like) would fit to some degree with O_SYNC.

The data-loss issue sounds problematic, but we seem to have lived with it with SOCK_DGRAM sockets so I suspect we will survive with pipes. Hopefully the FIONREAD ioctl will provide the number of bytes that can be read, so a well written program can avoid losing data.

Fixing the unfixable autofs ABI

Posted May 1, 2012 7:26 UTC (Tue) by butlerm (subscriber, #13312) [Link] (2 responses)

Discarding data isn't so much of a problem as knowing when data has been discarded. recvmsg(2) is nice for that. You generally can't have message per read semantics for arbitrary size messages in any case because the kernel buffers must store an entire message to avoid unusual complexities. (Ask the SCTP people about the fun problems you get trying to implement a point-to-multipoint socket that supports arbitrary size messages.)

Considering that reality, all you really need is some reliable way to know whether your message has been truncated. Sockets have MSG_PEEK / MSG_TRUNC for that, so I guess the question is why wouldn't it be practical to make recvmsg(2) work on a packetized pipe so that you can do the same thing?

Fixing the unfixable autofs ABI

Posted May 3, 2012 19:30 UTC (Thu) by kleptog (subscriber, #1183) [Link] (1 responses)

Interestingly, packetised pipes have existed for a while, see SOCK_SEQPACKET (possibly only supported for AF_UNIX). They're indeed useful in cases where you want to be sure that clients don't lose sync when the packets change size. Though a well designed protocol doesn't have the problem in the first place.

I suppose changing the type of pipe to userspace would be an even greater ABI change.

Fixing the unfixable autofs ABI

Posted May 10, 2012 5:53 UTC (Thu) by kevinm (guest, #69913) [Link]

Well, yes - changing the type of file descriptor used wouldn't fix the existing binaries, which is the whole problem (in the autofs protocol, the userspace program creates the pipe with pipe(2)).

I think it was mentioned somewhere in the thread that an SOCK_SEQPACKET socket would have been a better design, but hindsight is always an exact science.