Recent Features
| |||||||||||||
Fixing the unfixable autofs ABIFixing the unfixable autofs ABIPosted Apr 30, 2012 21:09 UTC (Mon) by quotemstr (subscriber, #45331)Parent article: Fixing the unfixable autofs ABI Packetized pipes seem like a bad idea. 1) O_DIRECT doesn't what this patch says it should mean. O_DIRECT means "no buffering", not "weird read(2) games". O_DIRECT on a pipe ought to mean something like "block write(2) until someone blocks in read(2), then memcpy directly to the reader's buffer", and with this patch, O_DIRECT can never mean that for pipes. 2) This "packetized pipe" system looks like an invitation for data loss. It's one thing to create a pipe mode that preserves message boundaries (like NT's PIPE_TYPE_MESSAGE named pipes), but the right behavior should be to cause read(2) to fail unless the reader gets the whole packet. (That's how NT message pipes work.) Because of the silent data loss issue, these packetized pipes aren't truly general-purpose. In general, this approach seems like an overly general solution to a very particular problem.
(Log in to post comments)
Fixing the unfixable autofs ABI Posted May 1, 2012 2:46 UTC (Tue) by neilbrown (subscriber, #359) [Link]
Applying an old flag to a new context is always a best-effort sort of thing. It is rarely perfect.
O_DIRECT typically has significant alignment restrictions, and this new semantic has some sort of alignment restrictions (if you squint a bit).
O_SYNC is also available for pipes and the man page say "Any writes on the resulting file descriptor will block the calling process until the data has been physically written to the underlying hardware", and if we understand "the memory in the reading process" to be "the underlying hardware", then your suggested feature (which I like) would fit to some degree with O_SYNC.
The data-loss issue sounds problematic, but we seem to have lived with it with SOCK_DGRAM sockets so I suspect we will survive with pipes. Hopefully the FIONREAD ioctl will provide the number of bytes that can be read, so a well written program can avoid losing data.
Fixing the unfixable autofs ABI Posted May 1, 2012 7:26 UTC (Tue) by butlerm (subscriber, #13312) [Link] Discarding data isn't so much of a problem as knowing when data has been discarded. recvmsg(2) is nice for that. You generally can't have message per read semantics for arbitrary size messages in any case because the kernel buffers must store an entire message to avoid unusual complexities. (Ask the SCTP people about the fun problems you get trying to implement a point-to-multipoint socket that supports arbitrary size messages.)Considering that reality, all you really need is some reliable way to know whether your message has been truncated. Sockets have MSG_PEEK / MSG_TRUNC for that, so I guess the question is why wouldn't it be practical to make recvmsg(2) work on a packetized pipe so that you can do the same thing?
Fixing the unfixable autofs ABI Posted May 3, 2012 19:30 UTC (Thu) by kleptog (subscriber, #1183) [Link]
Interestingly, packetised pipes have existed for a while, see SOCK_SEQPACKET (possibly only supported for AF_UNIX). They're indeed useful in cases where you want to be sure that clients don't lose sync when the packets change size. Though a well designed protocol doesn't have the problem in the first place.
I suppose changing the type of pipe to userspace would be an even greater ABI change.
Fixing the unfixable autofs ABI Posted May 10, 2012 5:53 UTC (Thu) by kevinm (guest, #69913) [Link]
Well, yes - changing the type of file descriptor used wouldn't fix the existing binaries, which is the whole problem (in the autofs protocol, the userspace program creates the pipe with pipe(2)).
I think it was mentioned somewhere in the thread that an SOCK_SEQPACKET socket would have been a better design, but hindsight is always an exact science.
|
|||||||||||||