Non-blocking buffered file read operations
Non-blocking buffered file read operations
Posted Sep 25, 2014 10:55 UTC (Thu) by mpr22 (subscriber, #60784)In reply to: Non-blocking buffered file read operations by simlo
Parent article: Non-blocking buffered file read operations
It seems to me that your proposal doesn't achieve what is desired:
The wish is to attempt to read the needed data in the non-blocking mode; should the data not be available, the request will be turned over to the thread pool for synchronous execution. If the thread pool is using the same file descriptor, its attempts to perform blocking reads will fail. If it uses a different file descriptor, it can run into weird problems relating to the surprising semantics of POSIX file locks (see this article for more information). So the ability to request non-blocking behavior on a per-read basis is needed.
Posted Sep 25, 2014 21:48 UTC (Thu)
by simlo (guest, #10866)
[Link] (1 responses)
Use of select()/poll() to get an event on a read- and/or write-able file descriptor, and read returns whatever is available, write writes unblocking until the kernel-buffer is full. In both cases the DMA interupt comes back and marks the filedescriptor read- or writeable again, such select or poll will wake up. Just as a the network interrupt wakes up my TCP socket.
I know with the current UNIX behaviour, where files are seen as an extension of the direct memory rather than IO, it doesn't make sense to be non-blocking for file-io.
So I turn the UNIX philosophy around: Instread of describing a network connection with a file descriptor, I will rather use a network descriptor for a files.
Posted Sep 26, 2014 23:45 UTC (Fri)
by giraffedata (guest, #1954)
[Link]
The problem discussed in the article is a different one from the one involved in blocking vs nonblocking socket I/O. In the socket case, it's a question of whether to wait for data to exist, while in the article it's about whether to wait to get existing data, since it may or may not be gettable quickly.
If you want to use select and nonblocking I/O to avoid waiting for file data to be read from the backing store, you need a rather different protocol. You need a system call to start a stream of file data into memory, and then read system calls to consume that data. (And you'd probably use a socket for that).
Non-blocking buffered file read operations
But files are much, much slower than memory, file-io can fail, file-io can via a network file system. It does not make sense to make file IO behave differently than network IO.
The analogous function for regular files to the select and nonblocking read function of a socket is for select to wait until there is data in the file at the current file pointer, and a blocking read to wait at eof until it isn't eof anymore because the file got bigger.
Non-blocking buffered file read operations