2009 Linux Plumbers Conference Call For Topics

Posted Feb 13, 2009 20:17 UTC (Fri) by aliguori (guest, #30636)
Parent article: 2009 Linux Plumbers Conference Call For Topics

How about "Making userspace I/O not suck" :-)

Zach Brown has proposed acalls as a replacement for the problematic linux-aio subsystem. KVM would be an obvious consumer of such an interface although it's still not sufficient for our needs.

Rusty Russell has proposed vringfd as a mechanism for improving userspace raw networking performance. Unfortunately, this does not seem to have much of a future so brainstorming is required for a better interface.

In general, the interfaces we've inherited from Unix leaves a great deal to be desired.

to post comments

Making User-Space I/O Not Suck

Posted Feb 17, 2009 18:12 UTC (Tue) by PaulMcKenney (✭ supporter ✭, #9624) [Link] (1 responses)

I must confess that my first reaction to the UNIX interfaces back in the early 1980s was not all that positive, so I can relate! Of course, these interfaces aren't all that much younger than I am, and much has changed during the intervening years, virtualization being but one such change! My guess is that you are primarily interested in the interaction of userspace I/O and virtualization, though I suspect that non-virtualized environments must be taken into account as well.

You mention Zach and Rusty. Who else would need to be involved to make good forward progress? What forward progress (patches!) would it be reasonable to expect from sessions at the Linux Plumbers Conference, and what prework would be required in order to make such progress possible?

Making User-Space I/O Not Suck

Posted Feb 17, 2009 21:07 UTC (Tue) by aliguori (guest, #30636) [Link]

There are two important issues for virtualization. The first is performance. Since virtualizations acts as a layer between hardware and traditional OSes, there is a great deal of pressure in the virtualization layer to perform as well as possible.

For block devices, the ideal situation would be to be able to submit multiple asynchronous IO requests from the single context, and receive a notification through a file descriptor upon completion. Since we lack a generic mechanism to submit asynchronous requests, that's the first stumbling block. Zach's acall interface and Ingo's syslets both try to address this.

You also have an issue of functionality. The interface that we have to a virtual guest is a bit richer than the existing IO interfaces. When emulating a SCSI adapter, for instance, we receive requests with Simple, Ordered, or Head tags which control the completion order of each request when submitted asynchronously. There is no way, today, to submit barrier requests from userspace (let alone this richer barriers).

Networking is much the same. Today, the interface we have is tun/tap for sending raw network IO. We very much want to improve the performance of network IO by having zero-copy TX/RX. For TX, we can do this reasonably with some of the patches Rusty has posted previously. RX is much harder because in general 1) we don't have an interface for publishing RX buffers to the kernel 2) you need an interface for the network drivers to have them receive to a preallocated RX buffer. There is some discussion around making use of VMDq or SR-IOV to enable this.

This is an ongoing activity involving a lot of people. I think the progress that could made be depends on who would attend to be honest. So far, all of the usual suspects have been involved in these discussions but a lot of things have been stalled by lack of consensus of a common direction (acall vs. syslets vs. linux-aio is a good example of this).