Async I/O [LWN.net]

Async I/O

Posted Apr 7, 2009 19:34 UTC (Tue) by mjthayer (guest, #39183)
In reply to: Async I/O by NAR
Parent article: Linux Storage and Filesystem workshop, day 1

At least traditionally, I think that AIO has been much more scalable than threaded I/O, not least because you don't need to keep a stack around for each thread. And some people prefer to keep everything on a single thread, because they don't want to deal with multi-threading issues, although I don't know if they are the ones people are interested in.

to post comments

They are

Posted Apr 7, 2009 20:35 UTC (Tue) by khim (subscriber, #9252) [Link] (2 responses)

Think NGINX. If you have 50'000 clients connected in the same time and your box have 100 separate disks with content (not unrealistic example - NGINX does have support for such extreme conditions) then AIO is suddenly much faster then threaded I/O and MUCH less resource-hungry.

Why does AIO need to be supported at the kernel level?

Posted Apr 8, 2009 1:16 UTC (Wed) by xoddam (subscriber, #2322) [Link] (1 responses)

Even if only operations that *need* to be asynchronous require kernel threads, and even if they are recycled instead of being created and destroyed on demand, that will still be quite expensive at the kernel level.

If one thread per outstanding operation or per client is too many, there are good userspace thread pool implementations that dedicate a few threads to waiting for IO completions whilst others get on with whatever work can proceed immediately.

I'm not convinced that pushing the thread pool down into the kernel is a performance win.

The Linux thread implementation chose for very good reasons to stick to a 1:1 relationship between userspace and kernel threads: it's because the job of multiplexing application tasks to a smaller number of system threads is hard to do in a generic way. All the choices are best made by the application developer, therefore thread pool implementations belong in userspace.

I don't really see the point of supporting POSIX signal-driven AIO at the kernel level if the implementation uses threads and sits on top of the existing synchronous IO. A userspace library could do it just as reliably using select() and kill(), for those few applications that insist on the POSIX AIO interface for whatever reason.

That said, the kernel handles asynchronous events all the time. Why exactly is it so hard to let userspace handle them asynchronously too at a low level, without going through the synchronous layer?

I think the idea is to support AIO for all objects with a single API

Posted Apr 8, 2009 12:46 UTC (Wed) by khim (subscriber, #9252) [Link]

While it's very important to have "light AIO" for some things (like TCP sockets) it's not so important to have them for other things (like in-memory pipes). If you have everything in kernel you can implement some things with threads and other without threads and userspace does not care. With userspace library any change require ABI schange - and that's PAIN...