LWN: Comments on "Ringing in a new asynchronous I/O API"

Guidance on using io_uring to support 60,000+ TCP connections with <1ms RTT

Tushar — Thu, 09 Nov 2023 02:30:21 +0000

Hi,

I am working on building a new application which is required to support 60,000+ tcp connections on a single server (preferably a single POSIX thread) with <1ms RTT. I am considering using io_uring for this.

I have not found any data for similar application of io_uring for other applications. Do you have some benchmark that I might refer to to see if something like this may be possible.

Thanks for your time and help.

Ringing in a new asynchronous I/O API

crzbear — Fri, 17 May 2019 19:30:26 +0000

this sounds awesome

is there any particular reason the kernel has to allocate those buffers
couldn't they be passed from userspace in the setup call
and then the kernel maps those into its address space

while this might obviously lead to not properly aligned buffers,
the kernel can check that and return with an error if needed

this would do away with the mmapping

How to register more files while using some registered files?

hnakamur — Sat, 11 May 2019 10:35:55 +0000

My understanding is you can register new set of file descriptors after unregistering all of old ones.

io_uring_register(ring_fd, IORING_REGISTER_FILES, fds, nr_files);
io_uring_register(ring_fd, IORING_UNREGISTER_FILES);
io_uring_register(ring_fd, IORING_REGISTER_FILES, fds2, nr_files2);

But, what to do if you want to add some more file descriptors while using some of already registered
file descriptors?

What about async metadata

josh — Tue, 26 Feb 2019 01:53:49 +0000

> It would be convenient to have a system call that declares 'I plan to read this file in the near future'.

The readahead system call does that.

Ringing in a new asynchronous I/O API

Wol — Thu, 24 Jan 2019 16:09:29 +0000

Can't you stick a "t" in there? uring -> turing

Cheers,
Wol

Ringing in a new asynchronous I/O API

joib — Thu, 24 Jan 2019 12:37:23 +0000

For inspiration in the IO world, there's IOCP (https://en.wikipedia.org/wiki/Input/output_completion_port ).

Ringing in a new asynchronous I/O API

joib — Thu, 24 Jan 2019 12:35:32 +0000

What is the mechanics of doing buffered AIO? I faintly recall all those old attempts (syslets, fibrils, whatever) failed partly because there was no natural in-kernel context for progressing those IO's? An in-kernel threadpool is of course always a possibility, but is that noticeably better than a user-space threadpool?

What about async metadata

epa — Tue, 22 Jan 2019 12:38:12 +0000

You mentioned posix_fadvise(). That is useful but not quite the stupidly simple interface I had in mind. It requires an open file handle. I envisaged a call that takes a filename and nothing else, works entirely in the background, and does not fail (not even if the file doesn't exist or whatever; it just does nothing in that case).

You could then sprinkle these calls all over your code -- including scripting languages -- and get a handy speedup without having to do any real programming.

What about async metadata

epa — Tue, 22 Jan 2019 12:22:51 +0000

Yes, I was thinking of a few large files, where the overhead really is in I/O and not in bookkeeping.

How about a generalized stat() that lets you open a directory and get info on all the files it contains? That would save a lot of time, and not just for parallel code. Network filesystems, for example.

What about async metadata

dw — Tue, 22 Jan 2019 11:35:37 +0000

Isn't this basically what posix_fadvise() gives us already? But IIRC that interface currently or previously blocked while readahead happened.

For zipping, imagine something like a 100k item maildir of tiny 1.5kb messages. While the compression is still relatively expensive, a huge chunk of the operation will be wasted on ceremonial serialized filesystem round-trips (open/close/read/stat/getdents/etc). To avoid that I'm not sure there is any way around it except a whole bunch of threads keeping as many FS operations in flight (either doing the CPU bits or any IO bits for uncached data) to get even close to a genuinely busy computer.

What about async metadata

epa — Tue, 22 Jan 2019 10:09:58 +0000

It would be convenient to have a system call that declares 'I plan to read this file in the near future'. The kernel would make a best effort to get that file into the page cache, using background I/O, while your process continues. So if you are about to zip up a directory, call plan_to_read() on each file, then continue reading them sequentially as normal. It wouldn't be quite as fast as a true parallel implementation, but for some tasks it could give you 80% of the performance gains without having to rewrite your creaky old sequential code.

Ringing in a new asynchronous I/O API

zse — Sun, 20 Jan 2019 13:22:51 +0000

Nice that we're finally getting a decent AIO option.

I haven't found the complete list of opcodes that are proposed, so don't know if this is already in the works, but I'd think you'll also need synchronization primitives (e.g. a barrier so that all io ops before it need to complete before those after the barrier can start).

In general this proposal kind of reminds me of the command queues you have for graphics hardware (OpenGL/Vulkan). I'm wondering if there is potential for (partial) unification or at least mutual inspiration...

Ringing in a new asynchronous I/O API

nix — Sun, 20 Jan 2019 10:20:27 +0000

Yeah, exactly. I was confused despite being sure I would be confused and checking the parent article before posting (my fault, not the article's).

Ringing in a new asynchronous I/O API

zdzichu — Sat, 19 Jan 2019 20:27:13 +0000

sqe and cqe are for “submit” and“completion” queues. Your confusion demonstrates the need for less brief identificators :)

Ringing in a new asynchronous I/O API

nix — Sat, 19 Jan 2019 19:00:22 +0000

Ooh that's honestly nicer than plain read()/write() would be. No -EINTR worries, no short reads, the only bit that makes me squint is the cqe/sqe acronyms, which are perhaps too concise to be easily understandable (it's a bit longer, but maybe _cmd and _stat suffixes would be easier to read? You're really only granting yourself one variable letter is the existing scheme, which is... a small naming budget.)

Ringing in a new asynchronous I/O API

nix — Sat, 19 Jan 2019 18:57:40 +0000

We're in "passing massive structures and/or arrays of structures" land, and trying to *not* produce a horror show like perf_event_open() is fairly high on my priority list! (Though, really, arguing against myself: the reason perf_event_open() is horrifying is that it does a lot, and honestly the same would be true of any attempt to wrap it in a uring, or in anything else. It's not really a win to move from 'this ioctl/syscall has security holes because the interface has too many edges to test exhaustively' to 'this uringed interface has security holes because the interface has too many edges to test exhaustively', alas. :( )

Ringing in a new asynchronous I/O API

axboe — Fri, 18 Jan 2019 17:22:56 +0000

It's the brain making that leap, since uring isn't a word it currently recognizes. This will go away as it becomes a bit more ubiquitous. I see no reason to change the name.

Ringing in a new asynchronous I/O API

zdzichu — Fri, 18 Jan 2019 12:06:37 +0000

Not only yours.

Ringing in a new asynchronous I/O API

NAR — Fri, 18 Jan 2019 10:36:45 +0000

My first association/misread was uring - urine and that's not that nice name. Maybe I just need new glasses.

Ringing in a new asynchronous I/O API

axboe — Fri, 18 Jan 2019 04:21:00 +0000

Might even makes sense to just move away from an ioctl, and provide an sqe entry into the driver through the file_operations, for instance.

Ringing in a new asynchronous I/O API

axboe — Thu, 17 Jan 2019 23:34:55 +0000

Since it already has all the mechanics to do buffered IO async, any ioctl could easily be channeled through the API. We're already grabbing the files/mm of the original process. Depending on how many arguments you need to the ioctl, we'd need to tweak the sqe a bit.

With liburing, it should be _very_ easy for applications to use. If you go native, yes, you need to be a bit more careful, and it's more hairy. But even with the basic support liburing has now, you just do:

{
struct io_uring ring;
struct io_uring_sqe *sqe;
struct io_uring_cqe *cqe;

io_uring_queue_init(queue_depth, &ring, 0);

sqe = io_uring_get_sqe(&ring);
sqe->opcode = IORING_OP_READV;
sqe->fd = fd;
[...]

io_uring_submit(&ring);

io_uring_wait_completion(&ring, &cqe);
}

as a very basic example.

Ringing in a new asynchronous I/O API

axboe — Thu, 17 Jan 2019 23:30:06 +0000

Don't give away all my secrets :-)

Ringing in a new asynchronous I/O API

nix — Thu, 17 Jan 2019 23:16:51 +0000

This is such a nice interface, I'm wondering if heavy ioctl() users might be able to reuse some of these ideas to redo their ioctl madness into a nice high-performance command/response ring :) I mean yeah it's not exactly simple for userspace to use, but if the complexity is wrapped away from users its properties are seriously slobbersome.

(yes, I help maintain one of those monsters, making heavy use of ioctl() passing intricate structures into and out of the kernel, and the massive use of ioctl() is one thing I at least am hoping to get rid of in the process of getting it ready for upstreaming.)

Ringing in a new asynchronous I/O API

nix — Thu, 17 Jan 2019 23:12:46 +0000

Nice library name. The plan is to lure people into using it, right? :)

Ringing in a new asynchronous I/O API

axboe — Thu, 17 Jan 2019 16:26:52 +0000

Just to follow up on this, if you check the latest repo, the ring_fd is now both pollable (in terms of poll(2)/epoll(2), not io-pollable), and io_uring also supports IORING_OP_POLL to offer the same functionality that aio does in that regard.

I believe this caters to both of your needs.

What about async metadata

dw — Thu, 17 Jan 2019 12:33:09 +0000

There's always newer and better technology around, but tech is only useful when it's compatible with what you already have :) And ZIPs are eeeverywhere

What about async metadata

Sesse — Thu, 17 Jan 2019 12:30:57 +0000

So you want to demonstrate that something is obsolete by implementing… an obsolete compression algorithm? :-)

(zlib/deflate is still around pretty much only due to huge transition costs, and a fragmented market among the alternatives. Try something like zstd if you want to make a clean break.)

Physically-contiguous buffers

nilsmeyer — Thu, 17 Jan 2019 09:45:33 +0000

And you could potentially allocate the buffers as huge pages, right?

Ringing in a new asynchronous I/O API

axboe — Thu, 17 Jan 2019 03:28:30 +0000

I'm addressing what is a misconception in the original comment. It referred to the "bored" part, which is when you have the kernel side doing polling for you. That has nothing to do with epoll(), and epoll() would not be a solution for this at all.

If the ring_fd should be pollable, in terms of epoll, absolutely. That would be trivial to add. It would NOT work for IORING_SETUP_IOPOLL for obvious reasons, as you can't sleep for those kinds of completions. But for "normal", IRQ driven IO, adding epoll() support for the CQ side of the ring_fd is straight forward. On the SQ ring side, there's nothing to epoll for. The application knows if the ring is writeable (eg can hold new entries) without entering the kernel.

Outside of that, my IOCB_CMD_POLL reference has to do with this:

https://lwn.net/Articles/743714/

and adding IORING_OP_POLL for similar functionality on the io_uring side.

Ringing in a new asynchronous I/O API

samroberts — Thu, 17 Jan 2019 03:12:09 +0000

I think you can forgive the OP for calling what epoll() does "polling", though it doesn't fit your definition, given its the name of the syscall!

The point stands: io_uring should be easily useable with poll/select/epoll so it can be integrated with existing event loop based code, networking code in particular are heavy users of these calls. Specifically, this fd

> The return value from io_uring_setup() is a file descriptor that can then be passed to mmap() to map the buffer into the process's address space.

should be epoll()able.

What about async metadata

dw — Thu, 17 Jan 2019 01:03:19 +0000

I have it on my todo list to write a fully CPU/IO-parallel ZIP implementation (because it's fairly straightforward), with an article around it highlighting most of the traditional UNIX tooling is utterly obsolete on pretty much all modern devices. Naturally it can't really benefit from the work here due to the parent comment, but yeah, the problem is very real, and frankly an entirely ridiculous state of affairs

Physically-contiguous buffers

ms-tg — Wed, 16 Jan 2019 23:31:19 +0000

This sort of comment by the patch author is a major example of the value of LWN

Ringing in a new asynchronous I/O API

HIGHGuY — Wed, 16 Jan 2019 20:09:44 +0000

Would the interface support operations that require 2 file descriptors, like splicing from a file to a socket and vice versa? I have the impression it doesn’t but could have easily missed something.

Slightly more complex operations could be useful in combination with primitives like the P2P PCIe transfers that are being worked on to avoid going through main memory altogether.

Ringing in a new asynchronous I/O API

axboe — Wed, 16 Jan 2019 19:30:56 +0000

It is, check the git repo!

Ringing in a new asynchronous I/O API

arjan — Wed, 16 Jan 2019 19:24:52 +0000

void *addr; /* buffer or iovecs */

hmm that makes 32/64 compat funky.. wonder if it really should just be a u64

Ringing in a new asynchronous I/O API

axboe — Wed, 16 Jan 2019 18:07:21 +0000

io_uring will grow support for IORING_OP_POLL, but it's outside the scope of the initial implementation. This will work similarly to the recently added IOCB_CMD_POLL support for aio.

Apart from that, I do think you're mixing up the polling with the io polling. One provides a way to signal when data is ready, the other skips IRQs in favor of busy polling for completion events.

Ringing in a new asynchronous I/O API

axboe — Wed, 16 Jan 2019 18:05:21 +0000

System calls should go into libc, but anything else will reside in liburing. You can clone that here:

git://git.kernel.dk/liburing

though not a lot of items are in there yet. It does contain helpers to setup/teardown the ring, and submit/complete helpers for applications that don't want (or need) to muck with the ring itself. This will grow some more features, the intent is that most applications will _probably_ end up using that instead of handling all the details themselves.

Physically-contiguous buffers

axboe — Wed, 16 Jan 2019 18:03:46 +0000

Not only that, but you could also pre-map the SG lists, instead of having to do map SG and unmap SG for each IO. Right now the registered buffers only avoid the get_user_pages() and put_pages() for each IO, which is (by far) the biggest overhead. But if we fix the kernel parts as well, then we can avoid the dma map/unmap for each IO. That'd bypass the split as well, some quick mental math shows we should be able to kill ~5% of the overhead on my box with that.

In general we have various pieces of low hanging fruit on the block layer side, which are readily apparent now that we have an efficient interface into the kernel. Work in progress! But I'd like to wrap up io_uring first.

Ringing in a new asynchronous I/O API

nix — Wed, 16 Jan 2019 16:19:35 +0000

You can just leave it as a separate library, like libaio is now, and libattr, and libacl, and many others. (Though if it's useful and glibc wants to use it itself, it's not unimaginable that it might find its way in there in time -- but glibc has harsh backward-compatibility constraints that argue in favour of a trial period in an external library in any case, until we know whether the API the library provides works well.)

Ringing in a new asynchronous I/O API

me@jasonclinton.com — Wed, 16 Jan 2019 16:15:56 +0000

> It's perhaps worth noting at this point that Axboe is working on a user-space library that will hide much of the complexity of this interface from most users.

What's the procedure for a user-space library tightly coupled to a kernel API, like this one, getting into glibc (or any of the rest of the libcs)?