|
|
Log in / Subscribe / Register

The rapid growth of io_uring

The rapid growth of io_uring

Posted Nov 3, 2023 22:19 UTC (Fri) by leo60228 (guest, #167812)
In reply to: The rapid growth of io_uring by farnz
Parent article: The rapid growth of io_uring

I tried implementing that. What I mean is that, for example, if you were trying to copy 0x8000 byte chunks at a time, and the first SQE splicing from the input file to the pipe only copied 0x4000 bytes, the linked SQE would still try to read 0x8000 bytes from the pipe and potentially get the wrong data. Additionally, I'm not sure there's a guarantee that these linked SQEs would be atomic (i.e. if SQE A was linked to SQE B, and SQE C was linked to SQE D, the order A, C, B, D could be allowed and result in data being written to the wrong offsets in the output).

Thinking about it more, I suppose this could be solved by creating a large number of pipes, and making sure that no two SQEs using the same pipe are in flight at the same time. I'm concerned that having many pipes could result in its own performance issues, but it'd probably be fine...?


to post comments

The rapid growth of io_uring

Posted Nov 5, 2023 10:12 UTC (Sun) by farnz (subscriber, #17727) [Link]

You don't have one pipe for all splices; you have one pipe per splice. If the first SQE copies 0x4000 bytes, then the linked SQE can only copy 0x4000 bytes out of the pipe, because there's only 0x4000 bytes in there to copy out. This is exactly what the kernel trick is - create a temporary pipe for the splice to use, so that you're always splicing in and out of pipes.


Copyright © 2026, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds