Improving fget() performance [LWN.net]

Improving fget() performance

Posted May 6, 2019 21:36 UTC (Mon) by roc (subscriber, #30627)
Parent article: Improving fget() performance

A new open() (or accept4() etc) flag could opt out of the sequential file descriptor allocation regime to allow a more performant allocation scheme.

to post comments

Improving fget() performance

Posted May 6, 2019 22:27 UTC (Mon) by djwatson24 (guest, #74976) [Link] (5 responses)

One complication is that often one thread accepts, but then is load balanced to another thread for most of its lifetime. So it may be necessary to somehow specify or change the fd range at a time later than accept() or open().

Improving fget() performance

Posted May 6, 2019 23:15 UTC (Mon) by Cyberax (✭ supporter ✭, #52523) [Link]

Do a dup2() call?

Improving fget() performance

Posted May 7, 2019 6:40 UTC (Tue) by smurf (subscriber, #17840) [Link] (3 responses)

I'd suggest that the only tasks where sequential allocation matters is when you close fd 0…2 right before dup()ing, and even that is highly unreliable in a threaded environment (thread 1 closes stdout, thread 2 calls accept(), thread 1 dup()s … oops – should have used dup2() …).

Thus, sequential allocation doesn't need a flag or its absence, it only needs to check that the refcount of the fd table is ==1.

Improving fget() performance

Posted May 7, 2019 17:24 UTC (Tue) by willy (subscriber, #9762) [Link] (2 responses)

It is mandated POSIX behaviour though, so we usually like to have a way for applications to opt-in to non-POSIX behaviour, be it a prctl, an fcntl or something else.

Let's see what the dup2() experiment buys us, then we can discuss how to implement it (if it is a win)

Improving fget() performance

Posted May 7, 2019 19:55 UTC (Tue) by abatters (✭ supporter ✭, #6932) [Link] (1 responses)

Instead of using non-sequential fds, how about indexing the table differently by shuffling the bits of the fd around before using it as an index, so that sequential fds don't share a cacheline?

Improving fget() performance

Posted May 7, 2019 20:11 UTC (Tue) by willy (subscriber, #9762) [Link]

Nice idea. Unfortunately, the problem is not that fds are assigned to threads in some kind of round-robin order. They're assigned to threads in a semi-random order in an attempt to keep worker threads equally busy. Adding in a shuffle isn't going to help.

Improving fget() performance

Posted May 6, 2019 23:50 UTC (Mon) by neilbrown (subscriber, #359) [Link] (1 responses)

> A new open() (or accept4() etc) flag could opt out of the sequential file descriptor allocation regime to allow a more performant allocation scheme.

FYI: https://lwn.net/Articles/236843/

Improving fget() performance

Posted May 7, 2019 4:21 UTC (Tue) by roc (subscriber, #30627) [Link]

The objection there about flags not being available on relevant syscalls seems to have become obsolete.

Improving fget() performance

Posted May 7, 2019 16:55 UTC (Tue) by rweikusat2 (subscriber, #117920) [Link]

As usual, there's no need to break working code. Threads of control may run in a shared address space without sharing a file descriptor table. Granted, this means explicit file descriptor passing would have to be used to bounce jobs among threads for the joy of delay but as people who do that obviously don't have any performance requirements (the c10k page is really old by now), this shouldn't matter.