|
|
Subscribe / Log in / New account

Gathering multiple system parameters in a single call

Gathering multiple system parameters in a single call

Posted Apr 7, 2022 1:42 UTC (Thu) by calumapplepie (guest, #143655)
Parent article: Gathering multiple system parameters in a single call

> But I really don't think this is something we want to do unless people can show enormous and real-world examples of where it makes such a huge difference that we absolutely have to do it.

htop -d 10 , which gives me a nice, reliable view of what my system is doing, consumes 12-15% CPU. htop -d 1, which gives a great picture of what my computer is up to that moment, goes up to 50-60% CPU. 12-15% may not sound like a lot, but its the highest or 2nd highest consumer most of the time. This is on a system with two web browsers with well over 100 tabs open, as well as several other apps (some in python) running nearly constantly. Having my performance monitoring tool eat up more CPU than Firefox's 117 tabs in 3 windows is a little bit sad.

> He noted that an lsof on his laptop did not take an inordinate amount of time, so the performance argument does not really make sense to him.

I should be clear that the machine I am writing this on (and running these tests on) is running in battery mode: cpu frequencies are locked at 800 mHz, everything in "extreme power save mode", etc. However, despite running this command thrice to ensure everything was nice and warm in the caches, I still got these results:

time lsof > /dev/null
real 0m29.248s
user 0m16.245s
sys 0m12.842s

This might not be 'inordinate', but it's certainly significant. Even if we assume that every last moment of the 12 seconds spent on system calls is vitally needed and cannot be optimized by this new call, what exactly is being done during the 16 seconds in userspace? How much of that time amounts to "parse what the kernel gave us"?

This isn't really a matter of user-facing performance for me: rather, it's a question of power efficiency. If I leave htop running in the background for a while, that shouldn't harm my battery life substantially. Performance monitoring tools in userspace are very common: think of how many there are running in the world right now. If we make all of them twice as fast, we will save megawatts of power. Using the standard "oh, but tools won't use it at first" argument seems pretty silly to me: tools can detect the availability of system calls, and further, that argument applies to every possible bit of functionality: why aren't we hearing that objection to argue that the performance gains of, say, io-uring are irrelevant? As for attack surface, this interface is very easy to fuzz, and doesn't need to be sophisticated or heavily optimized. Compared to what it's replacing (a confused mix of interfaces developed over decades), I think it's very probable that the reduced attack surface in 10 years when distros can start configuring out the IOCTLs far exceeds that added by this interface.


to post comments

Gathering multiple system parameters in a single call

Posted Apr 7, 2022 2:13 UTC (Thu) by anonymous_commenter (guest, #117657) [Link]

Amen to your statistics, that reminds me of some articles on disk failure rate that Valerie Aurora worked on (which one, don’t remember exactly but here’s the list: https://lwn.net/Archives/GuestIndex/#Aurora_Henson_Valerie ) and also, process monitors on one of my computers (a laptop).

Gathering multiple system parameters in a single call

Posted Apr 7, 2022 10:37 UTC (Thu) by Kamiccolo (subscriber, #95159) [Link] (3 responses)

$ time lsof > /dev/null

real 0m12.425s
user 0m1.496s
sys 0m1.544s

oh well...

Gathering multiple system parameters in a single call

Posted Apr 7, 2022 21:03 UTC (Thu) by calumapplepie (guest, #143655) [Link] (2 responses)

See the wild difference between realtime and usertime + systime? Is your system under heavy CPU load, or is the number of syscalls required for lsof causing the scheduler to keep sleeping the process uneccessarily?

Gathering multiple system parameters in a single call

Posted Apr 8, 2022 5:45 UTC (Fri) by jezuch (subscriber, #52988) [Link] (1 responses)

I think the difference is because lsof resolves hostnames by default. I got into a habit of running lsof -n for that reason.

Gathering multiple system parameters in a single call

Posted Apr 8, 2022 23:42 UTC (Fri) by HenrikH (subscriber, #31152) [Link]

I think you are quite correct:

henrik@Sineya:~$ time lsof > /dev/null
real 0m8,932s
user 0m1,899s
sys 0m1,497s

henrik@Sineya:~$ time lsof -n > /dev/null
real 0m3,276s
user 0m1,834s
sys 0m1,387s

Gathering multiple system parameters in a single call

Posted Apr 7, 2022 14:21 UTC (Thu) by Hello71 (subscriber, #103412) [Link] (1 responses)

> cpu frequencies are locked at 800 mHz

have you checked that this actually reduces battery usage? it has been the goal of Linux schedulers for quite some time to use the fastest reasonably-efficient speed available to complete a given task, then drop to a deep power-saving level. "off" uses so much less power than "800 mHz" (sic) that it saves energy significantly.

Gathering multiple system parameters in a single call

Posted Apr 7, 2022 21:01 UTC (Thu) by calumapplepie (guest, #143655) [Link]

I didn't actually 'lock' them to that frequency: just the cpufreq governor to maximize power saving.

I should note that I think the schedulers could use some fine-tuning on this, actually: I understand there is a latency (and accompanying power consumption) going from (say) C10 to C0, so it'd make sense to completely stop using some cores, but that isn't what happens, based on what I see in Powertop: every core sees some use. Of course, I don't actually know what the most efficent thing is: just an observation from my machine.

Gathering multiple system parameters in a single call

Posted Apr 13, 2022 15:06 UTC (Wed) by metan (subscriber, #74107) [Link] (1 responses)

I did have a look at the lsof code when this patchset has been discussed and as far as I can tell significant part of the overhead is caused by the fact that it uses FILE interface to parse single value proc files instead of much simpler open()/read()/close(). I bet that we can make it twice as fast without any kernel changes...

Gathering multiple system parameters in a single call

Posted Apr 15, 2022 14:55 UTC (Fri) by mrugiero (guest, #153040) [Link]

> I did have a look at the lsof code when this patchset has been discussed and as far as I can tell significant part of the overhead is caused by the fact that it uses FILE interface to parse single value proc files instead of much simpler open()/read()/close(). I bet that we can make it twice as fast without any kernel changes...

Considering it already needs OS specific calls, that's certainly some low-hanging fruit. Even if it didn't, the odd one out in terms of having open/read/close as first class citizens is Windows, and I'm not even sure lsof runs there. I think that's a good catch :)


Copyright © 2025, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds