User: Password:
Subscribe / Log in / New account

A SystemTap update

A SystemTap update

Posted Jan 29, 2009 10:08 UTC (Thu) by ctg (guest, #3459)
In reply to: A SystemTap update by eugeniy
Parent article: A SystemTap update

Me and a colleague were discussing just last night the thorny problem of how we work out which of many competing processes are using disk access - causing contention on the disk, so that everything queues up, and more and more disk access is caused... not really a lot of tools in linux todo that... (If we knew the worst offenders, then we could focus our effort on making them more efficient - having to put instrumentation in each process is really time consuming).

.. so reading this article was timely. Looks like systemtap would enable us to quickly home in on the big disk users..

.. the article quite clearly states that to get the best out of systemtap you need these patches, so when Mr Morton himself makes this sort of criticism, then its a bit of a concern.

Despite all that, I'm off to look at systemtap in a bit more detail (it's lack of ubiquity has put me off before), but the lack of decent tools for working out what is really going on in a complex system is pretty frustrating (I'm still suffering from the lack of the "W" flag in the output of ps(1) to show which processes are swapped out - I understand why it doesn't show that any more - but when your system goes into swap, it's useful to see which processes are being paged out.. I suspect systemtap might be able to help with this too).

(Log in to post comments)

A SystemTap update

Posted Jan 29, 2009 10:42 UTC (Thu) by mjw (subscriber, #16740) [Link]

The vfs tapset example in the article works without needing any additional user space hook patches.

Also take a look at some of the examples that come with Systemtap. disktop.stp probably does what you want:

A SystemTap update

Posted Jan 29, 2009 17:05 UTC (Thu) by knobunc (subscriber, #4678) [Link]

See also iotop.

A SystemTap update

Posted Jan 29, 2009 20:07 UTC (Thu) by epb205 (guest, #50182) [Link]

Why doesn't ps show which processes are swapped out anymore? Is that somehow a security hole?

A SystemTap update

Posted Feb 3, 2009 22:33 UTC (Tue) by oak (guest, #2786) [Link]

No idea, but you can get the same information also from /proc/PID/smaps.
It's separate for each of the memory mapping the process has i.e. you may
need to write a small script to process the data.

If the process has stuff that's marked as swapped, but not anymore as
dirty, it's completely swapped out. For some reason kernel/SMAPS doesn't
think swapped pages to be anymore dirty which loses the distinction
between shared dirty and private dirty that SMAPS shows for pages still in

A SystemTap update

Posted Jan 30, 2009 3:15 UTC (Fri) by SEJeff (subscriber, #51588) [Link]

Actually the block_dump feature of modern 2.6 linux kernels will show you which processes are writing to which devices. I wrote a proof of concept script to show them:

The output looks like this:
root@desktopmonster:~# ./top-disk-users
banshee-1 23999 8 READ sda9
kjournald 2494 131 WRITE sda5
kjournald 5182 5 WRITE sda8
pdflush 228 15 WRITE sda5
pdflush 228 1 WRITE sda8
pdflush 228 32 WRITE sda9

Copyright © 2017, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds