Advertisement
Front, Kernel, Security, Distributions, Development. See your byline here on LWN.net.
Weekly Edition
Return to the Kernel page
| |||||||||||||
|
| |||||||||||||
Kernel latency hits a new low
Tucked away in Andrew Morton's 2.5.54-mm3
patchset is a new bit of work aimed at reducing the latency of the Linux
kernel. Latency, from the point of view of this work, is the time lag
between when a high-priority process becomes runnable and when it actually
gets the processor. Scheduling latency is important in a number of
contexts, and it can be especially important for desktop users. When you
move your mouse, it is nice not to have to wait until the pointer on the
screen moves to keep up with it. Low latency is crucial for certain
applications, including streaming media recording and playback, CD
recording, data acquisition, and so on. If the system is not sufficiently
responsive, these applications just do not work at all.
The last source of long delays in the kernel, says Andrew, is in the page table teardown code. This delay is easily seen - simply shut down a large application (Mozilla or OpenOffice will do nicely) and try to get anything else done while the cleanup is happening. This delay happens because teardown code holds the process's page_table_lock for the entire cleanup task. If the process is large, the cleanup can take a long time. Since the kernel is holding a lock, it can not be dislodged from the processor even if the kernel is compiled for preemption. So anything else that wants to run has to wait until the whole cleanup job is done. The solution is to create a new "uber-zapper" function (unmap_vmas()) which handles the page table cleanup task. The page range to be torn down is split into smaller chunks (between 256 and 2048 pages, depending on the architecture and kernel config options); between chunks, the lock is dropped and the processor rescheduled if necessary. When the high-priority task has finished doing its thing, the lock is reacquired and the next block of pages is freed. Along with reducing latency, the patch has the additional advantage of cleaning up the separate unmapping code which was duplicated in three different places. The result, it is claimed, is a worst-case scheduling latency of 500 microseconds on a 500 MHz Pentium processor. At least, if you are using the ext2 filesystem and if you are not mounting and unmounting filesystems. That should be fast enough for most users. (Log in to post comments)
Developers denying the problem Posted Jan 9, 2003 12:06 UTC (Thu) by paulsheer (guest, #3925) [Link] Previous kernel releases have terrible latency that both developers and users deny exists. For example the following should be obvious to anyone: a terminal session that has been left idle sometimes takes as long as a full second to respond to a key press.Another example is that a mozilla process can start thrashing your swap space to a point where you have to reboot (or at leasy kill your X session). Users are actually most accepting of this - most will simply tell you that they got around the problem by installing more RAM. These faults are blatently obvious. Yet the current sentiment is that 2.4 is pretty ok, and that the new kernel will yield negligible performance improvement for desktop users. I would just like one prominent kernel developer to admit that all Linux kernels up to and including 2.4 have a serious problem in this regard. (I have yet to try 2.5.)
Developers denying the problem Posted Jan 9, 2003 13:57 UTC (Thu) by RobSeace (subscriber, #4435) [Link] It sounds like you're talking about slow-downs due to swapping/unswappingapps, due to running your system out of free RAM... I think that's always going to have to be a relatively slow process; swap space is inherently slow... I don't think anything mentioned in the above patch would do anything at all to address that... The basic problem is that you don't have enough RAM in your machine for what you want to be doing... So, you end up having to write to and read from a slow hard disk instead of speedy RAM... There just isn't a whole lot a system can do about that, I don't think... The solution is as you've stated: buy yourself some more RAM! Expecting your OS to somehow magically solve the problem of how to cram X+Y MB of data into only X MB RAM, without resorting to slow disk access, is simply asking for a miracle of some sort...
Developers denying the problem Posted Jan 16, 2003 11:49 UTC (Thu) by ketil (guest, #8958) [Link] Okay, so swapping is slow. Yet, a disk seek is a matter of milliseconds, and disk transfer rates are in tens of Mb/s. Half a second would let you swap in 5-10Mb, I think most applications would require a working set of less than that in order to respond to input.I saw somewhere (KernelTraffic?) a measurement of how paging was accessing disk, and the result was that the disk was accessed in a very random pattern. If this is indeed the case, response times can probably be improved a lot by making swapping smarter -- swapping in and out contigous regions of swap instead, for instance.
-- -kzm
Developers denying the problem Posted Jan 16, 2003 18:42 UTC (Thu) by kervel (guest, #9133) [Link] ok, if you are not able to move your mouse (or use other applications) whilethe terminal waits to respond the key, then there is a latency problem. otherwise this has nothing to do with latency. the problem is your xterm has been swapped out (maybe not only your xterm, but relevant part of X-clientside/serverside libraries too, maybe even a couple of megabytes, taking a second to swap in even on a fast computer) there is only one thing you can do about that: preventing your application to swap out. and that can be done by buying more RAM. i think you can tune your vm system to be more agressive on disk-cache, so it will less likely swap out applications (but free cache instead). offcourse, this will lower your disk performance. i'm sorry i can't provide more info (someone who does: i'm also interested)
Developers denying the problem Posted Jan 9, 2003 20:12 UTC (Thu) by xorbe (subscriber, #3165) [Link] "Another example is that a mozilla process can start thrashing your swap space to a point where you have to reboot (or at leasy kill your X session). Users are actually most accepting of this - most will simply tell you that they got around the problem by installing more RAM."Moz will suck down >1GB if you never kill it. Restarting Moz always works for me. That's a problem with Mozilla, not the kernel.
Re: Developers denying the problem Posted Jan 9, 2003 22:21 UTC (Thu) by larryr (guest, #4030) [Link] I think some prominent Linux kernel developers admit the situation exists, but I agree there does not seem to be a sense of urgency about it; I am guessing that is because other kernels do not handle the situation much better. The 2.6 Linux kernel is supposed to have better VM tuning support, but I think specifically what parameters there are is still TBD. [ by the way, this is latency, but not "scheduling latency". ] Larry
Latency and performance not the same thing. Posted Jan 10, 2003 11:39 UTC (Fri) by perlid (guest, #6533) [Link] Well, I think that you must see the difference between a slow system, and low latency. Just because it takes long time compiling big programs, rendering stuff etc. on my system, I won't complain about high latency. Some things use much computing resources, and if you don't have enough, things will go slow.In the case where the system is very slow due to swapping, a high priority task wanting to acquire the processor would probably still do it very fast. Besides, i think that the 2.5 kernels are more responsive in the case of high load, thanks to the reverse-mapping VM, and other things also, I think. However if you are using too much memory, and don't have much RAM, things will always be slow, no matter what kernel you are using.
Developers denying the problem Posted Jan 12, 2003 20:38 UTC (Sun) by iabervon (subscriber, #722) [Link] I have a habit when building machines for myself of not including swap space at all. I've foundthat this actually works pretty well for most use. Unless you are setting up 8 times as much swap as you have RAM or frequently replace machines, it will soon be trivial to replace your swap with RAM, given the rate that affordable memory grows. If you are actually in a position where swap makes sense, it makes at least as much sense to put it on a separate hard drive, so that you can get more swap later without moving the data on your existing hard drive; otherwise, upgrades will leave you in a situation where your swap is a trivial amount of storage, and anything that uses all of your RAM will just cause stuff to get swapped out and then run out. In order to size the swap space for a new computer, take the size of RAM, buy that much more RAM, and turn off swap. You'll never have unresponsive terminals again, and your machine won't run out of memory any sooner (or rather, it will have the same total amount. If it runs out, it won't slow down so much halfway through).
Kernel latency hits a new low Posted Jan 9, 2003 18:11 UTC (Thu) by chohman (guest, #5519) [Link] Ummm .... Paul is, at best, misinformed.Kernel latency is (no suprise) as described in the article. Paul is simply complaining about the fact that hard drives are slower than RAM at supplying data. Well, they are, always have been, always will be - perhaps 2 or 3 orders of magnitude slower. I've been computing for about 24 years now, and whenever a system starts swapping (or Checkpointing, or using Virtual Memory, or ...) performance takes a dive. The solution, as always, is add more RAM or run less code. To imply that this constitutes some great cover up or failing by the kernel developers is just foolish. Beware the pseudo-experts, for they are legion.
Kernel latency hits a new low Posted Jan 16, 2003 13:36 UTC (Thu) by paulsheer (guest, #3925) [Link] Ummm .... Paul is, at best, misinformedMisinformed about the fact that an idle terminal takes a full second to respond to keyboard input? Or misinformed about the fact that FreeBSD does NOT have the problem of runnaway processes making the system unusable? You are in denial
Kernel latency hits a new low Posted Jan 10, 2003 0:57 UTC (Fri) by cyanide (guest, #2236) [Link] Interesting to see how the kernel affects the desktop user experience.Just a thought, how well does windows deal with cleanup?
Kernel latency hits a new low Posted Jan 21, 2003 18:10 UTC (Tue) by Leknor (guest, #3359) [Link] > how well does windows deal with cleanup?I've read that the windows kernel have an understaning of which application is in the foreground. When you switch focus between applications/windows the kernel will make an effort to improve responsiveness of that app. For example the scheduling priority of that program will be increased. I don't know if Windows preloads the app from the swap for the user.
|
|
||||||||||||