|
|
Log in / Subscribe / Register

BFS vs. mainline scheduler benchmarks and measurements

BFS vs. mainline scheduler benchmarks and measurements

Posted Sep 7, 2009 3:53 UTC (Mon) by Tracey (guest, #30515)
Parent article: BFS vs. mainline scheduler benchmarks and measurements

Ingo's message seemed very polite considering the history of all of this. Ingo also took time to test and post the outcome of the tests, something the kernel devs seem to demand now-a-days more patches.

I have a lot of respect for both Con and Ingo, and hope that something constructive happens from this.

I test and use kernels that are built using the real time patches from Ingo and Thomas. The primary reason for my interest is to get better multimedia response on desktop machines. This is something that Con seems to be trying to fix.

Ignoring the history of all of this, it seems that all of these people are working towards the same goals. I'm hoping that everyone involved can work together on this and not create the friction that the press likes to feed off so well.

My view is that if the kernel needs more then one scheduler to optimize it for both the desktop and server, then it should be done. I feel that there is a bit too much stubbornness on everyone's part on this.

On the other hand, I've been using the real time patched kernels for years and know that they slowly make it into the mainline kernel; so maybe in the end a separate, low latency desktop scheduler won't be needed.

I just hope that these parties can work together, share ideas and code, and acknowledge what each is contributing.

A part of me feels that in the future will see most if not all of the real time tree will make it into the kernel and the need for a separate desktop scheduler won't be needed. If that happens, I hope that Con's work and ideas that are incorporated into the kernel are acknowledged with all do respect.

I hope that Ingo's offering an olive branch on here.

to post comments

BFS vs. mainline scheduler benchmarks and measurements

Posted Sep 7, 2009 4:34 UTC (Mon) by nash (guest, #50334) [Link] (41 responses)

If you follow up Con's response... I don't think we'll see any constructive discussions going on here. Which is a shame.

http://thread.gmane.org/gmane.linux.kernel/886319

BFS vs. mainline scheduler benchmarks and measurements

Posted Sep 7, 2009 4:52 UTC (Mon) by flewellyn (subscriber, #5047) [Link] (29 responses)

I begin to understand why Con's prior work was not included in mainline...

BFS vs. mainline scheduler benchmarks and measurements

Posted Sep 7, 2009 5:08 UTC (Mon) by tajyrink (subscriber, #2750) [Link] (27 responses)

I think Con's response was precisely reiterating what he was already accusing kernel devs about - everything has to scale. And additionally using things like compiling kernel and piping messages as "benchmarks".

How about perceived user experience blind tests when using Firefox on a netbook? (because it's hard to benchmark responsiveness to user interaction vs. completion time)

I think there would be demand for a new tool that tracks any user interaction with the time it takes to have a proper response. Probably not really generally doable, but a set of benchmarks that can be used to test this would be beneficial.

BFS vs. mainline scheduler benchmarks and measurements

Posted Sep 7, 2009 5:32 UTC (Mon) by flewellyn (subscriber, #5047) [Link] (26 responses)

All good points, but Con's attitude was terrible. Ingo's really not, I don't think, trying to drive him away, or invade his space, or anything. Just trying to work with him.

BFS vs. mainline scheduler benchmarks and measurements

Posted Sep 7, 2009 6:31 UTC (Mon) by nash (guest, #50334) [Link] (24 responses)

To be fair on Con, you could argue there was a bit of trolling going on Ingo's side, with the choice of hardware and the like.

However it was a troll with real numbers associated.

BFS vs. mainline scheduler benchmarks and measurements

Posted Sep 7, 2009 6:38 UTC (Mon) by flewellyn (subscriber, #5047) [Link] (23 responses)

A dual quad-core system with hyperthreading? That's hardly a bad choice of hardware for the "desktop system" scale. That's the standard CPU setup for a Mac Pro, for instance.

Granted, it's the upper end of the "desktop system" scale, but he did say as much.

BFS vs. mainline scheduler benchmarks and measurements

Posted Sep 7, 2009 7:51 UTC (Mon) by drag (guest, #31333) [Link] (20 responses)

There is not one single person I know that owns a dual-socket desktop.

It's just not a desktop machine. There is a very good reason for this... it dramatically increases the cost of the board and doubling the cost of cpu for pretty much no good reason. It's not going to benefit you in any way for surfing the internet or playing games or even processing media.

The only people who would benefit from a system like that is for compiling software, long render batch jobs, and the like. That is just not a typical desktop workload.

The mainstream desktop system is very obvious to me.

Core2Duo Intel laptop, Dual core AMD desktop, single core Atom processor. Those are the cpus that your going to see on a typical Linux system.

I know lots of people that P4 machines, a few people still using P3 laptops, a bunch of Core2Duo laptops, and a bunch of people owning netbooks for various reasons (high mobility, secondary computer, regular laptops are too expensive, etc).

Dual-socket Quad-core systems? That's just not the target audiance for the most part.

----

That being said I don't think it would make a big deal. Ingor's testing is probably going to reflect accurate performance for machines less powerful then that one. But I can't be sure about that. It would of really had more impact if the tests were carried out on a dual core machine.

That and the point of the BFS is to make things more friendly and more interactive. That is hard to benchmark and having something that very responsive to user input would probably be slightly less efficient overall even though users would actually prefer it.

BFS vs. mainline scheduler benchmarks and measurements

Posted Sep 7, 2009 7:58 UTC (Mon) by dlang (guest, #313) [Link] (15 responses)

today Intel is selling single socket systems with 6 real cores + hyperthreading (simulating 12 cores)

they have in their roadmap to be selling single socket systems with 8 real cores in less than a year.

so what today is a two socket 'business only' system is next summer's (or next christmas') power user system

just like about a year ago the only people with 8 cores were the high-end 4 socket systems, and the only people with 4 cores were dual socket server systems.

nowdays it's common for single socket systems to have 4 cores (+ hyperthreading)

Yes he did pick a system at the high end of that BFS claims to support (and I would like to see how it fares with 4 or so cores in use), but at the same time, the benchmark numbers weren't a matter of a couple percentage points of difference, on one benchmark the time went from < 4 seconds to > 40 seconds, 10x worse.

that doesn't mean that BFS is junk, just that it's not finished, but utterly dismissing (and ignoring) the results is not a good start for discussions.

BFS vs. mainline scheduler benchmarks and measurements

Posted Sep 7, 2009 8:15 UTC (Mon) by kragil (guest, #34373) [Link] (10 responses)

Most of the machines in the real world are single cores.

Most of the machines sold today are dual cores ( real or only with HT like Atom ).

Most people still don't buy big desktops with quad-cores, they buy cheap laptops/netbooks.

It will take a long time before most computers sold will have more than 16 cores as the computers that were sold the last 4 years are perfectly capable of doing everything a non-gamer/non-kernel dev needs.

BFS vs. mainline scheduler benchmarks and measurements

Posted Sep 7, 2009 8:44 UTC (Mon) by dlang (guest, #313) [Link] (1 responses)

actually, if you want to start playing the 'most of the computers are.. ' games

most of the computers in use in the world are 8 bit cpu's

most of the new computers sold each year are _still_ 8 bit cpu's (by a smaller margin than in prior years, true, but stil the winner)

so by that argument, both linux and windows are completely irrelevant since neither of them will run on themajority of computers around or being sold.

what Con should have done was to respond that 16 (simulated) cores is too many for the current stage of BFS code, and told Ingo that with X cores it is still solidly in it's sweet spot. Ingo could then go back and run the tests again to see what results he gets.

if with 4 cores his benchmarks still show the machine completely locking up, Con would then need to look at BFS to see why it's so bad for some workloads (which is exactly what he lambastes the kernel scheduler for)

BFS vs. mainline scheduler benchmarks and measurements

Posted Sep 7, 2009 9:43 UTC (Mon) by stijn (subscriber, #570) [Link]

Clearly the game is "most of the desktop computers are …". This makes the first half of your response rather moot (and detracts from the rest). Admittedly my own (this) response has little to offer except nitpicking, but I care about the particular nit where no effort is made to understand someones position. It accounts for about 99.9% of flame wars.

BFS vs. mainline scheduler benchmarks and measurements

Posted Sep 7, 2009 10:02 UTC (Mon) by xav (guest, #18536) [Link] (7 responses)

Most computers will shortly be smartphones running some kind of linux kernel.
And they'll be very picky about reactivity.

BFS vs. mainline scheduler benchmarks and measurements

Posted Sep 7, 2009 10:24 UTC (Mon) by kragil (guest, #34373) [Link] (6 responses)

That is totally OK _with me_.

I think Linux has a lot cruft that is only useful on supercomputers/monster X-cores and in future some kernel devs want to see.

Optimising for smartphones/smartbooks/MIDs/netbooks is really needed and the benchmarks should be very very different like response time to clicks under load or frame drops while playing video etc... at the moment the Linux desktop freezes and skips way too often.

BFS vs. mainline scheduler benchmarks and measurements

Posted Sep 7, 2009 12:53 UTC (Mon) by mingo (subscriber, #31122) [Link] (5 responses)

at the moment the Linux desktop freezes and skips way too often.

We take such problems seriously - please post to lkml about this, with the scheduler maintainers (Peter Zijstra and me) Cc:-ed.

We have many good tools that can get to the bottom to such skipping, if there are people willing to report problems and willing to trace latencies and test patches.

Both Peter Zijstra and me have and test on low-spec systems as well. I've got a 833 MHz Pentium-3 laptop that i (auto-)reboot new kernels into about 10 times every day with new -tip kernels. Peter has a 1.2 GHz Pentium-mobile laptop for interactivity testing. My daily desktop is a dual-core box - not some big honking server machine.

But ... we can only fix the scheduler if you help out too and report your interactivity problems on lkml.

BFS vs. mainline scheduler benchmarks and measurements

Posted Sep 7, 2009 13:20 UTC (Mon) by k3ninho (subscriber, #50375) [Link] (3 responses)

Are you in the process of testing BFS on your 'low end' PIII laptop?

How many people report bugs of stuttering, lockups and hangs anyway? I'd forgive you for thinking it's not a problem because the CFS and Deadline schedulers have been good for me and my home-use workload.

BFS vs. mainline scheduler benchmarks and measurements

Posted Sep 7, 2009 14:36 UTC (Mon) by mingo (subscriber, #31122) [Link] (2 responses)

Are you in the process of testing BFS on your 'low end' PIII laptop?

Not likely - it took 8+ hours to do the quad core tests and a single kernel build iteration takes 1-2 hours on this box.

But that box is perfect for audio skipping problems. Right now it can play an mp3 stutter-free while a make -j3 job is running on it. That's roughly in line with what i'd expect from that box.

How many people report bugs of stuttering, lockups and hangs anyway? I'd forgive you for thinking it's not a problem because the CFS and Deadline schedulers have been good for me and my home-use workload.

We have on the order of one such bugreport per kernel cycle (3 months). They generally get fixed if they are reported and if the reporter reacts to feedback and further testing requests.

BFS vs. mainline scheduler benchmarks and measurements

Posted Sep 7, 2009 18:57 UTC (Mon) by drag (guest, #31333) [Link]

Be sure to throw PulseAudio in there. :)

It'll end up being required desktop component since it's the only system developed so far that can handle hotplugging audio devices _and_ network audio in effective manner. This means on the fly audio configuration changes, which means that USB headsets for VoIP and gaming, bluetooth audio devices, and usb docking stations (etc etc) which are now increasingly common cannot be handled in a sane manner without PA's ability to do on the fly reconfiguration.

Then you'll need to do some graphical benchmarks. Maybe some of those things from Mesa or whatever. Their little things. Just stuff that runs for a few seconds at a time. Those phoronix folks have their benchmark suite and maybe that would be usefull for you guys.

The point for interactivity, as I see it, is adapting to changing workloads. Playing a mp3 + doing a kernel compile is fairly static and the system has time to adapt to it, and whatnot. The system should have a "peaky" workload with occasional high loads and whatnot.

Not that I experienced many problems with the modern kernel compiled with preemption enabled. At least nothing that stands out in my mind right now.

Measuring on down-to-earth hardware

Posted Sep 7, 2009 22:30 UTC (Mon) by man_ls (guest, #15091) [Link]

Not likely - it took 8+ hours to do the quad core tests and a single kernel build iteration takes 1-2 hours on this box.
Pity. It would be interesting to run your benchmarks on a PIII, even if it takes 5 days; or tune them to last less. Just about any current netbook would do too. Any takers?

As a socratic exercise: just what would it prove if BFS performed better than CFS? And then, what would we learn if the reverse happened and CFS bested BFS?

BFS vs. mainline scheduler benchmarks and measurements

Posted Sep 9, 2009 14:33 UTC (Wed) by k8to (guest, #15413) [Link]

I think the idea of 'normal users' going to LKML with their problems is unworkable. However, I am willing to give it a try with my next interactivity stall. I expect to give up rapidly if faced with derision or brush-off.

BFS vs. mainline scheduler benchmarks and measurements

Posted Sep 7, 2009 8:43 UTC (Mon) by iive (guest, #59638) [Link] (3 responses)

I'm not CPU expert or kernel expert, so feel free to correct me.

However I do have the feeling that hyperthreading is the reason of these suboptimal benchmarks. The BFS scheduler could have been made with the assumption that each core runs at same speed, so it would finish X work for Y time on any core. In hyperthreading this is not true, as both threads share same core. In general the CPUs have more computational units than could be used in any given moment. So the second h-thread is "lurking" behind and reusing the free units when first h-thread could not utilize them. This is why HT on P4 gave only 30% boost in best case.

This could also explain why only some people with Intel CPU notice issues, while others don't.

I also wonder how many of the stock CFS heuristics are tuned for HT scheduling and how many special cases are there.

BFS vs. mainline scheduler benchmarks and measurements

Posted Sep 8, 2009 18:13 UTC (Tue) by jzbiciak (guest, #5246) [Link] (2 responses)

I wonder if it might be a different effect. My dual dual-core Opteron box (4 CPUs across 2 chips) dynamically scales the frequency of the CPUs based on load.

What I don't know is the cost of doing so. That is, when it switches from 1GHz to 2.4GHZ, yes, it got faster, but was there, say, a 1ms hitch between the two? Did that hitch affect both cores on that die or just one? If there was a cache-to-cache coherence transfer at the time, did it also experience that hitch?

These details could vary by processor platform, vendor and maybe even chipset and BIOS if the switch is effected via SMM or the like. A sloppier CPU scheduler that kept all the CPUs in the high-frequency state (or low frequency state) would eliminate these sorts of hitches, whereas one that kept the load more concentrated might experience more such hitches when the occasional background load spills onto the CPU that was left sleeping.

BFS vs. mainline scheduler benchmarks and measurements

Posted Sep 9, 2009 10:08 UTC (Wed) by etienne_lorrain@yahoo.fr (guest, #38022) [Link]

I also have some strange behaviour on a no-name dual core all intel portable PC, kind of 2-4 seconds where mouse is not even moving, without any load whatsoever, no log in /var/log/messages, completely random.
This portable PC is cheap and "designed for the other OS" system even if it was sold without anything installed: the DMI information is blank, the ACPI information does not seem to be better.
I tend to think that it is a SMM problem, instead of a scheduler problem, the crappy BIOS (cannot update because no DMI name) does not like Linux, or was explicitely designed to give a bad experience. I would really like to be wrong here.
There was a time when Linux did not rely on any BIOS, but it is no more (SMM cannot be disabled, even under Linux - what is what is handling the forced power off by pressing On/Off button for more than 3 seconds).

BFS vs. mainline scheduler benchmarks and measurements

Posted Sep 10, 2009 22:23 UTC (Thu) by efexis (guest, #26355) [Link]

This I believe is something that was more of an issue than it is now, so CPU's can ramp up their speed much quicker than they could've done before. One problem was for example that higher CPU speeds requires higher volts which can cause delays with the CPU stalling while the voltage steps up. Now instead the voltage will be pushed up a split moment before the frequency is ramped up, so there's no stall. Otherwise, it's all down to the CPU, with different models taking different amounts of time to change frequency, it can make sense to jump to the highest frequency when the usage goes up and then slow it down if needed (such as the ondemand governor does) or scale it up step by step. You want to try set a lower watermark where responsiveness is important, so CPU's always running at say twice the speed that you need it, so you always have room to move into while you wait for the cpu to speed up (eg, when load goes from 50% to 80%, the CPU speeds up to bring the load back down to 50%. Only if loads reaches 100% have you not sped up quickly enough). Of course if you wish to conserve more power, you run the CPU at speeds closer to load. In Linux, there're many tuneables for you to play with to get the responses you wish (/sys/devices/system/cpu/cpu?/cpufreq/<governor>). To see what's available on the Windows platform, there's a free download you can find by googling rmclock that proper spoils you for configuration options. There's no one rule that has to fit all, during boot up the kernel will test transition speeds and set defaults accordingly.

BFS vs. mainline scheduler benchmarks and measurements

Posted Sep 7, 2009 8:14 UTC (Mon) by ketilmalde (guest, #18719) [Link] (1 responses)

> There is not one single person I know that owns a dual-socket desktop.

I have an old dual Pentium-II 450MHz. It's not actually in use anymore, though, so it probably doesn't count.

> That being said I don't think it would make a big deal.

I think it might - things like processor affinity is likely to matter a great deal more on multiple socket systems than on just multicore systems. Multicore chips typically come with a large, shared cache, so moving threads across cores isn't as costly as moving them across sockets.

From what I read, BFS doesn't even try to be NUMA-aware, it doesn't seem unreasonable that it would perform quite differently on single and multi-socket systems.

-k

BFS vs. mainline scheduler benchmarks and measurements

Posted Jun 8, 2010 13:22 UTC (Tue) by vonbrand (subscriber, #4458) [Link]

Way back when I confiscated a dual Pentium Pro (200MHz) to use as a desktop machine for use in a class I was teaching... the machine was old already (I actually canibalized two of them to get a working one).

BFS vs. mainline scheduler benchmarks and measurements

Posted Sep 7, 2009 13:34 UTC (Mon) by da4089 (subscriber, #1195) [Link] (1 responses)

At my office, everyone has an 8-core desktop.
At home, people tend to have single-socket, quad-core desktops.
Laptops are mostly dual-core, although the last two guys who bought one got quad-core, 17" monsters.

So, I think Ingo was reasonable in his choice of platform.

BFS vs. mainline scheduler benchmarks and measurements

Posted Sep 8, 2009 0:18 UTC (Tue) by awalton (guest, #57713) [Link]

> At home, people tend to have single-socket, quad-core desktops.

I want to live at your home. We've bought 4 new home PCs in the past two years, including one just a month ago. They're all dual cores. Even my brand new laptop is dual core.

BFS vs. mainline scheduler benchmarks and measurements

Posted Sep 7, 2009 10:30 UTC (Mon) by nye (subscriber, #51576) [Link] (1 responses)

>A dual quad-core system with hyperthreading? That's hardly a bad choice of hardware for the "desktop system" scale. That's the standard CPU setup for a Mac Pro, for instance.

I hope this doesn't sound trollish, but if you think that's even remotely realistic for even one percent of the PC user base, then you are living in a fantasy realm - or perhaps five years in the future.

I've never even *seen* a computer that powerful. A machine like that would cost *thousands* - nobody spends more than £500 on a computer unless they are a serious enthusiast who happens to be rolling in money - the average user, if they think about it at all, is around now thinking that it might be time to get one of those newfangled dual-core machines.

BFS vs. mainline scheduler benchmarks and measurements

Posted Sep 7, 2009 10:34 UTC (Mon) by nye (subscriber, #51576) [Link]

Okay I realise that that probably did sound trollish, and it's been better covered upthread. My apologies.

BFS vs. mainline scheduler benchmarks and measurements

Posted Sep 7, 2009 12:16 UTC (Mon) by fb (guest, #53265) [Link]

Yes, Con's response was outright rude. OTOH he had made clear that he wasn't interested in that kind of discussion.

However Ingo was obviously down the route of "lies & benchmarks". The point of the scheduler is low end machines, and responsiveness. Ingo posts a benchmark with ridiculously high-end machine, and measuring performance.

I just wish that Con had had the cool head to politely point that if you ask the wrong question, you get the wrong answer.

BFS vs. mainline scheduler benchmarks and measurements

Posted Sep 7, 2009 8:09 UTC (Mon) by sitaram (guest, #5959) [Link]

Maybe, but a couple of para's into Ingo's email I did wonder how relevant the testbed/tests were to my normal workload.

Con's email merely confirmed my suspicions.

Too bad... I might now have to take off my "user" hat (distro supplied stuff only, nothing gets compiled locally, etc) and actually try BFS to see for myself.

BFS vs. mainline scheduler benchmarks and measurements

Posted Sep 7, 2009 5:00 UTC (Mon) by MattPerry (guest, #46341) [Link] (10 responses)

I thought that Con's response was completely appropriate. The test machine that Ingo used, and the tests he performed, were not what BFS was designed for. Ingo is either being disingenuous or just didn't bother to read the FAQ. If Ingo wants to conduct a useful test, he should try the scheduler on a single processor, dual-core machine performing tasks that normal, non-programmer computer users would perform (music listening, web browsing, file copies, word processing, and so forth).

I understand Con's response

Posted Sep 7, 2009 5:46 UTC (Mon) by aorth (subscriber, #55260) [Link] (5 responses)

It's important to read the Gmane thread linked a few comments up. While I'm excited for Linux in the server/high performance space, my Thinkpad only has one core. I've seen a marked increase in responsiveness with Con's BFS. They are things which can't be benchmarked, but make all the difference (like the time for my gmrun box to pop up when I hit Alt-F2 in Fluxbox) when using Linux on a desktop.

"Can't be benchmarked" – No.

Posted Sep 7, 2009 6:14 UTC (Mon) by quotemstr (subscriber, #45331) [Link] (4 responses)

The explanation is less technical and more psychological. What you're seeing is observer bias. See that poster on LKML who claimed that he was seeing improvements in sub-seconds lag in a 3D FPS (which is probably spinning at 100% CPU anyway)? That's precisely the kind of environment most susceptible to observer bias: a supposed small effect in a noisy signal like game latency.

I'll believe there's something to this "can't be benchmarked" nonsense when I see a double-blind experiment run that shows a statistically significant effect. As the old saying goes, "data is not the plural of anecdote".

"Can't be benchmarked" – No.

Posted Sep 7, 2009 6:46 UTC (Mon) by flewellyn (subscriber, #5047) [Link]

Perhaps there is a way to benchmark such things: make a test 3D program which plays multimedia onto a rotating 3D cube, and outputs the frame rate and other latency data on the screen. Run this, then start up some other things that contend for the scheduler, like some I/O (copying a large file?), some network traffic (pinging a host in the LAN?), and such. See how the 3D app holds up under such strain, by watching the numbers.

I don't know how well this would work, but it'd be a test of some kind.

"Can't be benchmarked" – No.

Posted Sep 7, 2009 13:09 UTC (Mon) by cesarb (subscriber, #6266) [Link] (1 responses)

The same poster mentioned frame drops in mplayer. That would be somewhat easy to convert into a benchmark (if mplayer does not output to the console the number of frames dropped, edit its source code to make it do so; then write an app to move the mplayer window around the screen pseudo-randomly and drop it back to the desktop, and see how many frames you can make it drop).

All the other examples mentioned by that poster sound like they could be benchmarkable with some coding effort. For instance, in the Doom 3 example, you would not measure the frame rate, but the frame jitter (record the time of the end of the "flush" call which actually pushes the image to the screen for each frame, subtract from the time for the previous frame, and see which is the highest difference and how uniform the differences are). Even if for some reason you cannot change the source code of your game, you can change the source code of the libraries it calls to do the "flush", or even interpose with LD_PRELOAD or something like it.

You could even measure the "input lag" in his sound example by building a hardware contraption which "presses a key" (by pretending to be a keyboard), listens to the analog audio output, and logs the time difference between the input and the output.

This all seems benchmarkable without the need for a double-blind test.

"Can't be benchmarked" – No.

Posted Sep 7, 2009 17:12 UTC (Mon) by cesarb (subscriber, #6266) [Link]

This is what I meant by interposing with LD_PRELOAD:

http://github.com/cesarb/glxswapbuffersmeasure/tree/master

This is a small quick-and-dirty library I just wrote which hooks into glXSwapBuffers via LD_PRELOAD and prints some statistics to stderr on exit.

An example of its output with everyone's favorite "benchmark" tool, glxgears, on an outdated distribution (thus an older kernel):

LD_PRELOAD=./glxswapbuffersmeasure.so glxgears
1142 frames in 5.0 seconds = 228.375 FPS
1035 frames in 5.0 seconds = 206.474 FPS
934 frames in 5.0 seconds = 186.540 FPS
glXSwapBuffers count: 3947, avg: 0.004757, variance: 0.000045, std dev: 0.006699, max: 0.204504

I did some moving of windows around to make it stutter a bit more, and the output from my test library shows it (200ms max latency, which corresponds to around 5 FPS). Note that the average time between glXSwapBuffers calls approximately matches glxgear's FPS printout.

It should be quite simple for someone who sees latency problems which seem to be cured by BFS to try to run the same 3D game with something like this library both in the mailine scheduler and in BFS and see if it shows any differences in the output. Of course, the code I posted can be enhanced to get better statistics (like a histogram of the latencies); I put the code under Creative Commons CC0 (a bit similar to "public domain").

"Can't be benchmarked" – No.

Posted Sep 7, 2009 13:49 UTC (Mon) by job (guest, #670) [Link]

I disagree. You could, for example, easily count the number of buffer underruns with pulseaudio when playing an mp3 at the same time as you compile the kernel. Then you could do the same thing with a movie.

These are the kinds of things normal users do when latency really counts. The problem is not measuring it, the problem is that nobody is really interested.

Morton's Fork

Posted Sep 7, 2009 6:35 UTC (Mon) by quotemstr (subscriber, #45331) [Link] (2 responses)

First of all, the burden of proof is on BFS advocates to provide a better test. Ingo's test was well-described and performed under reasonable conditions. Kolivas provided no comparably rigorous numbers. Your suggestion, to test what users actually use, puts kernel developers in an unreasonable dilemma. One the one hand, kernel developers can test the tasks that "users would perform", but because numeric results of these tests are not easily measured, they are meaningless without an expensive, inconvenient double-blind satisfaction study. (And really, the onus is on BFS advocates to provide one if that's what it takes.)

On the other hand, kernel developers can use contrived tests like the pipe example that are easily quantified, but that only approximate user workloads. These tests can be improved, but one will always be able to claim that they don't measure what users "really" do. Either way, the claim that BFS is superior will have been made unfalsifiable and unscientific.

Morton's Fork

Posted Sep 7, 2009 12:57 UTC (Mon) by Lennie (guest, #49641) [Link] (1 responses)

Let's start with 'frames skipped' in mplayer or vlc or something.

Morton's Fork

Posted Sep 11, 2009 1:51 UTC (Fri) by Spudd86 (guest, #51683) [Link]

Ingo mention that he does test exactly this on low end machines further up

BFS vs. mainline scheduler benchmarks and measurements

Posted Sep 7, 2009 6:53 UTC (Mon) by bvdm (guest, #42755) [Link]

Did Ingo claim that he was testing BFS against the mainline scheduler for BFS's intended use cases? No.

"I'd guess that a machine with more than 16 CPUS would start to have less performance."

Even if you consider hyper-threading to result in 16 cores (which is not the case at all), in his FAQ Con claims that BFS should perform well up to 16 cores.

I realize how people can be very passionate about Linux and how the scheduler as something that critically affects user experience becomes important, but is all this emotion really necessary?


Copyright © 2026, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds