BFS vs. mainline scheduler benchmarks and measurements

Posted Sep 7, 2009 3:13 UTC (Mon) by sbergman27 (guest, #10767)
Parent article: BFS vs. mainline scheduler benchmarks and measurements

Could Ingo really not find a more appropriate machine for testing BFS? It would be more interesting, I would think, to see its performance on a netbook doing netbook-like things. I don't think BFS was really intended for kernel compiles on an octo-core.

to post comments

BFS vs. mainline scheduler benchmarks and measurements

Posted Sep 7, 2009 3:41 UTC (Mon) by Baylink (guest, #755) [Link] (4 responses)

He said 'hyperthreaded'.

Doesn't that make it a 16-core?

BFS vs. mainline scheduler benchmarks and measurements

Posted Sep 7, 2009 4:05 UTC (Mon) by Tracey (guest, #30515) [Link]

Baylink asked about 16 core.

We are looking to build a relatively inexpensive machine using a motherboard that can use two quad cores. This is pretty much what Ingo did his tests on(i7/i5s are hyperthreaded).

This machine will be used for compiling and desktop use.

There are wonderful people putting time and energy into the handhelds. Maybe these parties aretalking about fixing two completely different things then?

BFS vs. mainline scheduler benchmarks and measurements

Posted Sep 7, 2009 12:11 UTC (Mon) by nix (subscriber, #2304) [Link] (2 responses)

Yes, I'd say it does. The shared caches should affect scheduling decisions (but of course in BFS they don't :) ), and other shared resources (that is to say, pretty much all of them) would affect speed directly, but still you have to schedule 16 entities at once. They're just not symmetrical entities anymore. (In fact if it was dual-die you're into NUMA land, which means BFS is bound not to work well on it as it has no NUMA-awareness by design. I suspect my single-die quad Nehalem would work much better with it.)

[updated, quad core results] BFS vs. mainline scheduler benchmarks and measurements

Posted Sep 7, 2009 12:28 UTC (Mon) by mingo (subscriber, #31122) [Link] (1 responses)

Today i've measured and posted single-socket non-NUMA quad-core results as well:

"[quad core results] BFS vs. mainline scheduler benchmarks and measurements "

As the graphs show it, the quad results are similar to the 8-core results. So it wasnt NUMA or 16 cpus that made the difference.

Btw., you'd be wrong to treat an 8 core box with HyperThreading as a 16 core box. The physical resources are in essence that of an 8 core one - it's just more spreadable.

BFS should have no design disadvantage from HyperThreading, as siblings share the cache.

[updated, quad core results] BFS vs. mainline scheduler benchmarks and measurements

Posted Sep 7, 2009 14:13 UTC (Mon) by nix (subscriber, #2304) [Link]

Interesting.

(With regard to hyperthreading you are of course right that the physical resources are those of the physical cores, but surely unless you are HT-aware you will get lower performance on an HT system than otherwise, because you won't know to e.g. schedule threads of the same process on the same physical core if possible, to maximize cache sharing. But I know you know all this as the current mainline scheduler does it :) )

BFS vs. mainline scheduler benchmarks and measurements

Posted Sep 7, 2009 7:03 UTC (Mon) by kragil (guest, #34373) [Link] (11 responses)

Ingo lives in another world. His ridiculously large jpgs killed my Netbook. He is obviously one of those detached from reality kernel devs Con is talking about.

BFS vs. mainline scheduler benchmarks and measurements

Posted Sep 7, 2009 7:07 UTC (Mon) by quotemstr (subscriber, #45331) [Link] (10 responses)

I hope that's snark.

If not, well, it's ridiculous. Need everyone cater to the needs of your deliberately underpowered machine? Are you really claiming that using a high resolution for a graph intended for the consumption of a relatively small group of technical people constitutes a detachment from the real user base? Ingo can rightfully assume that anyone who reads LKML has the ability to resize a raster image.

BFS vs. mainline scheduler benchmarks and measurements

Posted Sep 7, 2009 7:35 UTC (Mon) by k8to (guest, #15413) [Link]

"deliberately underpowered"?

...

BFS vs. mainline scheduler benchmarks and measurements

Posted Sep 7, 2009 7:58 UTC (Mon) by kragil (guest, #34373) [Link] (8 responses)

Underpowered?

I still think having 1 GB of RAM should be more than enough to look at pictures on the internet :P
But if you have a few apps running and starting to load such a large jpg (really stupid format for graphs btw) it will start to swap and essentially lock for hours for most users ( after a minute of trying I was able to crtl-alt-2 and kill the browser, hail to Fedoras responsiveness!)

BFS vs. mainline scheduler benchmarks and measurements

Posted Sep 7, 2009 9:58 UTC (Mon) by jospoortvliet (guest, #33164) [Link] (7 responses)

darn, I hadn't even noticed they were so big... Used konqi, middleclick on
em and in the tab gwenview automatically resized them to fit the window.
Only after reading this thread I had a closer look and saw their size...
Really huge indeed. I guess 3 gb ram saved me :D

fixed those JPGs

Posted Sep 7, 2009 12:42 UTC (Mon) by mingo (subscriber, #31122) [Link] (6 responses)

Oops - good point. I typoed '600' as '6000' and Gimp was too fast for me to notice.

I fixed all the jpgs - they now have standard size, 50K apiece, 1024 pixels width.

[ One more proof that i'd make a sucky web artist i guess ;-) ]

worse than it should be

Posted Sep 7, 2009 13:42 UTC (Mon) by job (guest, #670) [Link] (5 responses)

Let me explain what Firefox did with the initially large pictures: It allocated memory. That practically kills a normal PC with Linux today. My machine took four whole minutes until I could ctrl-alt-f1 and kill the offending process.

I understand VM pressure is high, but why can't normal apps get at least a small timeslice now and then even in these extreme situations?

It would be a bit discouraging to say the least if this was a desktop users first impression of Linux; that it "hangs" (sort of) if you click on a large picture in your web browser.

worse than it should be

Posted Sep 7, 2009 14:18 UTC (Mon) by mingo (subscriber, #31122) [Link] (1 responses)

What happens during big VM pressure rarely depends on the process scheduler. If you monitor your system during such situations you'll see there's plenty of idle CPU time - just nobody is able to make progress because everyone will be swapping around small fragments.

[ Or if there's a lot of CPU time used, it's all kswapd's ;-) ]

worse than it should be

Posted Sep 7, 2009 20:40 UTC (Mon) by alankila (guest, #47141) [Link]

compcache seems to really help about with making progress. Even if you are swapping, it's so fast that you can actually use your system: start new terminals, run top, etc. Even if the task is in an allocation frenzy and ends up OOM-killed, it does so with relatively little disk activity.

I really have a love affair with it compcache---to the point that I have given up all other types of swap and am now married to this single solution. It can also help with large images, especially those that are mostly single color. I imagine those pages compress very, very well...

worse than it should be

Posted Sep 7, 2009 15:42 UTC (Mon) by epa (subscriber, #39769) [Link] (2 responses)

Oh, the ironing. Maybe 'open huge image in Firefox' should be added to the set of kernel benchmarks? The measurement would be how long it takes for some other process to allocate and use a mere twenty megabytes while Firefox is thrashing around.

worse than it should be

Posted Sep 8, 2009 8:12 UTC (Tue) by mjthayer (guest, #39183) [Link] (1 responses)

I wonder why other processes have to suffer so much at all for Firefox's memory allocation? It ought to be possible (*) to limit the rate at which a process can cause pages belonging to other processes to be swapped out, and to ensure that the other processes never go below a certain threshold of physical pages (possibly a lower threshold for processes that are rarely used, but even they are probably in memory in the first place for a reason).

(*) Yes, I know someone still has to do it. If no one else does, and I don't get told at once on LWN why this is such a bad idea, perhaps I will have a look at if I ever have a free minute...

worse than it should be

Posted Sep 16, 2009 20:06 UTC (Wed) by oak (guest, #2786) [Link]

Put Firefox to a container group of its own and set a limit on the active
pages it can have?