Windows Server 2003 better than Linux (Inquirer) [LWN.net]

This article in the Inquirer typifies the hype that is going around on Microsoft's latest OS offering. "The combatants fighting it out were Windows Server 2003, Red Hat 8.0 and Red Hat Advanced Server 2.1. The arena was serious enterprise, one of the machines used in the test was an 8 processor HP Proliant DL760 Xeon system with 4GB of RAM. The Linux systems were running Samba. The test was performed using ZD NetBench 7.02. And Windows Server 2003 gave the Linux system a right good stuffing, almost performing at twice the speed in many tests." (Thanks to Dan Kegel)

(Log in to post comments)

Windows Server 2003 better than Linux (Inquirer)

Posted May 6, 2003 20:47 UTC (Tue) by backtick (guest, #364) [Link]

If you read the article and the report, you see all the benchmarks are based on exactly one thing: PEAK throughput. Hrm. If I want to download a 100 MB file, let's review the possible scenarios involved.

A) Computer 1 downloads the file at a initial rate of 20 MB/sec for 1 second, then drops to 1 MB/sec for the remainder of the file. Total DL time is 81 seconds, with a peak rate of 20 MB/sec.

B) Computer 2 downloads the file at an initial rate of 8 MB/sec for 1 second, then drops to 4 MB/sec for the remainder of the file. Total DL time is 24 seconds, with a peak rate of 8 MB/sec.

sarcasm=ON
Oh yeah, CLEARLY computer #1 is what you want to buy. Who cares that in the real world, computer #1 took almost 3 times LONGER to actually accomplish the task at hand than did computer #2, when the PEAK rate for computer #1 was 2.5 times FASTER than computer #2?
sarcasm=OFF

Benchmarks without the supporting data (which the testing folks refuse to release) are useless. Only open becnhmarks, with the *complete* raw data, mean anything.

Windows Server 2003 better than Linux (Inquirer)

Posted May 7, 2003 5:38 UTC (Wed) by C.Gherardi (subscriber, #4233) [Link]

> If you read the article and the report, you see all the benchmarks are based on exactly one thing: PEAK throughput.

I was listening to Andrew Tridgells talk from LCA 2003 recently and he mentions this. Samba tries to have a good throughput in all scenarios whereas Microsoft has a peak performance and an exponential falloff as load increases. Some may find it an interesting listen.

http://mirrors.uwa.edu.au/lca2003/loopback/papers/Tridge_Talk/Abstract.html

Windows Server 2003 better than Linux (Inquirer)

Posted May 6, 2003 19:34 UTC (Tue) by alan (subscriber, #4018) [Link]

The title is still misleading. 'Better' at what? I only see fileserving mentioned,.. so it should maybe be 'Win2003 serves files faster than RedHat Linux 8 and RHAS 2.1'.

Though the underlying filesystem may be part of it,. I'd like to know if the default RH samba installation makes use of smbfs.

I'd also like to see a comparison of Win2003 to linux 2.5/2.6.

Very amusing to see such a broad and somewhat misleading claim made though. You'd think they'd forgotten their audience.

Windows Server 2003 better than Linux (Inquirer)

Posted May 7, 2003 1:37 UTC (Wed) by elanthis (subscriber, #6227) [Link]

smbfs? what does that have to do with the on-disk file system? iirc, smbfs is for the client, no?

Windows Server 2003 better than Linux (Inquirer)

Posted May 8, 2003 0:59 UTC (Thu) by alan (subscriber, #4018) [Link]

hrrm,. samba client and server I thought. smbfs has nothing to do with the on disk filesystem. I'm simply noting that they compared fileserving performance. I am less interested than the previous poster was about the ondisk filesystem format because using the kernel smbfs support would enhance fileserving performance,. which is what was actually being compared.

Windows Server 2003 better than Linux (Inquirer)

Posted May 7, 2003 0:10 UTC (Wed) by DaveK (subscriber, #2531) [Link]

Seems very reminisent of the Mindcr*p HPPTD tests of a couple of years ago using stock RH boxen and extremely uber tweaked redmond boxes that, when competent Linux admins turned up and tweaked the RH boxen for the re-run, were shown to be somewhat bogus.
how often do we get such test results, but I have yet to find anyone who, in a real world installation, has attained anything other than the reverse of the published results.
Oh well...

Windows Server 2003 better than Linux (Inquirer)

Posted May 8, 2003 20:26 UTC (Thu) by dsime (guest, #5764) [Link]

You might also remember that that test was what caused MAJOR changes to the kernel's tasking code. Windows WAS better, then Linux was improved. I hope this whacked benchmark will cause a similar improvement in Linux.

Windows Server 2003 better than Linux (Inquirer)

Posted May 6, 2003 19:51 UTC (Tue) by ami (guest, #5280) [Link]

I think this is quite amusing because it's just asking for a counter-benchmark right after the Linux 2.6 release. It also means that the counter-benchmark will get some good press when it comes out.

Note that these benchmarks focus on SMP performance - an area that has seen a lot of work in the 2.5 series.

Windows Server 2003 better than Linux (Inquirer)

Posted May 6, 2003 20:19 UTC (Tue) by parimi (subscriber, #5773) [Link]

Alas, there were no statistics and reports on the number of blue screens that each of the operating systems flashed. Boy Microsoft seems to have put so much research into their new Windows 2003 OS. It appears to me that Microsoft overpaid those guys to give their OS a favorable report. Perhaps some more things to compare are

1) What to do when you dont get the Graphical screen after bootup?
2) What to do when CTRL+ALT+DELETE doesnt respond at all
3) Trying to apply patches without rebooting the system
4) Hours of sleep lost while your webserver is running IIS as opposed to
running apache on a Linux box.
5) Trying to get windows recognize an ext3 partition(poor thing always fails,
and says "unknown type" for such partitions).

and many more such issues...

--ravi

Windows Server 2003 better than Linux (Inquirer)

Posted May 6, 2003 20:23 UTC (Tue) by iabervon (subscriber, #722) [Link]

It's certainly possible that this will turn out to be a real performance problem, like that Apache test a few years ago. In fact it's probably a known issue with 2.4, fixed in 2.5. What would be really amusing would be to treat this like a bug report, figure out where the bottleneck is, and fix it if possible. It's getting to the point where Red Hat (or someone else) could actually just as an enterprise customer for profiling output and figure out what the problem is without having a Linux-savvy user. Then they could thank Microsoft for funding a testing session which led to improving Linux...

Windows Server 2003 better than Linux (Inquirer)

Posted May 7, 2003 0:10 UTC (Wed) by mattdm (subscriber, #18) [Link]

For that, they'd have to provide a lot more actual data. As it is, this is the news equivalent of every ISP front-line-tech's favorite call: "The internet is broken!"

Windows Server 2003 better than Linux (Inquirer)

Posted May 6, 2003 22:48 UTC (Tue) by planet12 (guest, #4199) [Link]

Another point of note is that on the Windows side they set a registry value to disable last access updates on the filesystem, yet did not mention doing the equivalent on the Linux side of mounting the filesystems with the noatime option. How much difference that would make in this setting I don't know.

Overall though, this report does seem to be far less (at least obviously) imbalanced than previous ones, but it's still testing a Microsoft controlled protocol versus the people who have to reverse engineer it. Also, I'd rather they tested with an 'off-the-shelf' version of Windows 2003 Server; we've no way of knowing whether they got a 'tweaked' one.

So, when do we get the NFS test?

What about other clients? What about samba3?

Posted May 7, 2003 0:44 UTC (Wed) by ranger (subscriber, #6415) [Link]

The report states that Windows XP was used as the client for the test? MS has made some changes recently to allow bigger packets via SMB. How would win2k3 fair if it were serving clients that don't support sendfile (like win98, winnt, winme, etc)? What about non-windows clients (dos, OS/2, unix-based OSs)? Was signorseal support in XP turned off (could affect performance against Samba as the XP clients may try and negotiate it first and fail)? Why was the noatime option not used if the Windows equivalent was? Why were ext3 tuning options not used?

Samba2 has sendfile support, but it is disabled by default. Was samba2 recompiled with sendfile support and tested?

Samba3 has more mature sendfile support than samba2, and is arguably more stable than win2k3 at this stage, so was samba3 tested??

And I can't believe they couldn't find updated samba packages for Redhat from updates, I am sure RH has released 2.2.7a packages for AS 2.1!

Really, I don't think it would be difficult to get AS2.1 to beat Win2k3 by doing the same amount of tuning on the AS2.1 as they did to the win2k3 machines.

I would also like to see their full smb.conf ...

What about other clients? What about samba3?

Posted May 7, 2003 2:08 UTC (Wed) by xose (guest, #535) [Link]

They "forgot" update a lots of packages: http://fr2.rpmfind.net/linux/redhat/updates/enterprise/2.1AS/en/os/SRPMS/
the most necessary were samba-2.2.7-3.21as and kernel-2.4.9-e.16

Windows Server 2003 better than Linux ??

Posted May 7, 2003 2:45 UTC (Wed) by neoprene (guest, #8520) [Link]

This report begs more questions than it answers. Synthetic benchmarks are usually flawed, I dont know about this one. Just the fact that MS paid for the test makes it suspicious. What other tests have they paid for and _not_ published?
Anyone buying an $80,000+ (yeah I checked) 32-bit 8-way DL760 server would likely not be fooled by this "study".
Likely he would likely look into a 64-bit proposition, including AMD Opteron (cluster?), Itanium2, POWER4+ ($91,00+) and even something Sunny and Utrasparky ($86,000 V880) and not even have to bother with MickeySoft.
It looks like Open Source is rattling the cage in Redmond.

Windows Server 2003 better than Linux ??

Posted May 7, 2003 4:59 UTC (Wed) by Baylink (subscriber, #755) [Link]

I've seen press today that Merrill Lynch has downgraded their stock to
"neutral", precisely because of the danger to them from Linux (in the
person of IBM, mostly, but still)...

Our Plans For World Domination may be speeding up.

Merrill Lynch downgrades Microsoft

Posted May 10, 2003 16:03 UTC (Sat) by rwmj (subscriber, #5474) [Link]

More likely it's because Microsoft has been running an Enron-style accounting scam (legally, I hasten to add) for the last few years. They've actually been making loses & have a huge undisclosed liability.

See: http://www.billparish.com/msftfraudfacts.html for the basics and and the updates section of the site for further news.

Rich.

Windows Server 2003 better than Linux (for certain stock holders)

Posted May 7, 2003 3:02 UTC (Wed) by neoprene (guest, #8520) [Link]

http://www.etestinglabs.com/about/news/press/lionbridge_microsoft.asp
http://www.etestinglabs.com/about/news/press/pr_02-07-23.asp?visitor=X
http://boston.internet.com/news/article.php/1373161
http://www.lionbridge.com/kc/gp_intro.asp?kb=mlcm&wp=liox-msft_sol_brief

So who's blowing who?

Windows Server 2003 better than Linux (Inquirer)

Posted May 7, 2003 3:50 UTC (Wed) by andrewt (subscriber, #5703) [Link]

http://lse.sourceforge.net/benchmarks/netbench/results/february_2002/sched/

I ran some NetBench tests over a year ago on a 4 Pway III system, 22% slower clock rate (700 MHz), half the cache size (1MB) and 2/3 the memory (2.5GB). The best result achieved was 774 Mbps. This was on 2.4.17 with a development version of O1 scheduler and some other tweaks.

2.5 definitely has a lot of new scalability features that would help like TSO, Dcache RCU, etc, not to mention tweaks like enforcing case sensitivity in samba and using spinlocks for its database. Granted my run was with ext2, and ext3 was generally 9% slower, but considering the slower system this is well within striking distance of these results.

Windows Server 2003 better than Linux (Inquirer)

Posted May 7, 2003 5:48 UTC (Wed) by cott (guest, #6931) [Link]

Umm... Who uses a file server in this capacity, anymore? My company
certainly doesn't. I mean, being able to map a drive is great for sharing
files and all, but using Samba *or* Windows Networking for anything that
requires high-performance file access is just stupid.

We have an application that requires high-speed access to data and we
used to do that ourselves, using a Windows or Novell server. Then, we
discovered that our application was about 15x faster when we changed to
Unix and ran directly on the server. Now, the only way we recommend
running our software is either by Telnetting to the server or through a
DBMS.

I guess my point is: raw file service is a miniscule part of what the
typical file server does. Who cares if WS2k3 is better at that one thing?

Windows Server 2003 better than Linux (Inquirer)

Posted May 7, 2003 9:55 UTC (Wed) by standby (guest, #9796) [Link]

Wo!!! Excellent experiments _mislead_ people!!

"Peak" isn't a good metric (the counter-example given by backtick tells why it is so). No guy in networking field will tell you that higher "peak" throughput implies better performance.

Only naive guy will take this measurement to say it is better.

Also, can Red Hat really represent all Linux distributions? I think they should mind their words used.

Windows Server 2003 better than Linux (Inquirer)

Posted May 7, 2003 13:13 UTC (Wed) by cpm (subscriber, #3554) [Link]

I too would have liked to see a Samba on XFS benchmark
in this test. That would have actually been interesting.
ext3 ain't that fast. Robust, yes. Fast, no. ext2 is,
or rather IMHE much faster.

I did like like the tone of the article. I really enjoy
reading this guys stuff.

One issue I did have with the article was the mention
of RH Enterprise Edition as being cost equivalent with
Win twenty ought three. Not so. RHEE is not cheap, but
it still has Win-whatever beat to death over those
wonderful things known as CALS and other associated
nasty creeping costs. This point was very misleading.

They used antique and obsolete kernels

Posted May 9, 2003 18:09 UTC (Fri) by stock (subscriber, #5849) [Link]

The used linux versions i.e. RedHat Advanced Server 2.1 uses kernel 2.4.9-e.3enterprise and RedHat 8.0 uses kernel 2.4.18-14smp. Kernel 2.4.9 is known to have serious problems with Virtual Memory management and shouldn't be used at all today. kernel 2.4.18 may not seem to be that old, but the hardware which is used in the tests are the latest dual P4 Xeon 2.4GHz boards for which those kernels are ancient stuff.

a remedy to get it up to date is install the latest stable kernel 2.4.20 like this :

1. install and compile gcc-2.95.3. Linus Torvalds keeps telling us this, i guess he's not telling us any fairytales.
2. download kernel linux-2.4.20.tar.bz2 from ftp.kernel.org
3. apply the ext3 patch for kernel 2.4.20 from: http://www.zip.com.au/~akpm/linux/ext3/ and compile your patched 2.4.20 kernel using gcc-2.95.3
4. install e2fsprogs 1.29 of higher.
5. for the 100Mbit and 1000Mbit onboard Intel ethernet cards use e100.0 and e1000.o as the drivers. eepro100.o should not be used for Intel based network cards in those dual P4 Xeon machines. eepro100.o might be ok for a normal 32bit 33 MHz PCI interface, but the machines here have 64bit PCI-X interfaces which run at 66 MHz upto 133 MHz.

After applying those 5 points your brandnew dual P4 Xeon machine will blast like never before. I tested some things here with a Tyan Tiger GC-SL (S2727) board. Installing redhat 7.3 on that will be ok. But that board will only start to perform after running the above fixes.
After that a make -j 4 bzImage is finished within 60 seconds.

Robert