The future of the FUD mill [LWN.net]

There's yet another Microsoft-funded analyst study out there; this one, done by VeriTest, compares deployment times for Microsoft Windows Small Business Server and Red Hat Enterprise Linux. No doubt everybody will be surprised to hear that the study (available in PDF format) concludes that Windows is better. Four tasks were set out: install the system with basic services, set up performance monitoring and reporting, set up an intranet web site, and configure the network for remote management. Doing these tasks with Windows took, they say, 4 1/2 hours and 125 steps. Linux required 7 1/2 hours and 555 steps.

It is not hard to poke holes in the study, of course. Somehow it was possible to set up an intranet server on Windows with zero steps - but it still took seven minutes. Somehow the report didn't comment on the discouraging time per step required to accomplish this task on Windows. Errors made by the (Microsoft-hired) consultants performing the Linux installation were counted as steps. Tasks like checking the system with nmap were also counted. Setting up remote administration took 100 steps; we could suggest a shorter way of doing that:

Enable sshd.

The VeriTest people, instead, set out on a series of tasks involving installing the kernel source, setting up PPTP, and carrying out several tasks on the Windows client - all of which counted as steps, of course.

One could go on about this report for a long time; see, for example, the letter from Leon Brooks on this week's Letters Page. The more interesting development, however, is that Forrester Research has, after having Microsoft trumpet one of its studies, issued this statement on the integrity of its reports.

Recently, in two isolated and unrelated cases, we conducted privately sponsored studies for two vendor clients. We stand by the integrity of both studies. However, we erred in allowing those clients to publicize the research findings. In response to these two isolated events, Forrester has taken immediate steps to tighten our internal process and clarify our Integrity Policy. As part of this clarification, the company will no longer accept projects that involve paid-for, publicized product comparisons. This move revalidates and strengthens Forrester's research integrity.

Forrester, in other words, is getting out of the analyst-for-hire FUD business. Given that this business can only be lucrative, Forrester's decision to leave it behind is worthy of note.

FUD-for-hire has long been an important business tool in the technology world. Analysts have been happy to have the business, and they have been able to live with the fact that their output always seems to support the sponsor's agenda. Technical journalists have long liked these reports; they can easily be cast into a story without requiring much in the way of creative or critical thought. The whole system worked smoothly as a way of shaping public perception of technology products.

Something has happened over the last decade or so, however. The net has made it easy for interested parties to rip apart biased or poorly-done studies. And the rise of free software has greatly increased the number of people who feel some sort of ownership interest in the systems they use. As a result, anybody publishing a report critical of free software had better be very sure of his ground, because that report will be subjected to intense scrutiny. Some of the people performing that scrutiny will know far more about the subject manner than the analysts who wrote the text, and they will not be afraid to say, in public, what they think. Shoddy research and skewed studies do not fare well in the modern environment.

It has been noted for years that FUD attacks on Linux tend to backfire; even Microsoft has commented on this fact. The combination of the net and the Linux community has managed to neutralize - or at least strongly diminish the effect of - FUD. Analyst companies which are seen as taking part in outright FUD attempts have seen their own credibility suffer; remember MindCraft? Now some analyst companies, concerned about the perception of their integrity, are realizing that the FUD business is a poor place to be in the long run. That is a victory for the Linux community, and for the level of technology industry discourse in general.

(Log in to post comments)

Nmap supposedly 3+ steps!

Posted Oct 9, 2003 3:14 UTC (Thu) by fyodor (subscriber, #3481) [Link]

> Tasks like checking the system with nmap were also counted.

Not only was it counted, but many times! In just task #1 (pp 15), the command "nmap localhost" counted as step number 48, 135, and 138. This "analysis" is obvious FUD (and they could have just used netstat), but it is still good to recognize security validation and hardening as an important step when setting up a new server. They should have (but didn't) run Nmap against the Windows 2003 hosts too. Those are often less secure out of the box than RedHat AS is.

These guys used the Nmap version that comes with Redhat AS, but for users not under the pressures of a stopwatch and step-counter I recommend checking out Nmap 3.48, which was released Monday. It offers a major expansion of the new version scanning feature as well as many bugfixes. OK, I'll end my blatant plug now :).

Cheers,
Fyodor
http://www.insecure.org

Nmap 3.48

Posted Oct 9, 2003 5:40 UTC (Thu) by stock (subscriber, #5849) [Link]

geez, what a tremendous tool nmap has become :)
an example :

[jackson:root]:(~)# nmap -A -T4 -F www.sun.com

Starting nmap 3.48 ( http://www.insecure.org/nmap/ ) at 2003-10-09 07:22
CEST
Warning: OS detection will be MUCH less reliable because we did not
find at least 1 open and 1 closed TCP port
Interesting ports on 64.124.140.199.sun.com (64.124.140.199):
(The 1209 ports scanned but not shown below are in state: filtered)
PORT STATE SERVICE VERSION
80/tcp open http?
443/tcp open ssl Microsoft IIS SSL
1 service unrecognized despite returning data. If you know the
service/version, please submit the following fingerprint at
http://www.insecure.org/cgi-bin/servicefp-submit.cgi :

<servicefp-submit.cgi deleted>

No OS matches for host (test conditions non-ideal).
TCP/IP fingerprint:
SInfo(V=3.48%P=i686-pc-linux-gnu%D=10/9%Time=3F84F0FC%O=80%C=-1)
TSeq(Class=TR%IPID=RD%TS=U)

<rest of fingerprint deleted>

Nmap run completed -- 1 IP address (1 host up) scanned in 123.144
seconds
[jackson:root]:(~)#

" If SUN.COM. runs a 443/tcp open ssl Microsoft IIS SSL server, they are
bound for shit.... even from McNeally not knowing it. :)
SUN.COM. is a huge company... it wouldn't surprise me if professional
undercover trolls are on their payroll. "

Robert

Nmap 3.48

Posted Oct 9, 2003 6:09 UTC (Thu) by fyodor (subscriber, #3481) [Link]

"If SUN.COM. runs a 443/tcp open ssl Microsoft IIS SSL server, they are bound for shit..."

You never know - Microsoft is/was running www.microsoft.com through an Akamai Linux box (example scan output). With major sites like Sun/Microsoft/Ebay/etc. you usually aren't just scanning one machine. There are often load balancers, SSL accelerators, and who knows what other cruft in front of the machine(s). If you compile your Nmap to link with OpenSSL, it will scan through that and tell you what is behind. I also just added a service fingerprint for SunOne Webserver for the next version of Nmap:

#./nmap -A -T4 -F www.sun.com

Starting nmap 3.48 ( http://www.insecure.org/nmap/ ) at 2003-10-08 22:59 PDT
Warning:  OS detection will be MUCH less reliable because we did not find at least 1 open and 1 closed TCP port
Interesting ports on 64.124.140.199.sun.com (64.124.140.199):
(The 1209 ports scanned but not shown below are in state: filtered)
PORT    STATE SERVICE  VERSION
80/tcp  open  http     SunONE WebServer 6.0
443/tcp open  ssl/http SunONE WebServer 6.0
...

Cheers,
-Fyodor