LWN.net Logo

Re: Drivers -- below the OS?

From:  Linus Torvalds <torvalds-AT-linux-foundation.org>
To:  "Miller, Marc" <marc.miller-AT-amd.com>
Subject:  Re: Drivers -- below the OS?
Date:  Fri, 3 Aug 2007 18:12:57 -0700 (PDT)
Message-ID:  <alpine.LFD.0.999.0708031803240.31678@woody.linux-foundation.org>
Cc:  desktop_architects-AT-lists.linux-foundation.org
Archive-link:  Article, Thread



On Fri, 3 Aug 2007, Miller, Marc wrote:
>
> I recently came across an article claiming that virtualization "liberates"
> the OS from the hardware, basically implying that the hypervisor can now
> contain all of the drivers and that the OS just needs to have a standard
> abstraction layer for accessing the true driver involved.  

I think what you're seeing is virtualization proponents being absolutely 
_desperate_ for any reason to use virtualization.

The fact is, the absolute last place you want to see drivers is in the 
hypervisor, not only because the added abstraction layer is inevitably a 
big performance problem, but because hardware and drivers are by 
definition buggier than "generic" code that can be tested. 

I expect hardware and drivers to reach the same kind of stability we have 
in generic code the moment people no longer develop hardware actively. At 
that point, drivers probably become stable too, and can be put in the 
hypervisor. Any time before that happens, you want the drivers "further 
out", rather than "closer in" to the system.

So next time you hear about hypervisors doing anything at all, ask 
yourself where the message comes from. Does it come from a group of people 
who are desperately trying to make themselves appear relevant? Or does it 
come from forty years of actual real-life practice?

[ Side note: I don't doubt at all that virtualization is useful in some 
  areas. What I doubt rather strongly is that it will ever have the kind 
  of impact that the people invovled in virtualization want it to have. It 
  would appear that virtualization is the "message-passing microkernel" of 
  this decade, and that people have a really hard time accepting that the 
  reason operating systems still basically look 100% the same today as 
  they did almost forty years ago, is that that is simply a very practical 
  arrangement!

  So every decade, you'll find somebody who "reinvents" the OS. Apparently 
  just because they are bored with the fact that operating systems have 
  really been the same-old, same-old for a long time.

  Guess what? Wheels have been round for a really long time, and anybody 
  who "reinvents" the new wheel is generally considered a crackpot.  It 
  turns out that "round" is simply a good form for a wheel to have. It may 
  be boring, but it just tends to roll better than a square, and "hipness" 
  has nothing what-so-ever to do with it ]

			Linus


(Log in to post comments)

Re: Drivers -- below the OS?

Posted Aug 9, 2007 15:24 UTC (Thu) by heckler (guest, #46684) [Link]

Mr. Linus T. implies that virtualization is just another attempt to reinvent the wheel, focusing his analogy on the shape of the wheel. It turns out; his analogy is right on target as it helps to dramatize that the OS folks are running on empty when it comes to appreciating the reality of virtualization. It's not the shape of the wheel that virtualization is changing. Rather, virtualization is fundamentally changing the architecture of the tire while adding clear and present business value to the owner of the car. Where once our wheels were white walls with an inner tube and often they lost air, went flat and thus we carried around a full size spare in the trunk. We now have tubeless tires that resist punctures, self heal, alert the driver to low tire pressure, can be filled with Nitrogen to improve MPG and the full size spare has been condensed into a tiny little backup we seldom need to use. And when was the last time you needed to replace your everyday tires with special "snow tires" when the going got tough? While the wheel is still indeed round, today's tires, provide better reliability, are agile enough to handle changing conditions, improve energy usage, run longer and you need less full size items. So it's true the wheel remains round. However, what virtualization is changing, underneath the OS stack, is dramatic. Our tires will still be round, but with the value of virtualization, they sure run better, stay inflated longer, need to be replaced less often and decrease the amount of power I consume while keeping my business moving forward. Virtualization - the shape remains the same, while the value increase, or as others have more eloquently stated "Virtualization is the most non-disruptive, disruptive technology". Let's not lose site of the car while we're kicking the tires :-)

Re: Drivers -- below the OS?

Posted Aug 10, 2007 11:45 UTC (Fri) by landley (guest, #6789) [Link]

> while adding clear and present business value

Then you go on to mention "adding value" twice more, and then quote The
Innovator's Dilemma. It's a good book, which is why I reviewed it for
The Motley Fool in 1999:
http://www.fool.com/portfolios/rulemaker/1999/rulemaker99...

However, my question to you is, what's with all the pointy hair? Linus
is talking technology, and you're talking about marketing to corporate
customers. How are you _not_ making his point for him?

Adding layers neither reduces complexity nor adds stability. It never
has. It didn't with microkernels, it still doesn't now, and on a
conceptual level it's unlikely to start. One runaway driver does a DMA
to the wrong physical memory or lets a device lock the PCI bus and never
release it, and it's game over. Virtualization allows IBM to come up
with yet more glossy brochures to push mainframes as a replacement for
clusters of PCs (although the price of electricity and air conditioning
in rackmounted space has been making that case for them for years). But
for the millions of laptops out there, there's never enough RAM and
chopping it in half to share between two OS images is not an improvement.
(Of course I extensively use QEMU's ability to emulate Arm and Mips and
such, but I'm weird. :)

P.S. A few months ago I briefly had a HPaq laptop that was "built for
vista" and nothing else had drivers. (Ubuntu, Knoppix, nothing.) When I
returned it truthfully saying "Vista won't run any of my software and
nothing else recognizes this hardware", the salesbeing actually suggested
that I download Microsoft's Virual PC thing and run a copy of XP under
that, having Vista talk to the actual hardware. This was apparently a
common problem Best Buy was seeing, and that was the recommended
solution. I paid the 15% restocking fee to give the darn thing back, and
got a Dell preinstalled with Ubuntu, which I've been fairly happy with.

Re: Drivers -- below the OS?

Posted Aug 10, 2007 21:29 UTC (Fri) by heckler (guest, #46684) [Link]

Sadly, I've not read "The Innovator's Dilemma", perhaps I should (nice review by the way)... My overuse of the word value is probably an alliteration thing with virtualization...

The real issue here is, why would an astute technical person purchase such a limited machine from Best Buy? Likewise, why would an astute technical person confuse host based, bare metal based and pv based virtualization? Not to mention a lack of understanding for the architectural differences between where Xen based and the upcoming Viridian hypervisor places and handles drivers, - vs- how something like VMware's ESX handles drivers. In the OS based (Xen/Viridian), all your VMs on a box can get locked via misbehaving driver interaction as you state. Thankfully, with something like ESX that won’t happen. As for memory and chopping it in half or writing to the wrong location, have you considered shadow tables, memory overscription and balloon drivers found in some of VMware’s products. I'll not go into the elegant stuff they do in ring0, the world switch, and the efforts being done with AMD/Intel in ring-1. Doing so could hamper the growth of some pointy hair on my balding head :-)

> Adding layers neither reduces complexity nor adds stability. It never has.

Never is such a long time when you look backwards. Looking forward, perhaps the days of a bloated Os are coming to a close. Without endorsing it, perhaps some of the "layers" BEA's adding to the stack, owing to virtualization, might give you pause to rethink your statement. (http://www.bea.com/framework.jsp?CNT=index.htm&FP=/co...) In such a stack, I now have a ring-1 perhaps, a virtualization layer, an “OS shim” within their Liquid VM, a JVM and finally my app. Lot’s of layers, lot’s of stability. And it’s a lot easier to develop apps, debug and admin the system in that environment then when I wrote assembly code, on a microkernel on top of an AN/UYK-20

Lastly, get a MAC and stop dinkering around with dell, HP, etc. Then go get VMware's Fusion for the other 60 or so OSs that product will allow you to run. Perhaps they'll throw in a pre-IPO stock share....

Re: Drivers -- below the OS?

Posted Aug 13, 2007 7:57 UTC (Mon) by himi (guest, #340) [Link]

Virtualization allows IBM to come up with yet more glossy brochures to push mainframes as a replacement for clusters of PCs (although the price of electricity and air conditioning in rackmounted space has been making that case for them for years).

I think this is the thing that makes Linus' comment a bit strong - virtualisation isn't going to revolutionise the way we use computers, but that doesn't mean it's the kind of fad/theoretical ideal/whatever that Linus describes it as. It really does have serious practical uses, and it's important technology. The hype that may be out there about it doesn't invalidate that.

Your point about laptops is excellent, though, and it's one of the reasons I don't see virtualisation becoming the norm outside of server rooms - it makes demands on the system (and on the administrator) that aren't practical in the normal computing space. I've deployed Xen for production infrastructure, but it's never been worth the trouble to use it anywhere else.

himi

Re: Drivers -- below the OS?

Posted Aug 13, 2007 18:38 UTC (Mon) by mday_ii (subscriber, #25315) [Link]

"So next time you hear about hypervisors doing anything at all, ask
yourself where the message comes from. Does it come from a group of people
who are desperately trying to make themselves appear relevant? Or does it
come from forty years of actual real-life practice?"

It is just around 40 years ago that the IBM mainframe rolled out machine virtualization. Today Intel, AMD, storage and network vendors in the x86 market are introducing the same hardware virtualization technology that has forty years of actual real-life practice. Changes in processor features will force the OS to start using virtualization techniques.

Linus is right about device drivers in theory, but in practice VMware loads hardware drivers and the kernel is working with virtual devices. That is not the best design point, because drivers are not really robust enough, but it is the basis for a product that has about $1b in revenue and growing.

One difference between virtualization and message passing microkernels is that virtualization has been the basis of multi-billion $ hardware sales for decades - the x86 platform (and Linux) are late to the market with this same technology. Despite Linus' apparent bias, Linux will become a first-class virtualization infrastructure over the next few years. At least he deserves credit for allowing that to happen, even if he doesn't believe in it.

Copyright © 2007, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds