Quotes of the week [LWN.net]

Quotes of the week

Posted Jul 2, 2009 8:33 UTC (Thu) by MisterIO (guest, #36192)
Parent article: Quotes of the week

It'd be interesting if someone could explain to me Torvalds' statement. In particular, what other notions could the VM be designed around? What do we gain with the current design with respect to the others? And why would the VM have very little freedom if it were designed differently?

to post comments

VM options

Posted Jul 2, 2009 10:44 UTC (Thu) by neilbrown (subscriber, #359) [Link] (3 responses)

I don't completely understand Torvalds' statement either, but I think I can make some stabs in the dark...

I think you could look at a VM as following either the JIT or ASAP principle. A Just In Time VM would clean dirty pages by writing them to backing store (either files or 'swap') Just In Time - when there was no other way to find clean pages. An ASAP would write pages out As Soon As Possible - or at most after a short delay.

I think Linus is saying the Linux tends to the ASAP principle, so that there is never much dirty data. That is certainly true of file-backed pages - as soon as 20% of all pages are dirty, the VM will write dirty pages out quite agressively and will block further pages from being dirtied until the percentage drops a bit.

For anonymous pages which have to be written to swap, I'm less sure of what happens. I think they are written less urgently than file backed pages, but I don't know the details and would love to have someone summaries them for us.

The tricky issue is that you might need to allocate memory while writing data out to storage - so if you leave the writeout until there are almost no free pages, you could deadlock: you cannot write a page until you can allocate memory, and you cannot allocate memory until a page has been written.

This is where pre-allocation comes in. The way to avoid the deadlock described above is to pre-alloc enough memory so that you can always be sure of being able to write out one page, possibly having to wait for any pending writeouts to complete, but nothing more.

And that is where I don't understand Linus' comment. He says that Linux "generally" doesn't preallocate, yet there are lots of places in the writeout path that do preallocate. This is exactly what the "mempool" routines are for. And the swap-over-NFS patches that (I think) recently went in do even more pre-allocation.

I think Linux largely manages to avoid preallocation at the filesystem level, but one step lower in the block level there is plenty of it.

For your other questions - what we might gain with an ASAP VM design is reduced complexity with preallocation, and I guess it could be worse than it is in Linux.

And the 'freedom' is about having lots of free memory in a common pool rather than in lots of little pre-alloc pools, so the VM can use that common memory for whatever need pops up at the time.

So I think I can see where Linus is coming from, but I don't completely believe him, so I suspect there is something that I am not understanding correctly.

VM options

Posted Jul 2, 2009 13:18 UTC (Thu) by MisterIO (guest, #36192) [Link] (2 responses)

Thanks a lot for the explanation!

Tridge makes an important point

Posted Jul 2, 2009 16:27 UTC (Thu) by JoeBuck (subscriber, #2330) [Link] (1 responses)

Microsoft no longer has the freedom to break competitors' code that it used to have. The wakeup call for them was the massive pushback they got from the corporate world over Vista: most corporate users just refused it. It had some snazzy new features, but it broke stuff, was bloated, and seen as a downgrade in some respects.

Corporate networks aren't all-Microsoft very much. They have a mix of devices, and some use Samba. If Microsoft's new MegaOS doesn't work with Samba, it won't go into the corporate network. Likewise, Linux is the cheapest thing around and is going into a lot of embedded devices. If Microsoft's Service Pack 23 breaks people's digital cameras, word will get out and people won't install it.

Linux has more power than it used to have, because it's getting to the point where it's in everything. We really need to get some surveys done showing just how many copies of Linux the average person in various countries has. Many people have a router that runs Linux, a set-top box or DVR that runs Linux, a cell phone that runs Linux, and a digital camera that runs Linux. Microsoft can't break all that, or people will stick with the old release of Microsoft that interoperates.

Tridge makes an important point

Posted Jul 6, 2009 15:08 UTC (Mon) by Hanno (guest, #41730) [Link]

MS don't want to "break" Linux. They want to charge a fee from the manufacturers and/or distributors of people's digital cameras, routers, set-tob boxes, dvrs, cell phones.

"Nice product you have there. Wouldn't it be sad if we had to sue you over licensing?"