User: Password:
Subscribe / Log in / New account

Which I/O controller is the fairest of them all?

Which I/O controller is the fairest of them all?

Posted May 13, 2009 4:50 UTC (Wed) by jejb (subscriber, #6654)
Parent article: Which I/O controller is the fairest of them all?

It's not quite correct to say that only Vivek was in the room at the storage summit: Fernando Cao presented the state of dm-ioband and the reasons for wanting it quite eloquently. What was missing was io-throttle.

dm-ioband solves exactly the problem of allocating bandwidth to virtual machines because it operates at the VM level. The problem is that it's too high up in the stack efficiently (it's many layers away from the physical I/O driver, which is where the actual bandwith allocations are done). io-throttle looks to be substantially similar to dm-ioband in this regard, so it would suffer from the same problem. Conversely, io-controller can regulate exactly and efficiently the bandwidths at the physical layer, but it's too far away from the virtual machine layers to ensure exact compliance with VM based limits (in fact one can prove cases where one VM can run away with all the bandwidth).

Where I thought we'd got to at the storage summit was an appreciation that the physical layer is the correct one to regulate at (being the closes to the device) but that to solve the VM bandwidth allocation problem, we'd also need much of the machinery of dm-ioband (the page tracking and I/O accounting per VM) then what it would do is periodically apply corrections to the static limits of io-controller to ensure the per VM limits were respected (a bit like the way irqbalanced operates to keep interrupts well distributed).

Unfortunately the agreement seems to be misphasing again ... fortunately I can say this is Jens' problem now ...

(Log in to post comments)

Which I/O controller is the fairest of them all?

Posted May 13, 2009 14:23 UTC (Wed) by vgoyal (subscriber, #49279) [Link]

A higher level software (a daemon or something else) doing dynamic weight adjustments for the groups at physical device level to achieve the bandwidth goal at virtual block device sounds like a good idea. It might be little complicated to implement though. :-). At LSF we sort of had the agreement to go in that direction but dm-ioband developers have to respond to this scheme of things and see if it satisfies their requirements or not. If it does, then probably they can start development on daemon for weight adjustment while we stabilize the IO scheduler based io controller.

I am not sure what kind of storage configurations are common but IO scheduler based solution alone should just work fine for all kind of Hardware RAID solutions and for disks directly attached to system without any software RAID. The only problematic case seems to be software RAID where bandwidth allocation will take place at higher level logical device.

With-in software RAID also, one needs to figure out which configurations are of particularly of more concerns. For striped ones, may be it is fair to assume that IO is evenly distributed across the various disks and if we can provide proportional bandwidth on individual disk, it should also translate into proportional bandwidth division for logical device.

VMs, and multiple disks priorities

Posted May 14, 2009 7:26 UTC (Thu) by zmi (guest, #4829) [Link]

I was missing virtual machines (XEN, VMware, ..) from the article, just
found above user comment mention it. Isn't the need for I/O controllers
very urgent when you start to use VMs? For a normal machine, if it's I/O
intensive you normally run only one application on it (database, webserver,
SAP) and don't really need control - just a faster disk subsystem.

But with multiple VMs running, the need to prioritize them grows, and if an
I/O controller gets into the kernel, it should help here. XENserver (which
is free since April) already allows priorities for VMs/disks, I wonder how
they implemented it?

Also, the other place where I see need of priorities for different mount
points. Each mount point should get a priority, so that you can say
/importantdb has higher priority than /standarddb. They could go to the
same disk subsystem, or another, but the thing is I want I/O on
/importantdb to not block because of /standarddb doing it's backup.

Copyright © 2018, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds