Adding BPF to blk-iocost

The blk-iocost controller was first covered here in 2019, when it was still known as "io.weight"; it was revisited in 2024. To summarize, it works by allocating a percentage of a block device's bandwidth to each control group. Every requested I/O operation is then evaluated and an associated cost, expressed in terms of how much of the device's time the operation is expected to take, is assigned. Each device has a virtual clock ("vtime"), as does each control group; executing an I/O operation advances the requesting group's vtime by the assigned cost. Operations are only dispatched to the device when the group's vtime is far enough behind the drive's vtime to accommodate them.

See this paper for more details on how blk-iocost works. One key point is that its operation is split into two parts. The calculation of the cost for each operation uses a number of pre-computed parameters in order to be as efficient as possible and not slow down the I/O operations themselves. A separate "planning" phase, which runs every few milliseconds, adjusts the weights of all of the control groups in order to reallocate bandwidth from groups that are not using their full allocation to others that are more busy. The planning phase also tweaks the rate at which the drive's vtime advances in an attempt to keep the drive fully busy — but not so busy that response time begins to suffer.

Cui's cover letter describes how the current cost model works and its shortcomings. To summarize: requests that are deemed to be sequential are assigned a relatively low cost, while those that look random are assigned a much higher (112x) cost. Sequential access is faster, even on solid-state drives, so a group that is performing mostly sequential operations should be entitled to more of them than groups performing random operations.

The problem, Cui says, is that the heuristics for determining the randomness of any given request do not work well for all workloads. A group that has two processes performing sequential operations can, if those operations are interleaved, appear to be doing random I/O; the controller will penalize that group, causing its I/O performance to suffer. Meanwhile, random operations within a sufficiently small window can be judged to be sequential, leading to an unfair boost for the relevant group. Additionally, some devices can behave differently depending on the type or the placement of an I/O operation, but blk-iocost does not take that into account. Given the range of devices and technologies out there, it may not even be possible for a single, general-purpose controller to take all of the relevant factors into account.

The solution, as always it seems, is to enable the loading of a BPF program so that the cost calculation can be customized. Cui's patch set only addresses the cost-calculation part of the controller; the planning portion is unchanged.

Specifically, the series enables the loading of a new type of struct_ops program built around this structure:

    struct iocost_model_ops {
	u64 (*calc_cost)(u64 opf, u64 nbytes, sector_t sector,
			 struct blkcg *blkcg, u64 model_flags);
	void (*blkcg_online)(struct blkcg *blkcg);
	void (*blkcg_offline)(struct blkcg *blkcg);

	/* model name, used to select the model through io.cost.model */
	char name[16];
    };

The core of this structure, and the only required operation, is calc_cost(), which calculates the cost of an I/O request. The opf parameter is a set of flags describing the request, including whether it is a read or a write, whether a cache flush is needed, and so on. nbytes is the size of the request, sector is the location of the desired blocks on the drive, blkcg is the block control group that controls the requesting process, and model_flags may contain a single flag, IOCOST_COST_F_MERGE, indicating that the current request is being merged into another that already existed. The return value is the cost that will be assigned to this request.

The other two functions, blkcg_online() and blkcg_offline(), are optional; if they exist, they will be called when this controller is added to (or removed from) a control group, allowing the BPF program to manage any housekeeping it may need. Loading the program does not, in itself, attach the new cost function to any devices; that is done by writing to the io.cost.model file in the control-group directory using the name provided in the iocost_model_ops structure.

Included in the patch set is an example program that attempts to improve performance for control groups that contain multiple processes performing sequential I/O. It uses a BPF map to keep track of each I/O stream that it observes; if a request looks sequential within any of the tracked streams, it is charged as a sequential request. The example program is able to track up to four independent streams, though the number of streams actually tracked will be lower if there is also random I/O being done.

This is still early-stage work, in its third RFC revision. The interface has evolved considerably since the posting of the first version on September 8. So it would not be surprising to see further changes before this feature is merged. Eventually, though, a version of this work does seem likely to be accepted, adding yet another place where BPF can be used to modify the operation of a core kernel subsystem.