LWN: Comments on "In defense of per-BDI writeback"

In defense of per-BDI writeback

anton — Sun, 11 Oct 2009 12:59:13 +0000

I did use cp.

In defense of per-BDI writeback

mfedyk — Sun, 11 Oct 2009 06:30:43 +0000

I was doing a similar test, but over network filesystems (cifs in this case -- centos5 on both sides).

If I copied files with cp or mv, I noticed a marked improvement in throughput compared to the gnome file manager.

Try it again with mv or cp and see if there is a difference.

In defense of per-BDI writeback

anton — Sat, 03 Oct 2009 15:00:13 +0000

The fundamental problem with pdflush is that it would back off when the backing device appeared to be congested.

That might explain the huge slowdowns I saw (on Linux 2.6.19 and 2.6.27) when writing several GB to flash devices. One was a pretty fast 8GB SD card (SDHC class 6 (i.e., >6MB/s on a certain workload), and I typically saw >10MB/s when writing several hundred MB), yet it took several hours to fill up; I no longer remember if the system also suffered in other ways. Calling sync now and then seemed to help, but the whole thing still took a very long time.

I do not think that the problem was in the flash device, because it was originally new (no need to shuffle old data around), the slowdown occured pretty soon (not only near the end), and various measures taken at the host end helped (like invoking sync, or writing the data in smaller batches which syncing in between).

I had a similar experience when trying to fill my 8GB ogg player with music, except that this device was slow to begin with (3MB/s when writing a few hundred MB), but filling it up still should not have taken 8 hours (280KB/s).

In defense of per-BDI writeback

peter_w_morreale — Thu, 01 Oct 2009 14:54:37 +0000

There were other problems with the old writeback code - most especially if you had storage devices of varying throughput on the same system.

The old writeback code traversed super blocks in order, skipping over those currently congested and without regard to the throughput of the devices backing the supers. Recall that the old writeback code/pdflush indiscriminately issues writes until the memory threshold is reached.

This could have (and probably did) lead to possible performance penalties for applications referencing the *fast* devices while consequently improving the performance of apps on the slow devices. It certainly lead to unfairness issues wrt who dirties memory and who cleans it.

Consider the followed kludged example to illustrate the point. Two apps, both dirtying pages at the same rate, one app backed by a "fast" device, the other by a "slow" device. Both apps are contributing to the dirty page count at the same rate, so now pdflush and writeback are kicking in.

Since the slow device will maintain a "congested" state longer (since it is "slow"), the faster device will eventually account for more cleaning of pages than the slow device.

This has two effects:

1) Dirty pages for the app on the slow device potentially stay in memory longer and have a better chance of being re-referenced without I/O.

2) Dirty pages for the "fast" device are more likely to be written out and consequently require an I/O for re-reference.

So we wind up penalizing the app on the fast storage device. In theory at least. :-)

I haven't looked at the per-BDI code, but with such it is now possible to apply fairness to ensure that each device cleans its share of dirty pages. (Whether that is a good thing or not, I don't know, its just that it enables the capability.)

pdflush

axboe — Thu, 01 Oct 2009 08:33:24 +0000

pdflush() was not per-CPU, it didn't make any affinity guarantees of that nature.