Merging zswap [LWN.net]

By Jonathan Corbet
May 22, 2013

As reported in our Linux Storage, Filesystem, and Memory Management Summit coverage, the decision was made to merge the zswap compressed swap cache subsystem while holding off on the rather more complex "zcache" subsystem. But conference decisions can often run into difficulties during the implementation process; that has proved to be the case here.

Zswap developer Seth Jennings duly submitted the code for consideration for the 3.11 development cycle. He quickly ran into opposition from zcache developer Dan Magenheimer; Dan had agreed with the merging of zswap in principle, but he expressed concerns that zswap may perform poorly in some situations. According to Dan, it would be better to fix these problems before merging the code:

I think the real challenge of zswap (or zcache) and the value to distros and end users requires us to get this right BEFORE users start filing bugs about performance weirdness. After which most users and distros will simply default to 0% (i.e. turn zswap off) because zswap unpredictably sometimes sucks.

The discussion went around in circles the way that in-kernel compression discussions often do. In the end, though, the consensus among memory management developers (but not Dan) was probably best summarized by Mel Gorman:

I think there is a lot of ugly in there and potential for weird performance bugs. I ran out of beans complaining about different parts during the review but fixing it out of tree or in staging like it's been happening to date has clearly not worked out at all.

So the end result is likely to be that zswap will be merged for 3.11, but with a number of warnings attached to it. Then, with luck, the increased visibility of the code will motivate developers to prepare patches and improve the code to a point where it is production-ready.

(Log in to post comments)

Merging zswap

Posted May 31, 2013 3:40 UTC (Fri) by dakas (guest, #88146) [Link]

I think there is a lot of ugly in there and potential for weird performance bugs. I ran out of beans complaining about different parts during the review but fixing it out of tree or in staging like it's been happening to date has clearly not worked out at all.
So the end result is likely to be that zswap will be merged for 3.11, but with a number of warnings attached to it. Then, with luck, the increased visibility of the code will motivate developers to prepare patches and improve the code to a point where it is production-ready.

An engineering decision made under a premise "Then, with luck" is not what I call sound. In particular, if "with luck" involves the hope that someone(TM) will do something(TM).

If we follow the given premises, there are several conclusions:

It is quite probable that luck will not hold to the desired degree and the code will not become production ready.
But it is also probable that "luck" will not be as bad as to have nobody at all trying to fix things.
While things are getting worked on, other things will get worked on as well.

The net results from those points are that neither will production readiness be attained, nor will it be easy to revert the half-baked changes. So a lot of pain is intended to be inflicted in the hope that someone else will feel hurt enough to fix what has been designed beyond the resources (mental and/or time) of the original authors. That assumes that the authors' kernel coding abilities are sub-par, but the underlying design is superior to what others could come up with so that the whole is a net win.

Basically, it's a plan with the requirement of a miracle out of the authors' control designed into it.

Now nobody wants to rain on anybody's parade, but planning the parade across a river without bothering to look at the locations of bridges beforehand is likely to result in some wet pants.

Off by default.

Posted Jun 2, 2013 6:28 UTC (Sun) by gmatht (guest, #58961) [Link]

Rik van Riel suggested to merge it, turned off by default. That way, people can easily try it out and see if it is a win in their use case. This would provide immediate benefit to some and would give them more feedback on actual performance regressions to work on. By way of comparison, switching to Btrfs won't necessarily be a win either, and even having merged Btrfs trying out Btrfs is more than a quick sysctl.