CloudLinux busted — twice

Proprietary kernel modules have always had an unclear status in the kernel development community. Most (but not all) developers seem to agree that it is possible to make a module that is not a derived product of the kernel itself (and, thus, not subject to the requirements of the GPL); the Andrew filesystem, ported from Unix in the early days, has been cited as an example of this kind of module. Most developers also seem to agree that many other types of modules cannot be written as an independent work. There is far less agreement on where the line should be drawn, but there is a consensus around the idea that modules should not lie about their licensing status to gain closer access to kernel internals. The story of a company that was recently caught shipping that sort of dishonest module shows the kind of reaction that can be expected—and why closed-source modules are a bad idea in general.

Loadable kernel modules can only access kernel symbols that have been explicitly exported for that use. Two types of declarations are used to export these symbols. EXPORT_SYMBOL_GPL() is an indication that any module using the symbol is reaching so deeply into the kernel that it is, by necessity, a derived product of the kernel. Instead, EXPORT_SYMBOL() is used for symbols that only might indicate a derived-product status; it is also used for symbols that were already exported before the GPL-only mechanism was added.

The kernel's module loader includes an enforcement mechanism that prevents any module without a GPL-compatible license from accessing explicitly GPL-only symbols. To function, this mechanism clearly must know what the license for a specific module is. That is the weak point of the entire scheme: the kernel uses the honor system. Loadable modules include a special declaration that tell the kernel which license applies; the module loader looks at that declaration and decides whether the module will have access to GPL-only symbols or not. If the module lies about its license, the kernel will normally trust it.

That is where CloudLinux comes in. This company offers a CentOS-like distribution aimed at the needs of cloud computing providers. Essentially, CloudLinux has taken a RHEL clone, added its own container mechanism, and created a business around providing updates and support. The container mechanism is at least partially implemented by the company's "lve" kernel module; lve includes an access control and resource limit mechanism that, seemingly, is implemented independently of the kernel's control group subsystem. It can place limits on CPU usage, I/O bandwidth usage, and memory consumption, among other things; all this is managed through an administrator-friendly XML file.

Matthew Garrett recently became aware that, while the lve module claims to be GPL-licensed, there is not any source available for it; he responded with a patch causing the kernel to recognize and blacklist this module. Since lve uses GPL-only symbols, this change will prevent it from loading, essentially breaking it. This change has yet to be pushed into the mainline, but it has been accepted by the modules maintainer (Rusty Russell) and marked for stable updates as well. Unless something changes, it will soon become impossible to load the lve module into any current kernels.

The immediate effect of this change on CloudLinux is likely to be limited. Chances are that few users download the binary lve module and load it into their own kernels; almost all lve users are probably running the entire CloudLinux distribution. CloudLinux could easily patch its own kernel to remove the check for the lve module and restore the previous functionality. But such a move would almost certainly be noticed, and there is a distinct possibility that one or more kernel developers would react in an unpleasant way. It would not be an advisable course of action.

What CloudLinux did instead was to apologize and plead for time:

This response was not entirely well received; it is not at all clear why CloudLinux is unable to immediately ship the source corresponding to the binary module it is distributing. That said, the community will probably limit itself to grumbling as long as CloudLinux follows through and comes back into compliance with the kernel's licensing. Nobody likes a delay in a source release, but short delays are usually tolerated, grudgingly.

So the licensing part of this episode is, probably, resolved, but it is also interesting to look at why the lve module needs GPL-only symbols in the first place. The specific symbols it needs relate to the creation of binary attributes (essentially unstructured sysfs files) in the sysfs virtual filesystem. Much of the internal sysfs interface is GPL-only, and the functions for the management of binary attributes (which are generally discouraged) are certainly so. It turns out that lve creates binary attributes for a reason that would certainly never get through a proper review:

As a comprehensive security solution, this implementation leaves just a little bit to be desired.

CloudLinux is trying to reduce the amount of damage that could be done should an attacker successfully compromise a daemon running on the system. Kernel developers have been working for years to mitigate just this sort of threat; the results of that work take the form of security modules, namespaces, and more. Had CloudLinux done its work in the open, it would certainly have been directed toward solutions that have support from others and years of experience in actual use. Solutions, in other words, that might actually deliver some additional security. Instead, they got a one-off "not pretty hack" that could never get upstream.

If CloudLinux follows through on its plan to close the module entirely, it will have to cease use of GPL-only symbols and, thus, will not be able to implement this particular mechanism. That may be enough to convince kernel developers to leave it alone. But it should leave others wondering what other surprises may be lurking inside—surprises that cannot be investigated, since the source for the module is not available.

There may well be a viable business in the creation of a well-supported distribution aimed at the needs of cloud providers. With solid security support and some top-tier user-space management tools, a company like CloudLinux could well produce an offering that others are willing to pay for. But the most robust and trustworthy server distribution will come from using (and improving) the work that many developers have done for years to create a useful containers implementation. It is hard to imagine that a proprietary solution put together by a small company can work as well. Compared to that, the risk of licensing troubles seems like a case of adding insult to injury.

---

(Log in to post comments)