Tightening symbol exports

The kernel's loadable module mechanism does not give modules access to all parts of the kernel. Instead, any kernel symbol which is intended to be usable by loadable modules must be explicitly exported to them via one of the variants of the EXPORT_SYMBOL() macro. The idea behind this restriction is to place limits on the reach of modules and to provide a relatively well-defined module API. In practice, there have been few limits placed on the exporting of symbols, with the result that many thousands of symbols are available to modules. Loadable modules can access many of the obviously useful symbols (printk(), say, or kmalloc()), but they can also get at generic symbols like edd, tpm_pm_suspend(), vr41xx_set_irq_trigger(), or flexcop_dump_reg().

There are reasons for the concern over excessive symbol exports felt by some developers. Wrongly exported symbols can lead module authors to use incorrect interfaces; for example, the exporting of sys_open() is an active inducement for developers to open files directly inside the kernel, which is almost never a good idea. But such symbols, once exported, can prove hard to unexport. While the official line says that the internal kernel API can change at any time, the truth of the matter is that at least some developers are reluctant to break external modules when that can be avoided.

A more timely example would be init_level4_pgt, a low-level symbol exported only by the x86_64 architecture. The current -mm tree removes that export, breaking the proprietary NVIDIA module in the process. Andrew Morton describes this removal as "our clever way of reducing the tester base so we don't get so many bug reports." While many developers make a show of not caring about binary-only modules, there is still a good chance that this particular export removal (of a symbol which should not really be available globally) may not make it into the mainline as a result of this breakage.

The end result of all this is that there has long been interest in somehow cleaning up the modular API, though there have not been a whole lot of people who have put a lot of time toward that end. Occasionally somebody has remarked upon one piece of low-hanging fruit: symbols which are exported only to make it possible to modularize other bits of mainline kernel code. One example is a whole set of TCP stack symbols (things like __tcp_put_md5sig_pool()) which have exactly one user: the IPv6 module. Restricting these special-purpose exports has the potential to significantly narrow the modular API without making it harder to modularize the mainline.

Andi Kleen's module symbol namespace patch is meant to enable just this sort of narrowing of the API. With this patch, symbols can be exported into specific "namespaces" which are only available to modules appearing on an associated whitelist. In a sense, the term "namespace" is a poor fit here; there is still a single, global namespace within which all exported symbols must be unique. These "namespaces" are more like special exclusion zones containing symbols which are not globally accessible. They work like GPL-only exports, which also restrict the availability of symbols to a subset of modules.

To create a restricted export, an ordinary EXPORT_SYMBOL() declaration is changed to:

    EXPORT_SYMBOL_NS(namespace, symbol);

Where namespace is the name of a restricted symbol namespace. So, going back to the TCP example, Andi's patch contains a number of changes like:

    -EXPORT_SYMBOL(__tcp_put_md5sig_pool);
    +EXPORT_SYMBOL_NS(tcp, __tcp_put_md5sig_pool);

Note that there is no _GPL version; any symbol which is exported into a specific namespace is treated as GPL-only by default.

The other part of the equation is to enable access to a namespace. That is done with:

    MODULE_NAMESPACE_ALLOW(namespace, module);

Such a declaration (which must appear in a module exporting symbols into the namespace) says that the given module can access symbols in that namespace. Andi's patch creates three namespaces (tcp, tcpcong for congestion control modules, and udp), removing about 30 symbols from the global namespace.

A number of developers welcomed this patch, seeing it as a step forward in the rationalization of the loadable module API. It is seen as a way to prevent out-of-tree modules from using symbols which they should not be using. It also reduces the number of interfaces which must be kept stable in situations (enterprise kernels, for example) where changes are not allowed. And, finally, the symbol namespaces offer the ability to organize exports somewhat and document who the intended users are.

There is a bit of dissent, though. In particular, Rusty Russell fears that the patch adds unneeded complexity and threatens to make life harder for out-of-tree developers for little (if any) gain. Says Rusty:

Herbert Xu has similar concerns:

While these voices seem to be in the minority, they still carry quite a bit of weight. So your editor is unwilling to make any sort of guess as to whether this patch will be merged, or in what form. The desire to clean up the modular API is unlikely to go away, though, so, sooner or later, something is likely to happen.

---

(Log in to post comments)

I think the symbol export system in general is a bad thing for Linux. People are removing
exports to save memory and restrict access. It breaks drivers all the time. I had to use a
driver in the kernel tree, which didn't work because it missed a symbol. Somebody had tried to
be too smart about putting #ifdef's around EXPORT_SYMBOL to save a few exports.
Trying to restrict access like that only gives a lot of pain and in the end very little gain.

I worked with VxWorks in an earlier job. It exported _every_ global symbol. There where no
administrative issues associatated with putting in EXPORT statements. Why can't Linux just do
the same? There are 3 reasons, for all I can find a counter argument:

1) Memory. Storing symbols take a lot of memory. Solution: Let the symbols be stored in
swapable memory, for instance by moving the linking of modules of to a user-space deamon,
which keeps track of what is loaded and where. In embedded systems that wouldn't help much,
but there I believe you shouldn't use modules anyway. And even in VxWorks running on a very
memory constrained system, we didn't have problems before we started to use advanced C++
templates auto-generating a lot of very long symbols. Linux wouldn't have that problem as
there is no C++ in the kernel.

2) Restrict access for non-GPL symbols. This is legally nonsense. The end user are allowed to
load whatever into his kernel as long as he doesn't distribute the result. It is not within
the spirit of free software to block him from doing that. On the other hand, nobody can
distribute Linux along with non-GPL'ed modules as that is against GPL.
EXPORT_SYMBOL/EXPORT_SYMBOL_GPL doesn't change any of that.

3) Trying to mark some interfaces as more stable than others by exporting those and not the
others. Either you make a stable driver API or you don't.   Having a stable driver API
wouldn't allow non-GPL drivers anyway and is therefore not needed as the GPL'ed drivers should
put into the kernel tree once they are stable. Make it possible to run drivers in user-space
instead.
In general I don't think you should try to block people from using anything. It is against the
spirit of free software. On the contrary you should help people as much as possible. Therefore
almost all symbols should be accessible but of course you can try to help the coder writing
drivers, within or outside the kernel tree, choose the right symbols, by for instance sorting
the declarations in public and private header files.

From the point of view of developers who want to use these functions, I see your point.  But
from the point of view of developers who actually write these functions, I think it is totally
unacceptable to export all symbols.  What it means is that you don't know what you will break
when you make whatever modification.  In other words, with the EXPORT_SYMBOL mechanism,
developers can at least have some idea about what will break by the changes they want to make,
so they can try their best to minimize breakage and plan them ahead when needed.  Without the
mechanism, all they can do is to break their interface completely randomly, which I don't see
is a good service to module writers.  While the EXPORT_SYMBOL mechanism does have rough edges,
I think it is essential for keeping things maintainable even though the kernel
functions/interfaces are made and used by people who are not known to each other.

In practise EXPORT_SYMBOL has the complete opposit effect: Developers remove EXPORT_SYMBOL and
a lot of stuff breaks even though there is no _technical_ reason for it to break.

The rule when doing a change to the kernel is: You have to be able to compile the kernel and
all drivers after you changed stuff (which of course is unrealistic as that involves all
architectures, but I do hope that it is done before any "official" release.) The presence of
EXPORT_SYMBOL doesn't change that, there could still be non-module stuff which depends on what
you are changing. As there specifically is no stable API to outside drivers you do not have to
think about breaking stuff outside the kernel anyway.

The whole issue of using EXPORT_SYMBOL to limiting access is wrong. It was probably made to
avoid exporting everything for mere technical reasons (limiting memory). Using the same
mechanism to limit access is one of the typical hacks where you have one mechanism and abuse
it for something else. In the long term that kind of stuff only give problems.
I simply can't see why the access problem should in any way be related to the module boundary.
What you want is something along "public", "protected", "package". The one way I know to do
that in C is to take care about declaring the things in the right header files. Notice that
even Microsoft didn't connect public memmbers in C++ classes with DLL_EXPORT or visa versa.
The two things are orthogonal properties.

> As there specifically is no stable API to outside drivers you do not have
> to think about breaking stuff outside the kernel anyway.

I think most people would consider a symbol involved in EXPORT_SYMBOL to be "somewhat stable",
i.e., yes it breaks from time to time, but nobody would expect it to break every minor kernel
version.  Also, such a mechanism would save developer from having to search in the whole
kernel source tree after making whatever minor change to function prototypes.

> The whole issue of using EXPORT_SYMBOL to limiting access is wrong. It was
> probably made to avoid exporting everything for mere technical reasons
> (limiting memory).

I don't think this is a case for any technical reason: after all, kernel modules can easily be
loaded with user-mode helpers which will resolve all the symbols needed, and the symbol
information is easily made available if not already available via the System.map.

What about:

4) Security. Not exporting symbols that are not needed by drivers makes writing rootkits and a
like a lot harder?

But it doesn't. If something hostile is running as root you're dead. If it 
can get access to the symbol table from somewhere, it *really* doesn't 
matter if the symbol is exported or not.

Err, that's some pretty good nonsense you're spouting there.

Legal stuff isn't black and white. Intent matters. EXPORT_SYMBOL_GPL is a declaration of
intent stating that whoever uses that symbol will be assumed to be making a derived work of
the linux kernel. 

Claiming your work is GPL (by putting a MODULE_LICENCE() statement in) when it clearly isn't
is bound to get you hip deep in the muck in the courtroom if there were to be a GPL
infringement case against you. You just deliberately ignored a statement that certain symbols
were only available to GPL modules, so any infringement was willful making you liable for
triple damages. You just either a) fraudulently claimed your module was GPL or b) fraudulently
claimed it was not when it clearly is licensed GPL according to your own code! 

Of course IANAL - but I've read enough groklaw to understand that courts don't take kindly to
attempts to circumvent other people's rights, regardless of how smart they may sound to you.

> I've never been entirely sure what's to stop those people from simply putting
MODULE_LICENSE("GPL") in their non-GPL code.

It's been done: http://lwn.net/Articles/82306/

(Version 1.14 of Linuxant's hfpcimodem still contains the GPL\0not-really BS.)

>for example, the exporting of sys_open() is an active inducement for developers to open files
directly inside the kernel, which is almost never a good idea.

sys_open ha - if you really were to open files within the kernel, then better with
filp_open(), as it takes a kernelspace string and does not need the set_ds() cruft. Though,
you will have to assign the fd yourself - that is, IFF you plan to do that, and in most cases
(of the cases that you even /want/ to open files in the kernel), you do not.