|
|
Subscribe / Log in / New account

Heterogeneous memory management meets EXPORT_SYMBOL_GPL()

By Jonathan Corbet
June 12, 2018
One of the many longstanding — though unwritten — rules of kernel development is that infrastructure is not merged until at least one user for that infrastructure exists. That helps developers evaluate potential interfaces and be sure that the proposed addition is truly needed. A big exception to this rule was made when the heterogeneous memory management (HMM) code was merged, though. One of the reasons for the lack of users in this case turns out to be that many of the use cases are proprietary; that has led to some disagreements over the GPL-only status of an exported kernel symbol.

The HMM subsystem exists to support peripherals that have direct access to system memory through their own memory-management units. It allows the ownership of ranges of memory to be passed back and forth and notifies peripherals of changes in memory mappings to keep everything working well together. HMM is not a small or simple subsystem, and bringing it into the kernel has forced a number of low-level memory-management changes. After a multi-year development process, the core HMM code was merged for the 4.14 kernel, despite the lack of any users.

The immediate issue has to do with HMM's use of devm_memremap_pages(), which allows the mapping of pages that exist in device memory. Early versions of HMM used this function before switching to an internal version with some changes. Dan Williams recently posted a patch series adjusting devm_memremap_pages() and changing HMM to use it, getting rid of the duplicated code. That change is not controversial, but one other part of the patch set is: he changed the export declaration of devm_memremap_pages() to EXPORT_SYMBOL_GPL().

There are, of course, two ways to export symbols from the kernel to loadable modules, with and without the _GPL suffix. Symbols exported with that suffix will be unavailable to any module that does not declare a GPL-compatible license. It is a statement that, in the developers' belief, any use of those symbols will necessarily make the module a derived work of the kernel. In this case, the proposed changes will make it harder for proprietary modules to use HMM.

Jérôme Glisse, the author of HMM, is naturally opposed to this change, since it defeats part of the purpose for HMM in the first place. Dave Airlie has also questioned the change, noting that devm_memremap_pages() was exported normally for three years and wondering what has changed:

If something wasn't a derived work for 3 years using that API, then it isn't a derived work now 3 years later because you changed the marker. Retrospectively changing the markers doesn't really make any sense legally or otherwise.

Williams responded that the initial marking of the symbol was "an oversight" that is being corrected now. In support of the claim that any user of devm_memremap_pages() must be derived from the kernel, he pointed out that turning on this remapping capability changes the kernel fundamentally. The reverse of Airlie's logic also works: if a user of this functionality was a derived work of the kernel before, the non-GPL status of the export will not have changed that fact.

Williams further explained the reasoning behind his proposed changes as:

My concern is the long term health and maintainability of the Linux kernel. HMM exports deep Linux internals out to proprietary drivers with no way for folks in the wider kernel community to validate that the interfaces are necessary or sufficient besides "take Jerome's word for it".

The rest of his message perhaps gets closer to the real source of this particular dispute, though: the fact that there are no in-tree users of the HM functionality.

Every time I've pushed back on any HMM feature the response is something to the effect of, "no, out of tree drivers need this". HMM needs to grow upstream users and the functionality needs to be limited to whatever those upstream users exploit. Since there are no upstream users of HMM, we should delete it unless / until those users arrive.

Glisse has a response to all of these complaints. HMM, he says, is meant to isolate drivers from core memory-management internals rather than tying them together. There is a user now in the form of patches to the Nouveau driver for NVIDIA GPUs; he said he hopes to get that code upstream in 4.19. And upstreaming the pieces, he said, has been "a big chicken and egg nightmare" with a lot of independent pieces to prepare together; that has made it hard to get the users merged along with the infrastructure.

The merging of the Nouveau code, if and when it happens, should resolve the question of whether HMM should be in the kernel at all; it might reopen some questions about specific HMM interfaces, though. The question about the GPL-only export may prove harder to reach a conclusion on, though. There is no easy or objective standard for deciding whether the use of a specific kernel function makes a module into a derived work; it usually comes down to the judgment of the developers who wrote the code in the first place. In this case, those developers are Williams and Christoph Hellwig, who has stated that he is willing to enforce the GPL against users of devm_memremap_pages().

While a case could thus be made for changing the status of this symbol, it's not at all clear what will actually happen. Either Andrew Morton or Linus Torvalds will almost certainly end up making the final decision. It is more clear, though, that a number of developers are unhappy with the no-users status of HMM in the kernel. The most likely outcome of this particular episode may end up being a redoubling of the community's determination not to accept new subsystems into the kernel until users exist.

Index entries for this article
KernelCopyright issues
KernelMemory management/Heterogeneous memory management


to post comments

Heterogeneous memory management meets EXPORT_SYMBOL_GPL()

Posted Jun 12, 2018 19:45 UTC (Tue) by compenguy (guest, #25359) [Link]

I've long viewed EXPORT_SYMBOL* as an aid to self-auditing for mistakes/accidents, not as legal advice.

And while in the thread they were talking about the way it interacts with kernel internals as being a guide to deciding whether it should be EXPORT_SYMBOL_GPL or not, I think that doesn't align well with a reasonable copyright-informed interpretation.

I feel a better question to guide that decisionmaking might be "To what degree are the concepts and interfaces linux-specific?". This also aligns somewhat with Linus's stance that driver code written for other OSes and ported to Linux doesn't de facto derive from Linux (https://lwn.net/Articles/13398/).

Heterogeneous memory management meets EXPORT_SYMBOL_GPL()

Posted Jun 12, 2018 20:05 UTC (Tue) by Cyberax (✭ supporter ✭, #52523) [Link] (6 responses)

Oh, don't be such wussies. Just change it back to EXPORT_SYMBOL in your own fork.

After all, GPL enforcement for Linux is MIA.

Heterogeneous memory management meets EXPORT_SYMBOL_GPL()

Posted Jun 13, 2018 12:05 UTC (Wed) by Funcan (guest, #44209) [Link] (1 responses)

Sadly, unless you can convince Redhat, Debian and Ubuntu to use your fork, you've basically achieved nothing.

Heterogeneous memory management meets EXPORT_SYMBOL_GPL()

Posted Jun 13, 2018 17:47 UTC (Wed) by flussence (guest, #85566) [Link]

The first two have some integrity. Ubuntu practically bends over backwards to make filling the system with crapware as easy as possible.

That's how we ended up with the current swamp of proprietary and/or Electron-based software that advertise support for “Linux”, where in practice that usually means “Ubuntu $(($year - 6)), x86-32 only, good luck with anything else”. I don't think making concessions at this point will make that situation go away, unfortunately.

Heterogeneous memory management meets EXPORT_SYMBOL_GPL()

Posted Jun 21, 2018 7:27 UTC (Thu) by oldtomas (guest, #72579) [Link] (3 responses)

Oh, c'mon. By now I think everyone around here knows how much you hate the GPL.

Tell us something new, will ya?

Heterogeneous memory management meets EXPORT_SYMBOL_GPL()

Posted Jun 21, 2018 10:33 UTC (Thu) by Wol (subscriber, #4433) [Link] (1 responses)

I don't think Cyberax particularly hates GPL - he just doesn't consider it "fit for purpose". And I find I quite often agree ...

The other problem I see - I don't know if it's happening here - is that some people seem to have this need to relabel OTHER PEOPLE'S code as GPL. As far as copyrights and legalities are concerned, this can start to get VERY hairy, VERY fast. If I choose NOT to stick "GPL" on the end of my export, I don't want other people changing it.

Cheers,
Wol

Heterogeneous memory management meets EXPORT_SYMBOL_GPL()

Posted Jun 21, 2018 11:06 UTC (Thu) by anselm (subscriber, #2796) [Link]

The issue here is that in the kernel, some symbols (variables, functions, …) are marked as OK to use in non-free third-party modules, and a superset of those is marked as OK to use in GPL third-party modules. Sometimes it turns out that someone working on a non-free third-party module would really love to use one of the symbols that are only available to GPL modules. Since the only thing standing between such a developer and using the symbol is the missing EXPORT_SYMBOL_GPL declaration, Cyberax suggests that such a developer should go right ahead and relabel the symbol in their copy of the kernel, because nobody cares enough to police that sort of thing.

Personally I think if you do something like that and it comes out, as it inevitably will sooner or later, that will certainly not make you popular in the Linux kernel community. Whether that is an actual problem as far as you're concerned is for you to decide.

Heterogeneous memory management meets EXPORT_SYMBOL_GPL()

Posted Jun 21, 2018 11:00 UTC (Thu) by Cyberax (✭ supporter ✭, #52523) [Link]

I don’t hate GPL per se. I hate that it works against good corporate citizen, while rewarding bad behavior. And this won’t change unless the market is littered with steaming corpses of billion-dollar companies bankrupted by punitive damages from GPL violations.

The VMWare suit has shown us that GPL is pretty much unenforceable. That’s why I think that Apache-style licenses are,way better.

Heterogeneous memory management meets EXPORT_SYMBOL_GPL()

Posted Jun 13, 2018 1:37 UTC (Wed) by unixbhaskar (guest, #44758) [Link] (1 responses)

"....the core HMM code was merged for the 4.14 kernel, despite the lack of any users" Why?? any specific reason for that? Yet another thing to maintain for no apparent reason ...heck ...look like I am missing the big picture.

Heterogeneous memory management meets EXPORT_SYMBOL_GPL()

Posted Jun 13, 2018 11:51 UTC (Wed) by robclark (subscriber, #74945) [Link]

Work has been ongoing for a while for nouveau support, although it is quite non-trivial.. changes to kernel driver, ofc, but also bringing up clover/opencl on nouveau in userspace, lots of shader compiler infrastructure work, etc. So it is at least nice not to be having to carry around and rebase the HMM patchset in addition to all this.

Heterogeneous memory management meets EXPORT_SYMBOL_GPL()

Posted Jun 17, 2018 3:27 UTC (Sun) by benh (subscriber, #43720) [Link]

There are more users around the corner. Us (IBM) have a secure VM mechanism we are working on that relies on HMM to handle migration between secure and insecure memory for example. For various reasons, using HMM wouldn't have been an option for us had it not already been upstream (hint: distro dependencies and schedules) and it's saving us a huge amount of extra work and horrible hacking into the guts of the VM.

Heterogeneous memory management meets EXPORT_SYMBOL_GPL()

Posted Nov 12, 2018 16:12 UTC (Mon) by fxkuehl (guest, #128016) [Link]

I disagree with the claim that "many of the [HMM] use cases" are inherently proprietary. At AMD we're working on a fully open source and upstream driver that will use HMM's page table mirroring and migration features. In fact we were not able to submit the first round of changes in 4.19 because HMM was broken. That's just a reminder of the chicken-and-egg problem here.


Copyright © 2018, Eklektix, Inc.
This article may be redistributed under the terms of the Creative Commons CC BY-SA 4.0 license
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds