Oracle offering DTrace for Linux [LWN.net]

Oracle offering DTrace for Linux

Posted Feb 22, 2012 23:38 UTC (Wed) by dlang (guest, #313) [Link] (3 responses)

even assuming that they aren't doing anything questionable with the licensing. The fact that they are shipping with dtrace doesn't mean that the dtrace patches are going to be acceptable to the upstream kernel (that's been a large part of the problem with other tracing solutions)

I do look forward to the discussion though.

Oracle offering DTrace for Linux

Posted Feb 22, 2012 23:52 UTC (Wed) by tialaramex (subscriber, #21167) [Link] (2 responses)

A big part of the supposed benefit of DTrace has been "You can just do this stuff and it's safe and will do exactly what it says on the tin". If they deliver that on Linux then it will be far harder to argue that it shouldn't get merged. If they can't deliver that on Linux then it will be far harder to argue that DTrace is serving any purpose other than putting an Oracle-flavour stamp on things.

The middle option is that they deliver some of it, but only gradually "it's beta, of course it crashes but that'll get fixed" and meanwhile the alternatives mature to a similar level of robustness and production-worthy reliability.

Oracle offering DTrace for Linux

Posted Feb 23, 2012 9:42 UTC (Thu) by robert_s (subscriber, #42402) [Link] (1 responses)

"A big part of the supposed benefit of DTrace has been "You can just do this stuff and it's safe and will do exactly what it says on the tin". If they deliver that on Linux then it will be far harder to argue that it shouldn't get merged."

DTrace managed to do that for Solaris largely because it has a much more closed-box kernel.

Oracle offering DTrace for Linux

Posted Feb 24, 2012 13:40 UTC (Fri) by davecb (subscriber, #1574) [Link]

I'd suggest it did it not because the kernel was closed, but rather that the compiler shoved the functionality into a really cramped little closed box, so it had trouble causing the traced programs problems, either directly or even by side-effect.

--dave

Oracle offering DTrace for Linux

Posted Feb 23, 2012 1:56 UTC (Thu) by nevets (subscriber, #11875) [Link] (19 responses)

I can't see how they could do anything that the state they do without a GPL license. They would either have to modify the kernel core (which must be GPL) or if they use tracepoints, the interface is only EXPORT_SYMBOL_GPL().

Oracle offering DTrace for Linux

Posted Feb 23, 2012 7:17 UTC (Thu) by lkundrak (subscriber, #43452) [Link] (17 responses)

When they announced they're working on DTrace, I was curious about it and isolated a patch from their "UEK" beta kernel. Apart from there being practically no useful tracepoints, this is what I saw:

-EXPORT_SYMBOL_GPL(print_context_stack);
+EXPORT_SYMBOL(print_context_stack);

And yes, they actually use it from their MODULE_LICENSE("CDDL"); module.

Oracle offering DTrace for Linux

Posted Feb 23, 2012 12:40 UTC (Thu) by rich0 (guest, #55509) [Link] (16 responses)

I don't get how that is illegal, as long as they post their modified sources to the kernel proper. They distribute a modified GPL kernel, and a non-GPL module. I don't see how the GPL dictates who can change the interface between them...

Oracle offering DTrace for Linux

Posted Feb 23, 2012 13:18 UTC (Thu) by nevets (subscriber, #11875) [Link] (7 responses)

-EXPORT_SYMBOL_GPL(print_context_stack);
+EXPORT_SYMBOL(print_context_stack);

Is a clear attempt to circumvent the GPL. I don't know the CDDL license but just posting their modified source is not enough to satisfy the GPL license. You must also allow those that you distribute the code to, the same rights that you have with the code. Free to modify themselves, and they too must follow the rules as well.

Needing the print_context_stack makes their module a derived work. Not something that was ported straight from Sun OS.

Oracle offering DTrace for Linux

Posted Feb 23, 2012 17:17 UTC (Thu) by mjg59 (subscriber, #23239) [Link] (1 responses)

Needing print_context_stack is an indication that the kernel developers feel it's likely to be a derived work. Oracle's legal department may have disagreed.

Oracle offering DTrace for Linux

Posted Feb 23, 2012 17:38 UTC (Thu) by nevets (subscriber, #11875) [Link]

As the author of print_ftrace_graph_addr() I would want them to remove that from their version. There's no need for a module to use it unless it was a derived work. I'll let others talk about the rest. It looks like the rest of the function originated from Arjan.

Oracle offering DTrace for Linux

Posted Feb 23, 2012 17:52 UTC (Thu) by kvh (guest, #83085) [Link] (4 responses)

I am not sure what you diff'd the source code against, but the 0.1 beta release of the DTrace code for Linux was against 2.6.32, and print_context_stack was not an exported symbol in that release. We needed to be able to override the default walk stack functionality (which in turn calls print_context_stack) and therefore I added an export for print_context_stack, and made it match the export for dump_trace (which is not GPL-only) since it is part of the same code. The ability to specify a walk stack function was added as well.

As soon as we realized that in later kernels print_context_stack was exported as GPL-only rather than the non-GPL-only we had chosen, we changed our code to match that, and we moved the part of the DTrace code that uses it GPL as well. This is evidenced in the source code drop for 0.2 that will be placed on oss.oracle.com very soon.

Oracle offering DTrace for Linux

Posted Feb 24, 2012 2:47 UTC (Fri) by clump (subscriber, #27801) [Link] (2 responses)

Given that DTrace's copyright owner now distributes DTrace for Linux, should there be any reason not to simply dual-license DTrace as GPL? I'm assuming inclusion/harmony with kernel.org is a goal, in which case the GPL seems rather obvious.

Oracle offering DTrace for Linux

Posted Feb 24, 2012 3:29 UTC (Fri) by nevets (subscriber, #11875) [Link] (1 responses)

Well if they would like any changes into the kernel their module would have to be shipped GPL (dual license or not), otherwise you can expect big NAKs to any changes that help them out.

Oracle offering DTrace for Linux

Posted Feb 24, 2012 19:56 UTC (Fri) by clump (subscriber, #27801) [Link]

Precisely. Working around deliberately incompatible licenses might be necessary if Oracle didn't own the copyrights to DTrace. I hope the goal isn't to make DTrace harder to use on other versions of Linux.

Oracle offering DTrace for Linux

Posted Feb 24, 2012 7:03 UTC (Fri) by lkundrak (subscriber, #43452) [Link]

Thanks for the reply.

Sorry for not using the version that will be released "very soon" :) Thanks for the oss.oracle.com pointer though; I've been comparing the SRPMs. Seems like I did compare 2.6.32-201.0.1.el5uek to Red Hat's 2.6.32, where they seem to have backported the print_context_stack() independently. Certainly my fault, let me apologize.

Having recompared against 2.6.32-200.19.1.el6uek, on top of which was it based according to the change log, the following seems to have been exported:

+EXPORT_SYMBOL(syscall_nr_to_meta);

Now -- I don't get this; using symbols that are GPL-only from a non-GPL module is wrong, while using ones that have not been exported at all is fine?

Oracle offering DTrace for Linux

Posted Feb 23, 2012 17:07 UTC (Thu) by robert_s (subscriber, #42402) [Link] (1 responses)

Then you clearly don't understand the idea of marking symbols GPL and non-GPL.

Marking symbols non-GPL is effectively the kernel developers saying "we don't consider this symbol as an internal core function of the kernel" and as such a module using just those symbols would not be considered a derived work. i.e. "we won't sue over use of these symbols"

Oracle offering DTrace for Linux

Posted Feb 23, 2012 17:15 UTC (Thu) by khim (subscriber, #9252) [Link]

Actually it's the other way around: if symbol is declared EXPORT_SYMBOL_GPL then it means that linux developers believe it's impossible to use it without making module derived work of Linux kernel because it's hook to such an intimate part of it.

It's still perfectly possible to create module which is derived work of Linux kernel and which only uses EXPORT_SYMBOL hooks.

Oracle offering DTrace for Linux

Posted Feb 23, 2012 17:11 UTC (Thu) by lambda (subscriber, #40735) [Link] (5 responses)

Oracle is not the copyright holder on the kernel; they hold copyright on some parts, but not others. Each developer or company holds the copyrights to the portions which they contributed.

The GPL does not allow someone downstream to link and distribute modules to the GPLed software unless that code is also available under the GPL. However, as an exception, kernel developers have granted the ability for non-GPLed kernel modules to link against the kernel, as long as they only use certain symbols that are exported via EXPORT_SYMBOL instead of EXPORT_SYMBOL_GPL. Anyone can apply the GPL to their own work, and then grant additional freedoms beyond what the GPL grants, and that is what the kernel developers (and copyright holders) have done whenever they apply EXPORT_SYMBOL to an interface.

However, someone else cannot come along, and grant themselves additional freedoms. I can't take a piece of GPLed software, and add a few declarations saying "this interface is OK for non GPLed software to link against," and then add my own proprietary code to the software and distribute it. I do not own the copyright, so I cannot grant myself the additional freedom to do this.

That is what Oracle seems to be doing here; they seem to be trying to grant themselves additional freedom beyond what the GPL license of the kernel grants. And I can imagine that some kernel developers might wind up being fairly unhappy about that.

Oracle offering DTrace for Linux

Posted Feb 23, 2012 17:32 UTC (Thu) by khim (subscriber, #9252) [Link] (4 responses)

However, as an exception, kernel developers have granted the ability for non-GPLed kernel modules to link against the kernel, as long as they only use certain symbols that are exported via EXPORT_SYMBOL instead of EXPORT_SYMBOL_GPL.

Citation needed™. And it's needed really badly because historicaly Linux developers had the opposite POV. It's hard to interpret the infamous Linus's

If an author claims your work is derivative, you should step very very lightly.

And I claim that binary-only kernel modules ARE derivative "by default", by virtue of having no meaning without the kernel.

except as "if you lawyer believes your work is not derivative and if you are ready defend this POV in court, then maybe, just maybe your work is indeed not a derivative and thus can be non-GPL licensed".

However, someone else cannot come along, and grant themselves additional freedoms.

Yes, they can. This is just a marker. But yes, what Oracle did is a clarion call: they took symbol which was marked with "no way in hell can you use it without creating derived work" symbol and removed this mark. Their lawyers should be prepared to defend removal of said mark quite seriously.

And I can imagine that some kernel developers might wind up being fairly unhappy about that.

Not necessarily. If it can be shown that DTrace indeed uses this hook in way which does not imply intimate knowledge of kernel internals then this will be strong signal that this symbol was marked as EXPORT_SYMBOL_GPL by mistake.

Oracle offering DTrace for Linux

Posted Feb 23, 2012 19:28 UTC (Thu) by lambda (subscriber, #40735) [Link] (3 responses)

Sorry, you are right. I got the sense backwards in terms of which one is more explicit. EXPORT_SYMBOL is not explicit about what it is declaring; using it, you may or may not be creating a derived work. EXPORT_SYMBOL_GPL is a way of documenting "if you use this interface, you are clearly mucking about in the internals of the kernel and thus are clearly creating a derived work." The only explicit disclaimer is that any interface exported to user space is disclaimed from creating a derived work for purposes of the GPL.

That doesn't really change my point about the intent of changing an interface from EXPORT_SYMBOL to EXPORT_SYMBOL_GPL; that change says "I know that the kernel developers have said that what I'm doing will create a derived work, but I am going to do that anyhow, and I am not releasing my module under the GPL." That does not automatically mean that what they are doing is a copyright violation; but it shows that they are aware that there are kernel developers who believe that use of that interface constitutes creating a derived work, and they are proceeding anyhow.

Oracle offering DTrace for Linux

Posted Feb 24, 2012 0:51 UTC (Fri) by mathstuf (subscriber, #69389) [Link] (2 responses)

If you look at this[1] comment, it isn't a proper diff. The kernel that DTrace is based on didn't export print_context_stack at all and their patches exported it as non-GPL, whereas more recently, kernel developers exported it as GPL. A grep through the 2.6.32.57 tarball doesn't show print_context_stack being exported at all, so this seems, to me, to be taken out of context.

[1]https://lwn.net/Articles/483322/

Oracle offering DTrace for Linux

Posted Feb 24, 2012 13:49 UTC (Fri) by vonbrand (subscriber, #4458) [Link] (1 responses)

AFAIU, there are three cases here:

Not exported: For in-kernel use only; use is automatically GPLed
EXPORT_SYMBOL: For use in modules; just using it doesn't make your module GPLed
EXPORT_SYMBOL_GPL: For use in GPLed modules

If it wasn't exported, it's the first case; adding EXPORT_SYMBOL is shady.

Oracle offering DTrace for Linux

Posted Feb 24, 2012 21:14 UTC (Fri) by mathstuf (subscriber, #69389) [Link]

I would imagine if it still weren't exported today, there would be discussion of making it non-GPL enforcing export for merging. I imagine other symbols gained an EXPORT_SYMBOL after being unexported for a time. In any case, it seems a moot point since (apparently) the 0.2 version obeys the EXPORT_SYMBOL_GPL.

Oracle offering DTrace for Linux

Posted Feb 23, 2012 18:45 UTC (Thu) by renox (guest, #23785) [Link]

Or distribute DTrace as a patch for which endusers have to recompile their kernel.
Of course this makes it much less useful for Linux..

Oracle offering DTrace for Linux

Posted Feb 23, 2012 9:40 UTC (Thu) by robert_s (subscriber, #42402) [Link] (2 responses)

Nothing against systraceperfmonUtraceNG, but DTrace seems a lot nicer and better thought out.

So a system designed for a whole other kernel is better than what the actual linux kernel developers designing this stuff for years could come up with?

Oracle offering DTrace for Linux

Posted Feb 23, 2012 13:30 UTC (Thu) by nevets (subscriber, #11875) [Link] (1 responses)

Note, the tracing infrastructure in the kernel has to take a second seat to the rest of the kernel. That is, tracing must not impact the normal flow of the kernel. If this was a company that owned Linux, and the upper management said, we want the tracing infrastructure to be #1 priority, then we could add a strong infrastructure even if the maintainers of the subsystems that we modified disagree.

But this is not how Linux works. Linux is a community, there's a lot of give and take. The subsystem maintainers are responsible for their code, and the tracing folks can't just come in and add hooks where they want without good rational. It is really up to the maintainers to add the hooks.

Then there's more than one player in this game. And tracing is a lot like personal editors. Everyone has their favorite and no one will back down. This slows down the development but hopefully in the long term the best of all utilities will be incorporated. Yes it takes time. This is part of the way open source works.

Oracle offering DTrace for Linux

Posted Feb 23, 2012 17:03 UTC (Thu) by robert_s (subscriber, #42402) [Link]

Indeed - I was rather alluding to what you said in your more comprehensive answer rather than showing disbelief in the kernel development process.