LWN: Comments on "KS2008: Tracing"

dtrace on Linux

njs — 2008-09-22T17:33:13+00:00

Well, there is Sun's ongoing refusal to answer questions like "Are you willing to grant a license to any ZFS patents you hold to other FOSS implementations?", which creates at least a cloud of uncertainty about what they might do if they became evil, or just decided that Linux was catching up with Solaris and threatening them.

Unless they have changed their mind and answered this at some point. I hope so -- haven't followed closely.

But paulj is right re: copyrights; Sun wrote the CDDL in such a way that combining CDDL and GPLv2 violates the GPL but not the CDDL. This has the same effect in practice as not granting permission, but it means that for copyright issues we don't need to worry about Sun-turns-SCO -- we need to worry about SCO-turns-SCO.

dtrace on Linux

zooko — 2008-09-22T15:42:13+00:00

Wait, what happens if Sun decides to try to use all of its copyrights to sue Linux distributors or users or otherwise to do harm to Linux development? Haven't they already granted a permanent, irrevocable Free Software licence over the current versions of DTrace and ZFS, in the form of the CDDL?

I have a feeling that you already addressed this in your "What if Sun turns Evil?" article, but I don't recall any actual problems that could result from use of DTrace or ZFS source code.

dtrace on Linux

paulj — 2008-09-22T14:54:35+00:00

There's no hazard from Sun, as the CDDL licensed DTrace bits can be combined just fine with GPL bits, as far as the former licence is concerned. The hazard is from a Linux kernel copyright holder turning SCO on Linux distributors, I thought we had established in an earlier discussion.

dtrace on Linux

corbet — 2008-09-22T13:57:44+00:00

The number of Linux distributions shipping binary-only drivers has gotten quite small; there are good reasons for that.

DTrace presents a different hazard; imagine a Sun-turns-SCO scenario, for example. The fact that Sun does not appear to be interested in taking that path now is irrelevant; neither was Caldera, once upon a time.

dtrace on Linux

paulj — 2008-09-22T10:58:07+00:00

And yet Linux distros still manage to ship these proprietary, binary-only drivers, despite these terrible legal constraints.

With DTrace being free-software, there isn't even much of a moral hazard...

dtrace on Linux

njs — 2008-09-21T18:38:32+00:00

>But this original argument is not true. At least, it is true only inasmuch as it is also true that the licensing issues of nvidia drivers make them undistributable by anyone, so therefore not not of any practical interest.

It's still of no practical interest to kernel developers, distributors, enterprise users, and many others. Jon's (not bronson's) comment was totally reasonable in context -- a kernel developers' discussion about what enterprise-funded developers should work on!

But fair enough -- there may be (probably are) others who find linux-dtrace of practical interest.

But arguments 2 and 3 are linked to this -- to the extent that such people use linux-dtrace, the harms described in those arguments swing into effect. To the extent they avoid linux-dtrace, the harms are abated -- with a trade-off: then they lose dtrace's benefits. There's a tragedy of the commons danger here; the costs of supporting inscrutably broken systems and inflammatory bickering are borne by the community, while the benefits of dtrace are received only only by individuals. Plus, people using/supporting linux-dtrace are not doing themselves any favors in the long run, because it's clearly a dead-end; it's better than nothing, but getting a great solution will require abandoning it and switching to one of the other systems that linux-dtrace sucks the oxygen away from.

So I guess 2 & 3 are arguments for why if we encounter someone for whom linux-dtrace is of practical interest, we should attempt to stop them ;-).

>dtrace and ZFS are exactly as legally-portable-to-linux as are proprietary graphics card drivers.

As a side-point, I'm actually not convinced, at least for dtrace. The nvidia driver uses two tricks: it makes a serious attempt not to be a derived work of the kernel, by including a Free shim layer to basically a windows driver. And it's always distributed separately from the kernel -- otherwise you're distributing a combined, thus derived, thus un-distributable, work. Even the little distros that tried to play fast and loose seem to have given in on this point.

Dtrace is far more intrusive than a graphics driver, and in copyright relevant ways -- it needs to muck around with other people's code to put hooks in. That makes both of the above tricks hard to achieve. Maybe not impossible, I don't know. From his rhetoric about licenses, I haven't gotten the impression that Paul Fox is being that careful. I would not tell people that linux-dtrace as it exists is legal to distribute at all without knowing many more details.

>Argument 2 is a strong argument when we're talking about proprietary, closed-source software, but dtrace and ZFS are Free and Open source, so I'm not sure that it would be as fragile and problematic as proprietary drivers.

It's true the problems are worse for proprietary drivers, but they're quite bad even for plain old out-of-tree drivers. (Note part of the discussion above is about RH's systemtap team's trouble keeping sync with mainline!) And worries about license contamination are an extra burden on top of that.

Sorry for nattering on so!
-- Nathaniel

dtrace on Linux

zooko — 2008-09-21T12:43:58+00:00

That's a really good point that DTrace is Free Software. I just now eventually came around to realizing the relevance of that in another post on this thread.

However, my point here was that resulting restrictions on use and redistribution are similar:

The copyright holder on DTrace has granted permission to use it in Linux systems, but the GPL which governs redistribution of Linux source code forbids redistributing derived works which include non-GPL'ed code such as DTrace.

This winds up imposing the same sort of restriction on users and distributors that a proprietary driver does: the copyright holder of the proprietary driver grants you the right to use it in your Linux kernel, but the Linux licence forbids you to redistribute it (assuming the current standard interpretation of GPL applied to this issue).

dtrace on Linux

zooko — 2008-09-21T12:39:10+00:00

Wait, that's a different argument. The original statement by bronson was Argument 1: that the licensing issues make dtrace "undistributable by anybody, so therefore not of any practical interest". NJS added that it "only matters if major distros start redistributing it".

But this original argument is not true. At least, it is true only inasmuch as it is also true that the licensing issues of nvidia drivers make them undistributable by anyone, so therefore not not of any practical interest.

Now you're making two other arguments. If I understand correctly, Argument 2: If people do use kernel modules or patches with this kind of legal restriction, it is hard to support and tends to break. and Argument 3: It distracts the community with inflammatory bickering about legal issues.

I don't necessarily disagree with either of those. I personally have gone to great effort to get Free Software drivers for my graphics cards. On the other hand, I have occasionly fallen back to using the proprietary graphics card drivers when needed. I think it could be useful to people to understand that even if the latter two arguments are valid, the first one wasn't: dtrace and ZFS are exactly as legally-portable-to-linux as are proprietary graphics card drivers.

Regards,

Zooko

P.S. Oh wait, I'm not sure if I agree with Argument 2. Argument 2 is a strong argument when we're talking about proprietary, closed-source software, but dtrace and ZFS are Free and Open source, so I'm not sure that it would be as fragile and problematic as proprietary drivers.

dtrace on Linux

Jonno — 2008-09-21T11:22:46+00:00

But it's not GPL compatible, so the *legal* constraints are the same.

Morally and technically it's another matter however, as the source *is*
availible, and *can* be fixed if nessesary...

dtrace on Linux

paulj — 2008-09-21T09:29:19+00:00

The legal constraint on using dtrace or ZFS on Linux is exactly the same as the legal constraint on using a proprietary hardware driver on Linux.

Uh, not really. DTrace is free software..

dtrace on Linux

njs — 2008-09-21T04:43:07+00:00

That sounds accurate.

I note that proprietary hardware drivers *suck* in many, many ways. They make your system impossible to provide support for, which is kind of a problem for a product whose core market is high-end enterprise production servers. They break things horribly and regularly -- and dtrace is obviously *much* more intrusive than your average driver. They're a huge pain to use and maintain -- is there any reason to think that the guy porting dtrace will be more successful at tracking linux mainline than RH's engineers are with systemtap? And will it even be possible to load as a module, or will you have to patch your core kernel/rebuild/reboot?

And, perhaps worst, they create all kinds of obnoxious arguments that divide the community -- look at any discussion of nvidia drivers, or the dtrace discussions here. Part of the magic of FOSS is that normally the enthusiasts, the law-skirters, the enterprise distros, the big-iron vendors, the small consultants, etc. etc. can all work towards common goals, and all provide necessary value at different parts of a system's lifecycle. When people spend their time yelling at each other over whether nVidia's drivers are legal or whether systemtap/btrfs/whatever is worth the effort, we all lose.

Which maybe shouldn't be surprising, since all the evidence suggests that one of Sun's strategic goals in choosing to license dtrace and ZFS in a Linux-incompatible way was to create exactly this sort of division within the FOSS community, and thus give Solaris a competitive chance. What I'm still not clear on is which of these titanic monsters you're going after, exactly...

dtrace on Linux

zooko — 2008-09-20T14:43:23+00:00

That's a good point. A more precise comparison then would be copyright-restricted codecs or the Microsoft fonts, or copyright-restricted drivers.

How's that for a comparison? The legal constraint on using dtrace or ZFS on Linux is exactly the same as the legal constraint on using a proprietary hardware driver on Linux.

dtrace on Linux

rahulsundaram — 2008-09-20T01:37:09+00:00

DTrace != Marketing endeavor

bcantrill — 2008-09-19T22:21:13+00:00

To be fair, I should point out that the lack of marketing focus on DTrace was somewhat deliberate on our part: we kept any and all marketing folks in the dark until we had satisfied customers. (I've always believed in talking as little as possible about what you're working on until you have cut enough code to back up any claims.) Once we had a bevy of happy customers, our marketing folks gladly talked about it -- but it was never a focus on their part, and the reception of DTrace in the marketplace certainly has nothing to do with their efforts. Anyway. you can imagine the frustration in repeatedly hearing the success of DTrace being attributed to non-existent marketing efforts... ;)

DTrace != Marketing endeavor

nix — 2008-09-19T22:02:32+00:00

I'm surprised by dtrace not getting marketing $$$: I've heard more people
raving about dtrace than every other feature of Solaris put together, even
than things like ZFS and zones. dtrace *should* be a marketing endeavour
in the sense that marketing loves it and pushes it hard, because it rocks
and customers love it. (The license incompatibility is really annoying.
I'd like to see dtrace everywhere but this is impossible until someone
shoots the current copyright system or reimplements all of dtrace, what a
waste of time.)

i.e. this is as opposed to the sort of marketing endeavour where marketing
lies to customers with ridiculously overblown promises ("our program will
make your coffee for you and can replace all your staff with AIs which
don't need paying"), makes up features that don't exist, or outright makes
up features that are only implementable in a parallel universe with
different maths ("[system] analyses your programs and warns about all
those that will halt" as one project I declined to work on was described).

DTrace != Marketing endeavor

bcantrill — 2008-09-19T17:19:17+00:00

This piece -- as with your several others on the subject -- is generally quite strong, and balanced with respect to DTrace.

There is, however, one issue I would like to lay to rest. DTrace is not, contrary to ongoing assertions here and elsewhere, a marketing endeavor. DTrace has succeeded because we (Team DTrace) worked damned hard for a damned long time solving a damned hard problem -- and we succeeded at a time when the world didn't want to hear anything that wasn't "Linux" (trust me on this). Far from putting "a lot of marketing energy" into telling customers about DTrace, Sun has put virtually no marketing dollars into DTrace, and it (ironically) took a herculean effort on our part to get even the most basic marketing support. (As an example: when we held a DTrace un-conference this past March, it was a Sun partner who stepped up to pay for it.) It's especially grating when we are accused of "hyping" aspects of the technology -- safety in particular -- that are foundational in nature. We are not and have not "hyped" our safety -- but we (and particular, I) have also not hesitated to point out the architectural differences between DTrace and SystemTap in this regard, e.g.:

http://blogs.sun.com/bmc/entry/dtrace_safety

Providing motivation for architectural decisions is not "marketing", it's technology -- and you mar your otherwise strong work when you dismiss it as being something less...

dtrace on Linux

zooko — 2008-09-19T12:22:02+00:00

Hm... Wouldn't this mean that the dtrace-for-linux code is in the exact same legal status as the "proprietary codecs" that various distributions have various ways of dealing with?

For example:

http://fedoraproject.org/wiki/ForbiddenItems#MP3_Support

https://help.ubuntu.com/community/RestrictedFormats

In both cases, there is some source code which is legal for end-users to use, but (perhaps?) not legal for Linux distributions to distribute.

If I'm right that this is the same legal situation, then this implies that it is legally possible for dtrace-on-linux to be as widely used as MP3-on-linux is. :-)

Regards,

Don Quizooko

P.S. I love tilting at windmills. Every now and then you get a solid hit on those titanic monsters.

KS2008: Tracing

mjw — 2008-09-19T10:37:35+00:00

Nice to see immediate results from this in the systemtap code base.

Also useful is the ability to run tracing tools in a "flight recorder" mode, where an administrator can look at historical data after something goes wrong.

commit 2fa2a091a0b248855d7f77aa20677ef4c7a7cc61
Author: Nobuhiro Tachino <tachino@jp.fujitsu.com>
Date:   Tue Sep 16 22:04:02 2008 -0400

    add new stap -F (flight recorder) option that just passes through to staprun -L

dtrace on Linux

njs — 2008-09-19T01:48:39+00:00

But the point is that it doesn't matter if you redistribute it; in the big picture it only matters if major distros start redistributing it.

And no major distro's legal department is going to let them blatantly violate the copyrights of everyone on lkml and open themselves up to legal action. (But at least SCO would finally have a valid case against Novell!)

So don't let me stop your fun, but you're kind of tilting at windmills, to prove some sort of point that I don't quite see...

dtrace on Linux

SEJeff — 2008-09-18T20:10:57+00:00

http://www.crisp.demon.co.uk/blog/ his blog is pretty interesting to watch also. He doesn't update it super often, but it is always something good.

dtrace on Linux

zooko — 2008-09-18T16:56:33+00:00

It could be of practical interest if someone wanted to download it and install it and use it, right?

Also, I find it hard to imagine that some Linux copyright holder would sue you if you redistributed it to a third party. That would be interesting. I would almost want to be sued (for what -- monetary damages? An injunction forbidding me to keep redistributing it?) just to see it happen.

Unfortunately, I don't have time today to download the dtrace port, combine it with the Linux kernel, and put it up on my web server for redistribution. Maybe tomorrow.

dtrace on Linux

zdzichu — 2008-09-18T10:15:04+00:00

It's a learning material. From all discussion about Linux tracing I get a feeling that no Linux kernel developer seen DTrace in action. And no one exactly knows what they compete against. It would be nice if kernel developers tried DTrace, learn about problems it helped to solve (there are many reports on blogs) and saw how powerful yet easy it is.

DTrace is wonderfully usable by Joe random admin and is generic enough to stop blooming of specialised tracing frameworks. Sure, latency top is cool, powertop is cool, usbmon is cool, blktrace is cool, but each of them needs its own hooks in kernel. On Solaris, powertop is implemented on top of DTrace, without modifing kernel.

dtrace on Linux

corbet — 2008-09-18T05:19:54+00:00

No mention of the DTrace port. The licensing of Linux makes a DTrace port undistributable by anybody, so it is not of any practical interest.

dtrace on Linux

fuhchee — 2008-09-18T04:35:07+00:00

LWN doesn't allow a simple "no" for an answer, so
this is a sentence stating "no".

dtrace on Linux

bronson — 2008-09-18T04:27:14+00:00

Did anybody mention the Linux dtrace port? ftp://crisp.dynalias.com/pub/release/website/dtrace/