DRI, BSD, and Linux [LWN.net]

By Jonathan Corbet
September 2, 2008

The Direct Rendering Infrastructure project has long been working toward improved 3D graphics support in free operating systems. It is a crucial part of the desktop Linux experience, but, thus far, DRI development has been done in a relatively isolated manner. Development process changes which have the potential to make life better for Linux users are in the works, but, sometimes, that's not the only thing that matters.

The DRI project makes its home at freedesktop.org. Among other things, the project maintains a set of git repositories representing various views of the current state of DRI development (and the direct rendering manager (DRM) work in particular). This much is not unusual; most Linux kernel subsystems have their own repository at this point. The DRM repository is different, though, in that it is not based on any Linux kernel tree; it is, instead, an entirely separate line of development.

That separation is important; it means that its development is almost entirely disconnected from mainline kernel development. DRM patches going into the kernel must be pulled out of the DRM tree and put into a form suitable for merging, and any changes made within the kernel tree must be carefully carried back to the DRM tree by hand. So this work is not just an out-of-tree project; it's an entirely separate project producing code which is occasionally turned into a patch for the Linux kernel. It is not surprising that DRM and the mainline tend not to follow each other well. As Jesse Barnes put it recently:

Things are actually worse than I thought. There are some fairly large differences between linux-core and upstream, some of which have been in linux-core for a long time. It's one thing to have an out-of-tree development process but another entirely to let stuff rot for months & years there.

The result of all this has been a lot of developer frustration, trouble getting code merged, concerns that the project is hard for new developers to join, and more. As the DRM developers look to merge more significant chunks of code (GEM, for example), the pressure for changes to the development process has been growing. So Dave Airlie's recent announcement of a proposed new DRM development process did not entirely come as a surprise. There are a number of changes being contemplated, but the core ones are these:

The DRM tree will be based on the mainline kernel, allowing for the easy flow of patches in both directions. The old tree will be no more.
A more standard process for getting patches to the upstream kernel will be adopted; these will include standard techniques like topic branches and review of patches on the relevant mailing lists.
Users of the DRM interface will not ship any releases depending on DRM features which are not yet present in the mainline kernel.

The result of all this, it is hoped, will be a development process which is more efficient, more tightly coupled to the upstream kernel, and more accessible for developers outside of the current "DRM cabal." These are all worthy objectives, but there may also be a cost associated with these changes resulting from the unique role the DRI/DRM project has in the free software community.

There is clearly a great deal of code shared between Linux and other free operating systems, and with the BSD variants in particular. But that sharing tends not to happen at the kernel level. The Linux kernel is vastly different from anything BSD-derived, so moving code between them is never a straightforward task. GPL-licensed code is not welcome in BSD-licensed kernels, naturally, making it hard for code move from Linux to BSD even when it makes sense from a technical point of view. When code moves from BSD to Linux, it often brings a certain amount of acrimony with it. So, while ideas can and do move freely, there is little sharing of code between free kernels.

One significant exception is the DRM project, which is also used in most versions of BSD. One of the reasons behind the DRM project's current repository organization is the facilitation of that cooperation; there are separate directories for Linux code, BSD code, and code which is common to both. Developers from all systems contribute to the code (though the BSD developers are far outnumbered by their Linux counterparts), and they are all able to use the code in their kernels. When working in the common code directory, developers know to be careful about not breaking other systems. All told, it is a bit of welcome collaboration in an area where development resources have tended to be in short supply - even if it benefits the BSD side more than Linux.

Changing the organization of the DRM tree to be more directly based on Linux seems unlikely to make life easier for the BSD developers. Space for BSD-specific code will remain available in the DRM repository, but turning the "shared-code" directory into code in the Linux driver tree will make its shared status less clear, and, thus, easier for Linux developers to break on BSD. Additionally, it seems clear that this code may become more Linux-specific; Dave Airlie says:

However I am sure that we will see more of a push towards using Linux constructs in dri drivers, things like idr, list.h, locking constructs are too much of a pain to reinvent for every driver.

Much of this functionality can be reproduced through compatibility layers on the BSD side, but it must carry a bit of a second-class citizen feel. Dave has, in fact, made that state of affairs clear:

The thing is you can't expect equality, its just not possible, there are about 10-15 Linux developers, and 1 Free and 1 Open BSD developer working on DRM stuff at any one time, so you cannot expect the Linux developers to know what the BSD requirements are.

The fact that fewer people will be able to commit to the new repository - in fact, it may be limited to Dave Airlie - also does not help. So FreeBSD developer Robert Noland, while calling this proposal "the most fair" of any he has heard, is far from sure that he will be able to work with it:

I am having a really difficult time seeing what benefit I get from continuing to work in drm.git with this proposed model. While all commits to master going through the mailing list, I don't anticipate that I have any veto power or even delay powers until I can at least prevent imports from breaking BSD. Then once I do get it squared away, I'm still left having to send those to the ML and wait for approval to push the fixes. I can just save myself that part of the hassle and work privately. If I'm going to have to hand edit and merge every change, I don't see how it is really any harder to do that in my own repo, where I'm only subject to FreeBSD rules.

On the other hand, it's worth noting that OpenBSD developer Owain Ainsworth already works in his own repository and seems generally supportive of these changes.

Given the difference between the numbers of Linux-based and BSD-based developers, it seems almost certain that a more Linux-friendly process will win over. There is one rumored change which will not be happening, though: nobody is proposing to relicense the DRM code to the GPL. The DRM developers are only willing to support BSD to a certain point, but they certainly are not looking to make life harder for the BSD community. So they will try to accommodate the BSD developers while moving to a more Linux-centric development model; that is how things are likely to go until such a time as the BSD community is able to bring more developers to the party.

to post comments

DRI, BSD, and Linux

Posted Sep 2, 2008 21:23 UTC (Tue) by felixfix (subscriber, #242) [Link] (7 responses)

Perhaps a bit off topic -- if Apple had chosen X11 instead of their proprietary display system, would they be contributing to DRI/DRM?

Even further off topic -- why did Apple choose their own proprietary system? It can't be for secrecy, since it must be the apps which they want to keep closed source. Was it something they inherited from Next, was it a Steve Jobs preference, was X11 too slow and/or limited?

DRI, BSD, and Linux

Posted Sep 2, 2008 21:45 UTC (Tue) by Darkmere (subscriber, #53695) [Link] (5 responses)

This post at Slashdot by Mike Paquette is probably one of the best explanations you'll ever see http://developers.slashdot.org/comments.pl?sid=75257&...

DRI, BSD, and Linux

Posted Sep 3, 2008 4:58 UTC (Wed) by quotemstr (subscriber, #45331) [Link] (4 responses)

At the risk of straying off-topic: in the past eight years, most of Mike Paquette's concerns have been addressed:

1. Font server support for antialiasing, vectors, etc. render/fontconfig/xft
2. Drawing with PS-like path operations cairo; XRENDER
3. Dithering and phase controls moot: Apple has dropped paletted mode supported
4. Color calibration could use some improvement
5. Compositing COMPOSITE, DAMAGE, etc.
6. Affine transformations of windows Compositing window managers
7. Mesh-warps of windows Compositing window managers
8. OpenGL video playback XvMC
9. Performance Better drivers

Would it really have been all that much more work for Apple to implement the necessary X extensions?

DRI, BSD, and Linux

Posted Sep 3, 2008 5:17 UTC (Wed) by felixfix (subscriber, #242) [Link] (1 responses)

When did X split, after the big license change fiasco? If it hadn't split yet, maybe the X organization and their hidebound policies scared Apple away.

DRI, BSD, and Linux

Posted Sep 3, 2008 8:29 UTC (Wed) by rsidd (subscriber, #2582) [Link]

"X" didn't split -- it's closer to the truth to say it reunified. XFree86 split. At the same time, the X organisation (formerly a hidebound organisation aimed at proprietary vendors, that hosted an antique X codebase containing almost none of the XFree86 improvements) saw the light and accepted the people who left XFree86 (which was nearly all the important people). The result is today's X.org.

Mac OS X is indeed from before the XFree86 split. But I don't think Apple would have cared to base their windowing system on X, either way. They do supply an X server for those who want it.

DRI, BSD, and Linux

Posted Sep 3, 2008 8:31 UTC (Wed) by rsidd (subscriber, #2582) [Link] (1 responses)

Well, they probably didn't want to wait 8 years. And the XFree86 community (as it was then) probably wouldn't have been interested in a huge number of dramatic changes being dumped on them out of the blue.

DRI, BSD, and Linux

Posted Sep 3, 2008 13:06 UTC (Wed) by felixfix (subscriber, #242) [Link]

That's what I was thinking. If Apple had done the coding or at least paid for it, it wouldn't have taken 8 years, but the X group was so fractious that there would have been a more acrimonious split, possibly involving Apple keeping their changes to themselves.

DRI, BSD, and Linux

Posted Sep 3, 2008 18:21 UTC (Wed) by pphaneuf (guest, #23480) [Link]

Their current display system is based on the NeXT Display PostScript system. It's heavily reworked and improved, but the general architecture dates from back then.

NeXT and X11 are from approximately the same era (mid to late 80's), XFree86 appearing years after (early 90's).

DRI, BSD, and Linux

Posted Sep 3, 2008 8:37 UTC (Wed) by sylware (guest, #35259) [Link] (14 responses)

I'm a GPL one: I do not like the BSD open source model (the "I can close your code to my own benefit part", cf Darwin/MacOS is a song of pain for me). So I contribute as (L)GPL only. I do acknowledge that the state of the graphic stack is quite bad and intended to improve that situation. My help was refused because my code is GPL as in the Linux kernel.

DRI, BSD, and Linux

Posted Sep 3, 2008 9:00 UTC (Wed) by regala (guest, #15745) [Link] (13 responses)

you don't understant anything, and you mix everything. The "BSD open source model" is not "I can close your code to my own benefit part". It is "Before theorizing with some Rule #1, #2, #3 or more, try to stick to Rule #0: don't tell me what I can do with my own code." Trying to force liberty or fair behaviours upon people never works.

DRI, BSD, and Linux

Posted Sep 3, 2008 9:26 UTC (Wed) by sylware (guest, #35259) [Link] (12 responses)

I wanted to provide Linux GPL code and because of that, my help was refused. There is nothing more to add.

DRI, BSD, and Linux

Posted Sep 3, 2008 10:15 UTC (Wed) by regala (guest, #15745) [Link] (5 responses)

well, do not provide Linux GPL code to a project not compatible by license, even FSF advocates would tell you that. Do not take legal issues personnally, that's useless.

DRI, BSD, and Linux

Posted Sep 3, 2008 10:19 UTC (Wed) by regala (guest, #15745) [Link] (4 responses)

> do not provide Linux GPL code to a project not compatible by license

I think this one came out not quite right. Do not submit code with a license that would imply a GPL relicensing to a project that cares for its licensing. Because it would be legally difficult (because of all contributors have to agree upon relicensing)
sorry if you felt the refusal that way.

DRI, BSD, and Linux

Posted Sep 15, 2008 12:27 UTC (Mon) by robbe (guest, #16131) [Link] (3 responses)

> Because it would be legally difficult (because of all contributors have
> to agree upon relicensing)

As relicensing from BSD to GPL only places additional restrictions on the
licensee this can be done without approval from all contributors.

Of course the point remains that contributors happy with their project
being BSD-licensed will probably prefer rejecting your code to moving to
a (from their point of view) suboptimal license.

DRI, BSD, and Linux

Posted Sep 15, 2008 16:25 UTC (Mon) by dlang (guest, #313) [Link] (2 responses)

however, these same people apparently have no problem with their code being taken and put under a proprietary license.

if they don't like the code being put under a sub-optimal license they should pick a icense that prohibits it.

DRI, BSD, and Linux

Posted Oct 5, 2008 16:13 UTC (Sun) by sylware (guest, #35259) [Link] (1 responses)

You are right. It's the hypocrite part of the BSD community... they hate GPL but love proprietary. BSD and GPL really define different open source development models. The BSD seems to push for sub-optimal open source version compared to proprietary forks(cf darwin/macos). I think that's what RMS saw a few decades ago and that's why he designed the GPL:to make the open source version THE optimal version of a software.
Nevertheless, some companies/people successfully work around the GPL protection against making sub-optimal open source version compared to proprietary forks:check out mysql/opensolaris/lzo... That RMS missed: how successful would be a license that's protecting code from its authors themselves?
Big buck company can easily buy all significant authors from Linux and make a optimal proprietary fork...

DRI, BSD, and Linux

Posted Nov 14, 2012 8:11 UTC (Wed) by marcH (subscriber, #57642) [Link]

> It's the hypocrite part of the BSD community... they hate GPL but love proprietary

GPL is really proprietary... for the public. I guess that's why (some vocal people in...) the BSD community must feel better when only a few companies "steal" their code as opposed to the whole public: that's far less people and easier to pretend you don't know :-D

DRI, BSD, and Linux

Posted Sep 4, 2008 9:33 UTC (Thu) by liljencrantz (guest, #28458) [Link] (5 responses)

You where trying to force your personal license opinions on a project consisting of many individuals who had already agreed on a different choice. Are you truly surprised that the entire project did not decide to throw away their own license choice and shut out a large chunk of their own user base for the sake of one single developer?

DRI, BSD, and Linux

Posted Sep 4, 2008 10:56 UTC (Thu) by sylware (guest, #35259) [Link] (4 responses)

Hu? Where did you see I was *forcing* people? I used my mouth, not a gun or such. :)
I know, I would refuse BSD like code in my projects as they refused my GPL code. (MIT/public domain is fine since it is allowed to relicense under the GPL).
I understand them because I would do the same, period.
Then, I'm trying to code on the side of it, but for Linux (no cross kernel bloat) with the Linux GPL. It means I ate the bullet and I'm trying to work around the issue.
There are many aspects that can be shared despite the different open source models. For instance GPU hardware programming.

DRI, BSD, and Linux

Posted Sep 5, 2008 9:39 UTC (Fri) by liljencrantz (guest, #28458) [Link] (1 responses)

My bad, 'force' was too strong a word.

My issue with your original comment was that the tone strongly implied to me that you where surprised, possibly even bitter about the DRM peoples refusal to accept your patches simply because doing so would have forced them to change their license and drop support for the BSDs. This comment makes it clear that this was what you expected all along.

On a side note, I'm surprised about how many people refuse to contribute to projects with the 'wrong licenses'. Sure, I understand that people have strong license opinions, I have them myself, but I will happily contribute to projects under any free software license, even ones that I consider suboptimal. Even GNU seems to take this stance, as they have at sponsored the development of flex, ncurses and various other non-gpled projects.

DRI, BSD, and Linux

Posted Sep 6, 2008 7:36 UTC (Sat) by rsidd (subscriber, #2582) [Link]

Indeed, Stallman has written in support of the strategic value of using the BSD license in particular circumstances, for example, the Ogg Vorbis tools.

DRI, BSD, and Linux

Posted Sep 14, 2008 11:32 UTC (Sun) by eriwik (guest, #53902) [Link] (1 responses)

"I know, I would refuse BSD like code in my projects as they refused my GPL code. (MIT/public domain is fine since it is allowed to relicense under the GPL)."

You seem to be under the misconception that the MIT license allows you to freely change the license while the BSD does not. This is not true, the difference between the new BSD and MIT license is the non-endorsement clause (the old BSD license also had the advertisement clause). Neither of them allow you to change the license.

DRI, BSD, and Linux

Posted Oct 5, 2008 15:43 UTC (Sun) by sylware (guest, #35259) [Link]

sublicensing is explicit in the MIT license. Then I can fork the code and rebase the lot with the GPL.
I cannot sublicense with a BSD license, namely fork and get rid of the BSD license.

DRI, BSD, and Linux

Posted Sep 3, 2008 10:21 UTC (Wed) by jsbarnes (guest, #4096) [Link]

My hope is that this process change will actually make things a bit
easier for the BSD guys. Our old process was something of a "dogpile on
DRM master and sort things out later" approach. This meant that the BSD
guys were always playing catch up, so they had to either fork & update
periodically (like we did for Linux) or use the DRM code directly (what
Robert chose to do for FreeBSD). With the new process, we'll have a much
more disciplined approach to contributions: they'll be posted to the
mailing list and discussed prior to commit, which means everyone,
including the BSD developers, will have a chance to comment and ask for
changes before things are pushed upstream where they might break things.

I suspect Robert will want to change his development model anyway, since
forking + stabilizing with periodic updates would probably improve
stability & keep untested features out of releases, but even if he
chooses not to I think his life *should* be easier going forward (though
I have yet to convince him of this :).

Those interested in some of the background on this change can check out
the thread titled "[PATCH 1/1] Adapt on_each_cpu" in their favorite
dri-devel@lists.sf.net archive.

DRI, BSD, and Linux

Posted Sep 3, 2008 10:41 UTC (Wed) by anholt (subscriber, #52292) [Link] (1 responses)

(For background, I was the FreeBSD DRM guy before I switched to Linux, and Robert Noland has taken over what used to be what I did on FreeBSD -- port inside of the shared-core tree)

> When working in the common code directory, developers know to be careful about not breaking other systems.

This was never the case. The shared-core tree only ever really worked on FreeBSD when I was the latest commit in the tree with "fix Linux developer's commit for FreeBSD". I was always grumpy about this, and would always do my best to avoid breaking Linux and often test (though often fail) on it, so I could have the high ground and point to how they weren't being careful enough with me. Even when it wasn't OS-specifics in the shared-core directory, non-shared components would move forward and I would need to go and re-diff linux versus bsd and port changes over.

But I also struggled to bring the other OSes into the shared-core tree. I was baffled by how the NetBSD, OpenBSD, and Solaris people didn't want to work in the shared tree. I tried to convince people to submit patches that could be merged (though usually I got a massive diff of "here's where I took shared-core 6 months ago and then ported it to my OS", which I couldn't manage).

But then, looking at the just netbsd/freebsd integration that was done once, the code was becoming an unmaintainable mess. I never even managed to integrate the next-NetBSD patches I got once, which made the situation worse. The shared-core was already breaking down for managing 2 versions of 2 OSes. It was only going to get worse.

What I'm realizing now, is that the process I tried to maintain with shared-core was hopeless. We had this "shared" tree, that actually only ever shipped on one OS -- FreeBSD. Linux distros shipped distro DRM plus a few backports of drm.git stuff, NetBSD ships their own port that isn't in drm.git, OpenBSD ships their own port that isn't in drm.git, and OpenSolaris ships their own port that isn't in drm.git. So this drm.git tree was only ever being shipped wholesale on FreeBSD when I ran my little sed script to import it into FreeBSD.

What we're doing is acknowledging that the drm.git code was only for release on one out of the 5 OSes that DRM exists on. The developers for that OS should bear the burden of porting to their OS. Linux developers aren't too interested, shockingly, in writing code to an API to allow FreeBSD to release their code, when they can't even integrate that code they write into their own kernel for release!

Yes, this means FreeBSD now has to port the features that are developed on Linux (where all the features are developed, since the feature development I did on FreeBSD something like 5 years ago) out of a linux-2.6 tree. They'll join the ranks of the 3 other OSes porting the DRM from Linux, which the other OSes didn't seem to have big complaints about. Luckily for them, we have git now, and the process for porting can be easier than it ever was back when I was doing it.

DRI, BSD, and Linux

Posted Sep 3, 2008 18:40 UTC (Wed) by rsidd (subscriber, #2582) [Link]

Interesting post. I used to use FreeBSD till about 5 years ago and remember your name as the DRM guy. So it's a bit startling to learn that you've now moved to Linux.

Similar to the (Open)AFS situation

Posted Sep 4, 2008 17:22 UTC (Thu) by utoddl (guest, #1232) [Link] (1 responses)

I see lots of parallels with the OpenAFS situation. OpenAFS is a distributed file system that predates Linux and runs on lots of OSes. Maintaining a codebase that can compile kernel modules across several OSes and many versions of each (based on feature, not version number) is no small task. Integrating patches requires not breaking other OS builds. And there are licensing issues. There's no way to relicense the code, and some Linux kernel folks have been borderline hostile to the idea of even allowing the OpenAFS kernel module to be linked/loaded into the GPLed Linux kernel; there's no way to bring the existing OpenAFS kernel module in-tree. Yet somehow the OpenAFS core team manages to keep code working across versions of multiple OSes. I'm glad they do it, as I use/need OpenAFS, but I'm glad it's not my problem.

Similar to the (Open)AFS situation

Posted Sep 15, 2008 12:19 UTC (Mon) by robbe (guest, #16131) [Link]

> There's no way to relicense the code, and some Linux kernel folks have
> been borderline hostile to the idea of even allowing the OpenAFS kernel
> module to be linked/loaded into the GPLed Linux kernel; there's no way
> to bring the existing OpenAFS kernel module in-tree.

With OpenAFS apperently being under the IBM Public License 1.0
(GPL-incompatible according to the FSF) this is not that surprising.