LPC: Fitting into the kernel ecosystem [LWN.net]

LPC: Fitting into the kernel ecosystem

Posted Sep 18, 2008 5:01 UTC (Thu) by dowdle (subscriber, #659) [Link]

If Ubuntu/Canonical wants to modify the source code of a software package and get their patch upstream so they don't have to continuously carry it from version to version... they should be dealing with the direct upstream software and NOT Debian. Debian is not going to be a patch middleman between Ubuntu and an upstream software project.

If you are talking Debian produced software packages or packaging related changes that are distro specific... then it would be appropriate for Ubuntu to go through Debian.

So the claim that you can't see Ubuntu because they are go through Debian is just wrong. I don't think anyone from Ubuntu/Canonical nor Debian would claim that.

It is my understanding that LWN started doing this type of analysis on the Linux kernel some time ago, scripts that I believe GKH is using and has perhaps improved upon. I bring this up because LWN's findings have been exactly the same so far as I can tell. When Jon started publishing articles with his results I emailed asking why Ubuntu/Canonical and Debian didn't show up anywhere. If there was a flaw in the way he was gathering the data that was somehow excluding Ubuntu/Canonical and Debian... or if it was basically harder to identify them if some developers were using non-organizational email addresses. I believe Jon's answer was basically... no, I don't think I have a flaw and I don't think that their developers are being attributed to other catagories (like unknown or independent)... and that it just appears they do not contribute much to upstream kernel development. Now, I don't want to speak for Jon... so he can certainly clarify if he wants to. :)

Here is a clarification I want to make though... the blog posting protesting GKH's presentation says that his data is flawed and inaccurate with regards to Ubuntu/Canonical. It isn't that it is somehow flawed JUST toward them... but that it isn't perfect and is flawed in general. Measuring what he is trying to measure is quite difficult. Jon has pointed out near the beginning when he started reporting on the kernel... that there was no perfect way to measure... but that it was worth trying to measure even if flawed... and can certainly be improved over time. The methodology they use is the best they have been able to come up with.

I don't think adding meters like... how many patches per capita/employees... and is going to offer useful information... but if the consensus is to add such things, be my guest. It would probably be just as useful as adding patches as a percentage of userbase size.

LPC: Fitting into the kernel ecosystem

Posted Sep 18, 2008 6:42 UTC (Thu) by dlang (✭ supporter ✭, #313) [Link]

given that userbase is at best an educated guess, making charts based on employees or developers would be far better (you do have the problem of how to count debian employees or developers)

I also find it odd (as in, showing an agenda) that Canonical with 100 patches is considered the ultimate evil, but Debian with <300 isn't mentioned at all.

LPC: Fitting into the kernel ecosystem

Posted Sep 18, 2008 7:33 UTC (Thu) by filipjoelsson (subscriber, #2622) [Link]

There are flaws in your reasoning. Patches / employees will not deliver more flattering numbers for Canonical, it will generate an error. Since neither Debian nor Gentoo has any employees at all (and that's why Debian isn't mentioned), their contribution in this respect is infinitely higher than that of Canonical. In the same way, counting spare time developers from Gentoo and Debian on par with salaried developers of Red Hat or Novell is irrelevant.

Perhaps comparing users / patches is a better statistic? Well, different distros attract different types of users. I'd guess that the communities of Debian and Gentoo are more technically inclined than the community of Ubuntu. Besides, Ubuntu has a pretty large user base - so that'd be neither better nor more flattering for Ubunto, I think.

So, what is there to compare? I suppose what's left is patches from companies generating revenue selling Linux. And if purely community driven distros show up on the charts, let them stay as a reminder to those who actually pay salaries to developers. Wind River, rPath and Mandriva may find themselves similarly challenged.

This does not imply that I agree with Greg. I think it was shortsighted to not include Gnome/KDE/XFCE in the comparison - not that I know if it would have made a difference.

It does imply that I think ubuntuers are whining, however. A more suitable reaction would have been: "Shit! Are we at the bottom? Well, we'll overtake rPath before the next time he gives a talk. And, Mandriva's next after that!"

LPC: Fitting into the kernel ecosystem

Posted Sep 18, 2008 7:36 UTC (Thu) by rahulsundaram (subscriber, #21946) [Link]

Yes, counting users tends to be fuzzy but Fedora has published some statistics which provides a good estimate. If others do too (along with smolt profiles), we would get a better understanding of our user base and use that to convince vendors and ourselves to support Linux better.

https://fedoraproject.org/wiki/Statistics

It is possible to be more accurate but at the cost of privacy which is not a good balance but more details at

https://fedoraproject.org/wiki/Statistics/Ideas

I am sure there is a agenda involved in getting commercial vendors involved to participate more which I think is a good thing. If there are accusations of anything else. being explicit is (somewhat) courteous. I don't see anybody mentioning "evil". Debian contributing as many patches as it has as a volunteer based organization seems pretty remarkable to me neverthless.

LPC: Fitting into the kernel ecosystem

Posted Sep 18, 2008 7:36 UTC (Thu) by alex (subscriber, #1355) [Link]

I also find it odd (as in, showing an agenda) that Canonical with 100 patches is considered the ultimate evil, but Debian with <300 isn't mentioned at all.

I don't. All it says is a purely volunteer based effort like Debian still submits more patches than the more widely used and commercially funded Ubuntu.

LPC: Fitting into the kernel ecosystem

Posted Sep 18, 2008 16:43 UTC (Thu) by mdz@debian.org (subscriber, #14112) [Link]

...which is fantastic, and shows that Debian is a large, mature and relatively healthy community of developers. The most significant group of contributors to Linux, Greg tells us, is still individuals who don't acknowledge corporate sponsorship of their work.

Debian has very successfully enabled thousands of developers to contribute, and has grown into a very successful and independent project. I hope that Ubuntu is in such good shape when it's 15 years old.

For now, however, Ubuntu has a relatively small developer community which is highly dependent on Canonical. The Debian kernel team is larger than the Ubuntu kernel team and has been active for a much longer time.

LPC: Fitting into the kernel ecosystem

Posted Sep 18, 2008 17:16 UTC (Thu) by alex (subscriber, #1355) [Link]

I agree it's good and I'm not really into giving Canonical/Ubuntu a shoeing over their perceived lack of contribution upstream. The numbers are what they are and if people don't want to feel picked on because of Canonicals poor showing compared to the other major distros then they know what they need to do.

IIRC Google got similarly picked on at last years OLS and I think their stats have improved this year. I'm not sure if it's cause and effect though...

LPC: Fitting into the kernel ecosystem

Posted Sep 23, 2008 19:56 UTC (Tue) by chema (subscriber, #32636) [Link]

"IIRC Google got similarly picked on at last years OLS and I think their stats have improved this year. I'm not sure if it's cause and effect though..."

I think Google stats enhancement is called Andrew Morton ;)

LPC: Fitting into the kernel ecosystem

Posted Sep 19, 2008 14:20 UTC (Fri) by hmh (subscriber, #3838) [Link]

There are some key differences between Debian and Ubuntu as far as the kernel and kernel contributions go.

Debian kernels are really, REALLY close to mainline kernels. They sometimes contain backports of patches that are not in mainline yet, but have been already accepted for eventual inclusion. And the rare Debian-originated patch is almost always sent upstream nowadays (I am not sure it was like that a few years ago, and sometimes the patches are not ACCEPTED upstream but still remain in the Debian kernel).

Ubuntu kernels have a lot more custom changes inside than Debian's as far as I know (they certainly used to), and you WILL get a major black eye when you do that and don't send your changes upstream. I am not sure if Ubuntu still deserves it, but they DO have a reputation of not bothering to send any of their kernel changes upstream in LKML (and that reputation obviously falls on Canonical shoulders).

Also, most Debian people I know that happen to also work on the kernel (and that includes myself!), do it directly *upstream*. Our work almost always gets into Linux mainline well before it ever makes it to any Debian kernel. Since Debian simply doesn't pay anyone to do any work (it is 100% volunteer-based), that work is NOT credited to Debian by most (any?) of us kernel developers that are also Debian developers.

For reference, you could credit something like 200 commits from me alone to Debian's name (I didn't check the date of the earliest commit, so some of those might be too old for the time-frame GregHK is using)... and I am not even one of the most prolific Debian developers that do upstream kernel work, AFAIK.

In the end, we Debian developers [that don't do kernel work in paid time by someone else] work upstream in the kernel without crediting Debian, and Debian nowadays doesn't have much to send upstream that is not in mainline. The obvious result is that Debian has a low patch-count.

THAT is why nobody is on Debian's case for a low patch-count. It isn't low at all, if you consider the "Debian developers doing work in the kernel the same way they do work for Debian", and that it is all unpaid work.

On the other hand, I *expect* Canonical to have a rule that anyone doing upstream work of any sort during paid time by Canonical, must do so using an email that credits it to Canonical. I am certain that RedHat, SuSE, Novell, IBM, Intel, and all others HAVE such rules in place.

So one really can't compare the Debian patch-count to the Canonical patch-count in any way that is even remotely favorable to Canonical...

LPC: Fitting into the kernel ecosystem

Posted Sep 18, 2008 7:37 UTC (Thu) by harinath (subscriber, #47697) [Link]

Since Debian doesn't pay its developers, there's no value to contributing upstream under the aegis of Debian: you might as well be a upstream developer directly, after all.

LPC: Fitting into the kernel ecosystem

Posted Sep 18, 2008 16:34 UTC (Thu) by joey (subscriber, #328) [Link]

Debian has several developers who do work on the kernel. Ted T'so comes to mind. Problem for this kind of study is how to categorize such a person. Does Ted work for IBM, or is he a Debian developer, or is he a core kernel contributor? Well, all three. So who gets credited for his work in the statistics? Personally, I think that Ted should. We don't say that IBM, or VA Linux, or Debian is a major force behind ext3/4. Ted is. If the stats don't reflect this, there's something a bit fishy about them.

LPC: Fitting into the kernel ecosystem

Posted Sep 22, 2008 21:03 UTC (Mon) by niv (subscriber, #8656) [Link]

"Debian has several developers who do work on the kernel. Ted T'so comes to mind. Problem for this kind of study is how to categorize such a person. Does Ted work for IBM, or is he a Debian developer..."

Yes, Ted works for IBM.