Kees Cook: yay for barriers

Posted May 18, 2010 14:42 UTC (Tue) by ricwheeler (subscriber, #4980)
Parent article: Kees Cook: yay for barriers

I think that others have clarified that Fedora/RH developers actually did engage.

As part of the RH file system team, I think that the right way to answer users (even when they work for distros) is to help them understand how to engage effectively and leverage the community.

For ext3/4 bugs, we have a weekly ext4 call, an open IRC channel and the ext4 mailing lists. We get a lot of issues to debug and everything needs to be prioritized. You don't need to hire developers, but clearly having active developers gives you more weight in pushing for your issues.

No slight meant or intended, but a part of open source is learning how to engage and motivate others to fix your issues first for free (often on their own time!).

Kees Cook: yay for barriers

Posted May 19, 2010 20:39 UTC (Wed) by plougher (guest, #21620) [Link] (5 responses)

No slight meant or intended, but a part of open source is learning how to engage and motivate others to fix your issues first for free (often on their own time!).

In other words you're advocating what many RedHat/Fedora fanboys regularly bash Canonical for doing. Surely the whole point of this thread is you can only take this so far, once you get a reputation for freeloading, you loose the goodwill of upstream developers and this winning strategy ceases to work.

As an ex-kernel maintainer for Canonical I'm no fan of Canonical, however, the whiter than white holier than thou attitude of RedHat/Fedora fanboys astonishes me. As an upstream author I have had instances where both Canonical and RedHat have "engaged" with me to fix bugs, and frankly their behaviour is identical, if anything I've had more patches from Canonical than RedHat.

Kees Cook: yay for barriers

Posted May 20, 2010 0:14 UTC (Thu) by jspaleta (subscriber, #50639) [Link] (4 responses)

Hey credit where credit is due. The criticism aimed at Canonical over this particular issue is not being initiated by Redhat/Fedora. This particular round was touched off by Ted (a Google employee) in the original launchpad ticket and brought to the attention of lwn by (arjan)an Intel employee. Is Ted Ts'o a fan of Red Hat/Fedora? That's news to me. Considering his quick response in the original launchpad ticket I would have thought he was a fan of Ubuntu.

Ted's very expansive comment in the launchpad ticket commenting on Canonical's resource staffing touched off some latent frustration to be sure. But I'm pretty confident that latent frustration goes beyond the Red Hat fenceline and further even than the Fedora bleachers where I'm munching on popcorn watching the show.

If that comment from Ted wasn't there coloring the discussion would have this been an issue at all? probably not. I doubt Dave Airlie would have dropped a note in his blog about the incident. I doubt arjan would have broadcasted it to lwn. What's sensational and atypical about this situation is Ted's comment... not anything in particular that Kees has done. If you look really closely people have been reacting primarily to Ted's comment not to the handling of the bug itself.

Ted's comment even colored the response to the Fedora ticket, which has been apologized for:
http://www.outflux.net/blog/archives/2010/05/17/yay-for-b...

And furthermore, the historic Canonical criticism over upstream contributions have come primarily from other quarters than Red Hat/Fedora.
It's a bit revisionist to lay this at the feet of Red Hat. The video where GKH(Novell employee) made his initial off-the-cuff remark about Canonical not contributing was actually a talk meant to shame Google into doing more upstream contributions. I find it more than a little ironic that a Google employee is now harshing on Canonical's staffing practises so publicly given that Google's less than stellar track record with regard to mainlining kernel code..cough android patches..cough. Doubly ironic that somehow Red Hat/Fedora is getting a black eye over getting suckered into reacting to an emotive display from a Google employee when Canonical and Google are ostensibly partners on ChromeOS development. Perhaps its time for GKH to ratchet up the scrutiny of Google again.

You see more upstream patches from Canonical as an external corporate entity? Great! What codebase are you talking about exactly? I'm sure Canonical supporters would find that a refreshing change from the linux kernel development reports that lwn produces which don't rank Canonical as a significant contributor.

In fact I would love to see the sort of contribution analysis that has been used in the kernel and plumbing layers extended further up the stack. I'd love to see the extent to which corporate entities are contributing to GNOME and KDE for example or the collections of projects hosted under the freedesktop.org banner. Credit where credit is due.

-jef

Kees Cook: yay for barriers

Posted May 20, 2010 0:58 UTC (Thu) by paulj (subscriber, #341) [Link] (1 responses)

To be fair, I think I may have started that particular sub-thread, with an oblique but obviously still reasonable clear reference to Airlied's blog entry. Airlied's blog is aggregated via Kernel Planet and is how I came to read about this bug.

You're right though, this isn't about RedHat per se. People representing various vendors and/or projects have vented about their competition at various times. It's not that abnormal an urge. However, it's an urge that's best resisted - it doesn't look good. Worse, given we all depend on the same code base, it raises barriers between people.

There are some big questions here about how different vendors should inter-operate when each specialise in different areas, each employ maintainers of different parts of a shared software stack, and hence all are inter-dependent (to various degrees) on each other. Clearly this is raising tensions.

I wonder how those tensions could be resolved though?

Kees Cook: yay for barriers

Posted May 20, 2010 3:01 UTC (Thu) by jspaleta (subscriber, #50639) [Link]

You've made an underlying assumption. Different vendors are specializing at different layers. Are they? Are they really?

Red Hat isn't. They are up and down the stack from kernel up into user facing GNOME UI.

Novell isn't. They are up and down the stack from kernel up into user face GNOME UI.

If I were a betting man I would wager that Red Hat and Novell's relative contributions compared to other corporate entities are probably similar at every level of the software stack...up through freedesktop components into the GNOME project...as well as Apache projects as well.

What about Intel and Nokia? What about Google? What about Oracle/Sun? Are they specializing at one level of the stack are are they investing in engineering expertise at all levels? Obviously Intel and Nokia are building a common stack in MeeGo..but its an entire stack. It may diverge from GNOME and KDE...but its an entire stack from the kernel on up and both companies are contributors at multiple layers.

Who are the top ten corporate code contributors to GNOME? Who are the top ten corporate contributors at the freedesktop component level? Do you think they are significantly different than the corporate entities that are pulling the work in at the kernel level? Noone has done the analysis of contribution beyond the plumbing layer

And let me remind you...that Canonical executives specifically and repeatedly has been trying to make the argument that there should be a core around which many vendors agree to work from and then specialize out from there. These sort of statements add to the tension because its clear that Canonical isn't doing the upstream work to shape the core. Whatever they are specializing in.. its not the core components. And its not GNOME,not KDE and its not freedesktop, and it doesn't appear to be MeeGo either.

Canonical wants to sell the story that they are a design specialist up high in the stack. Great. Good for them, if you are willing to buy that story. But that doesn't imply that other vendors are also specialists..and it doesn't mean its a successful strategy for a vendor who ultimately is relying on support contract revenue for the entire integrated stack.

And even then the specialist concept rings hollow for Canonical. How much effort on behalf of ARM OEMs is Canonical putting into in-house kernel and kernel plumbing engineering? Would you say its a trivial amount? How much of that work is making its way into the upstream ARM kernel? I don't know the answer to any of those questions. I suspect its a non-trivial amount of engineering work going on there deep in the stack.

-jef

Kees Cook: yay for barriers

Posted May 20, 2010 2:57 UTC (Thu) by plougher (guest, #21620) [Link] (1 responses)

Hmm why concentrate on Redhat/Fedora? Simply because I was responding to a RedHat employee (the Manager & Architect for the File System Group no less) that mentioned practices rather similar to the practices that have caused Canonical so much criticism... Anyone who knows the reasons why I left Canonical will know I have absolutely no love of Canonical.

But that's slightly misinterpreting the point of my comment, my point was that all distros rely on the goodwill of upstream authors, and to repeatedly single out Canonical for this practice is disingenuous. Ted Tso criticised Canonical for immediately passing a bug upstream to him with tight time constraints, but this is no different to what Redhat and other distros regularly do with their practice to quote Ric Wheeler "engage and motivate others to fix your issues first for free (often on their own time!)".

Kees Cook: yay for barriers

Posted May 20, 2010 3:38 UTC (Thu) by jspaleta (subscriber, #50639) [Link]

I think the kernel development stats and the plumbing development stats stand in stark contrast to the point you are trying to make.

I don't think anyone could suggest that Red Hat isn't pulling its fair share of the upstream project engineering weight anywhere in the software stack.

I'm really not sure you can make the argument stick that Red Hat's engineering practises encourage passing along difficult bugs to developers outside the corporate fenceline when time constraints are tight. Considering the amount of involvement that Red Hat has up and down the stack. I really doubt anyone feels pressured to fix an issue for Red Hat on Red Hat's behalf on Red Hat's release timescale. Assuming that really was Ted's motivation for venting in the launchpad bug..which I can't really know for sure.

I think what Ric was trying to talk about how one specific Red Hat team tries to set prioritizes in a collaborative open fashion at the upstream project level to maximizing long term benefit to the project's goals by taking into account both vendor staffed manpower as well as volunteer manpower to get the most work done.

There's always tension between business interest and volunteer interests. I think Red Hat does a very good job of delineating the two and staffing paid manhours for the items that are critical to their business interest. So when push comes to shove...Red Hat can allocate its own manpower to solve its business problems. In this way Red Hat's engineering investments do more to serve the larger upstream project's interest than the volunteers serve Red Hat's business interest. I get the sense that with Canonical that relationship is backwards. Canonical is trying to organize as much volunteer interest to serve their business interests primarily. And that is where the frustration lies. Not with Kees specifically.. but with a engineering management practise that expects more from upstream projects than is given back.

-jef