|
|
Subscribe / Log in / New account

LCA: How to destroy your community

LCA: How to destroy your community

Posted Jan 18, 2010 16:08 UTC (Mon) by lkundrak (subscriber, #43452)
Parent article: LCA: How to destroy your community

This definitely does not apply to me. For free-software products developed by Sun I've had issues with (Hudson, OpenGrok, VirtualBox; never needed to be involved with MySQL or Java communities), the Sun engineers were always very kind, responsive and overly helpful. All of those are perfectly maintained and have prospering communities.

An example of outstanding care of a Sun engineer for the community comes to my mind; once a VirtualBox developer analysed and fixed an issue with RPM Fusion-packaged VirtualBox-OSE directly on my machine as the kernel core dump was too big for me to upload. Without me having paid for support.


to post comments

LCA: How to destroy your community

Posted Jan 18, 2010 16:47 UTC (Mon) by rahulsundaram (subscriber, #21946) [Link]

Doesn't all of these projects require you to sign copyright assignment
agreements with Sun? In the presence of those, thriving *developer*
communities are unlikely to happen. How many non-Sun developers does
VirtualBox have for example?

LCA: How to destroy your community

Posted Jan 18, 2010 17:51 UTC (Mon) by cyperpunks (subscriber, #39406) [Link] (3 responses)

Take a walk into the closed land of "Open"Solaris.

LCA: How to destroy your community

Posted Jan 19, 2010 9:33 UTC (Tue) by marcH (subscriber, #57642) [Link] (2 responses)

OpenSolaris is trying to build a community from scratch. They obviously cannot "destroy" it before it even exists.

LCA: How to destroy your community

Posted Jan 20, 2010 16:17 UTC (Wed) by bronson (subscriber, #4806) [Link]

No, but they can prevent it from ever existing. That's even more effective!

LCA: How to destroy your community

Posted Jan 29, 2010 18:51 UTC (Fri) by segedunum (guest, #60948) [Link]

Hmmmm, you mean the thing they've been building for five years?

You've just touched on another tactic hinted at by the article: If you don't have a community, even if you supposedly started one years ago, and people call you out on it then just claim that you're still building it!

LCA: How to destroy your community

Posted Jan 18, 2010 22:39 UTC (Mon) by zooko (guest, #2589) [Link] (18 responses)

When I wanted to contribute patches to make Python's build process on Solaris more like the
build of Python that ships with Solaris, the Solaris engineers were very friendly and helpful. :-)
More so than the Python developers, I'm afraid. :-(

Also I recently had a very pleasurable interaction with the ZFS crypto developers when they asked
for some general advice on crypto design. :-) They didn't end up using most of my proposals,
but I think this was probably for the very good reason that what they have now is "Good Enough"
and they need to hurry up and finalize it and ship it before Oracle takes the reins. In contrast
when I offered suggestions about crypto to the git developers, Linus publicly ridiculed my
suggestions. :-(

Also when I tried to find out why the GPL'ed UltraSPARC T2 processor had its crypto accelerator
omitted from the source code the Sun employees responsible for the T2 were polite and
responsive to my inquiries.

At the moment I can't recall any negative interactions that I've had with any Sun employees on
open source projects, while I find it easy to think of negative interactions I've had with developers
on various other open source projects.

LCA: How to destroy your community

Posted Jan 19, 2010 4:42 UTC (Tue) by lacostej (guest, #2760) [Link] (1 responses)

I agree.

My interaction with Sun has always been good.

You don't make conclusions on one person's experience , but I feel sad to see
Sun pointed out like that when they do provide some great service to the
community.

I would have preferred a talk made on the positive aspects (how to foster
your community), with references to projects properly managed. And I would
have personally given Sun as a reference, in particular with Hudson (a
popular continuous integration / build server) which has a tremendous
community.

LCA: How to destroy your community

Posted Jan 19, 2010 5:05 UTC (Tue) by zooko (guest, #2589) [Link]

Oh, and I forgot to mention in my list of positive interactions with Sun folks that I ran into a guy who hacks the OpenSolaris installer in a coffeeshop the other night and when I told him about the awesome "apt-clone" hack that the Nexenta folks have written he asked me to put him in touch with them in order to share code. Long live the wonderful positive tradition of collegial sharing and peer review!

LCA: How to destroy your community

Posted Jan 20, 2010 6:08 UTC (Wed) by patrick_g (subscriber, #44470) [Link] (3 responses)

Just for the record could you tell us why the UltraSPARC T2 processor had its crypto accelerator omitted from the source code ?

LCA: How to destroy your community

Posted Jan 20, 2010 7:03 UTC (Wed) by zooko (guest, #2589) [Link] (2 responses)

NSA told them to exclude the crypto engine. This bothers me, because by law in the U.S. free/open implementations of crypto are freely exportable and in addition disclosure of crypto *to* other U.S. persons is completely unregulated. So I feel that it was improper for NSA to pressure them to withhold the GPL'ed crypto engine.

LCA: How to destroy your community

Posted Jan 21, 2010 21:50 UTC (Thu) by mrdoghead (guest, #61360) [Link] (1 responses)

This is appalling. And as usual anything the NSA asks the computer corporate structure in the US - and elsewhere, worse - it gets with little argument. And what the intel-LE types somehow don't get all too willingly they find ways to get otherwise. This stuff needs to stop.

LCA: How to destroy your community

Posted Jan 24, 2010 7:21 UTC (Sun) by zooko (guest, #2589) [Link]

I don't know how to fight against this sort of shenanigans. Giant corporations, even though they are powerful in some ways, are ironically very susceptible to semi-legal pressure from shadowy governmnet agencies. If they had resisted NSA's pressure, there could have easily arisen some unfortunately legal snag that could have, for example, prevented them from exporting their wares until it got cleared up, thus costing them millions of dollars per day until the government gave them the go-ahead.

I politely asked the Sun employees that I was talking to if they would give me the crypto engine source under GPL, given that I was a U.S. citizen living in U.S. borders and thus it was clearly legal for them to share it with me. They didn't write back to that request. ;-)

(I had a vague notion that I might try exporting it myself and become and Internet martyr like Phil Zimmermann did in 1991. It's probably for the best that I never had the chance, as my family depends on me to bring home regular paychecks rather than to engage in dangerous civil disobedience.)

LCA: How to destroy your community

Posted Jan 23, 2010 8:37 UTC (Sat) by sfink (guest, #6405) [Link] (1 responses)

Sure, personal contact is often great. That's why this FOSS stuff works
in the first place. But this article is more about how a controlling
organization can screw things up, independently of how nice
the actual developers or other surviving community members
might be.

LCA: How to destroy your community

Posted Jan 23, 2010 15:51 UTC (Sat) by zooko (guest, #2589) [Link]

Well, I'm not sure what to say. My communication with Sun about the crypto engine of the T2 was fairly "formal" -- it wasn't with people that I had previously met. Also Sun's "Startup Essentials Program" (a hardware sales division) donated a thumper to my little Free Software startup company (http://allmydata.com). That was nice! Basically, I've had nothing but good experiences dealing with Sun as a seller of hardware, a research institute, a sponsor of open source projects, and a collection of humans. This is in stark contrast to the Linux kernel developers, with whom I've had, I'm sorry to say, mostly unpleasant interactions.

Let's turn the burden of proof around: what is the evidence that some specific corporate policy or habit is bad for open source community involvement? Surely many of the open source projects sponsored by Sun have little or no non-employee participation but this doesn't make those projects any different from *most* open source projects. Almost all open source projects have no contribution from anyone other than their founders or the company that sponsors them. So this is not specific evidence that Sun's policies are particularly destructive.

I would be very interested to learn more specific, reliable patterns about community involvement in open source. I've devoted much of the last fifteen years of my life to such projects, and I can say that the open source projects that I've led have had higher levels of community contribution than average. (Where average is zero.)

That's why I read this article with interest (and also read previous posts on the web about Josh Berkus's speech). Surely the points that Josh Berkus makes seem eminently sensible. I'm sure they are good things to keep in mind. But I'm skeptical that they are actual empirical explanations of the failure of OpenSolaris to attract external contributors, or of the success of Hudson to do so. I suspect there are other factors that are more explanatory.

LCA: How to destroy your community

Posted Jan 24, 2010 3:01 UTC (Sun) by johnflux (guest, #58833) [Link] (9 responses)

> In contrast when I offered suggestions about crypto to the git developers, Linus publicly ridiculed my suggestions. :-(

Ah yes, I remember. Your email:

http://www.gelato.unsw.edu.au/archives/git/0506/5298.html

And his reply:

http://www.gelato.unsw.edu.au/archives/git/0506/5299.html

I understand that you see his reply as a bit stinging, but your whole argument was based on the assumption that you could crack md5 in a way that lets you generate a meaningful exploit and then on top of that manage to inject that into the kernel. I can see why Linus responded with sillyness :-)

LCA: How to destroy your community

Posted Jan 24, 2010 7:09 UTC (Sun) by zooko (guest, #2589) [Link] (8 responses)

> Ah yes, I remember. Your email:
>
> http://www.gelato.unsw.edu.au/archives/git/0506/5298.html
>
> And his reply:
>
> http://www.gelato.unsw.edu.au/archives/git/0506/5299.html

That was one of the conversations that I was thinking of, but the message 5298.html that cite above wasn't written by me. I wrote an earlier message in the thread that IIRC was forwarded to lkml by someone else.

> I understand that you see his reply as a bit stinging, but your whole argument was based on the assumption that you could crack md5 in a way that lets you generate a meaningful exploit and then on top of that manage to inject that into the kernel.

I'm not entirely sure what you mean by "crack md5 in a way that lets you generate a meaningful exploit". In the exchange that you cite above, the other person, <linux@horizon.com>, was right and Linus was wrong in respect to the question of whether git users depend on the collision-resistance property of the hash function or not. The truth is that they do, but in a subtle way that most people (including Linus at least at the time he wrote that) don't understand.

At this time (in 2005), when Linus was deciding to stick with SHA-1 for git, certain certificate authorities were deciding to stick with MD5 for signatures, for the same reason -- it seemed to them that they didn't rely on the collision-resistance property. In 2008 it was demonstrated that they did actually rely on that property:

http://www.win.tue.nl/hashclash/rogue-ca/

A similar attack is probably possible on git. It currently costs substantially more than USD 1 million to build a computer that can generate SHA-1 collisions (how much more is not publicly known, but probably less than USD 10 million). For now, only the rich can play.

So while I'm sure that the cryptographers who generated the rogue Root CA (above) can't inject their own code into your git pulls (because they work at public academic institutions and don't have the budget), I'm not sure that the NSA or the Chinese cyberwarriors can't.

> I can see why Linus responded with sillyness :-)

I understand that it seemed ridiculous to him at the time. However he was qualitatively wrong about the properties that git users rely on, and both he and <linux@horizon.com> were quantitatively confused about the cost to generate SHA-1 collisions. (See the rest of the thread that you cited, in which they talk about SHA-1 collisions costing 2^80 computations, when in fact the known upper bound at that time was 2^69. Today the known upper bound on the cost to generate a SHA-1 collision is 2^63.)

One effect of mocking things that seem ridiculous to you is that it deters certain kinds of people from participating in the conversation. I suppose this could be useful if you are right and they are wrong and progress is achieved by getting them to shut up, but of course you take the risk that you were wrong in the first place and by doing this you stay wrong.

I, for one, was reading that conversation at the time, and decided not to join in and try to explain more, in part because I didn't want to have my feelings hurt by mockery and in part because it didn't seem like I would have a good chance of making my point understood.

So to attempt to swerve back onto the topic of this LWN article, when I offered some suggestions to the engineers who are adding crypto to ZFS, they responded with technical arguments that were expressed in polite language. I was therefore emboldended to think that they might actually be listening, and went on to offer more ideas: http://opensolaris.org/jive/thread.jspa?threadID=117092&... . From my very specific, narrow, limited viewpoint, Solaris open source development has been easier to participate in than Linux development. ;-)

LCA: How to destroy your community

Posted Jan 24, 2010 8:44 UTC (Sun) by johnflux (guest, #58833) [Link] (7 responses)

An interesting response, thanks.

Just sticking to the technical side, Linus has been critical of "masturbating monkeys" crowd (his words), that concentrate "on security to the point where they pretty much admit that nothing else matters to them" (his words again). I am not at all surprised at his response to you, whether you were technically right or not (I can't judge sorry).

On the reaction side - he can be a real ass and you got off lightly compared to his scathing on svn developers (just for an example). You do need a thick skin to work on the kernel, and it has actually been something that people are trying to address.

LCA: How to destroy your community

Posted Jan 24, 2010 15:47 UTC (Sun) by zooko (guest, #2589) [Link] (6 responses)

Yes, I'm quite sympathetic to the sentiment, if not to the terminology, that too many security engineers don't understand the concept of "trade-off", or if they do they seem to think that the security knob should be turned to "11" regardless of the consequences.

I don't fault Linus for being impatient with security worriers in the sense of "not wasting his time listening to them", but I do fault him for being impatient in the sense of insulting them.

One thing that has always bugged me about git's use of SHA-1 is that there was very little engineering cost, and probably not too much cost in CPU cycles, to making it secure -- just use SHA-256 instead of SHA-1. (I believe that the reason git uses SHA-1 is the Monotone did. The earliest prototype of git used MD5 because BitKeeper did.)

The engineering cost for upgrading git from SHA-1 to a new algorithm is much higher. I'm not sure how it can be done well. First of all we probably need to deploy a version of git (let's call it version 2 for this conversation) which allows there to be a slot for a new hash value even though it doesn't read or use that space -- it just uses the SHA-1 hash value which is in the other slot. That way once we *eventually* deploy yet a newer version of git, let's call it version 3, which produces SHA-3 hashes in addition to SHA-1 hashes, people using git version 2 will be able to continue to interoperate. (Although, per this discussion, they may be vulnerable and people who rely on their SHA-1-only patches may be vulnerable.)

As far as I understand people using today's git, git version 1, will not be able to exchange patches in any way with people using some future version (which I called "version 3" above) that uses a new algorithm.

LCA: How to destroy your community

Posted Jan 24, 2010 16:09 UTC (Sun) by johnflux (guest, #58833) [Link] (5 responses)

Looking at: http://www.cryptopp.com/benchmarks.html

It seems, roughly, that SHA256 takes about 40% more time than SHA1. From what I understand, the speed of git is determined most by the speed of the SHA1 implementation (Based on a long thread called 'Linus' sha1 is much faster!'). So roughly, switching would make everything 40% slower.

I think that's a trade-off that they wouldn't be willing to make. However, just playing with the C code of the SHA1 code by the git developers ended up making it nearly twice as fast, so I don't know what the optimal speed difference is against SHA256.

If the numbers stay about the same, I think the git guys wouldn't accept a 40% speed decrease.

(On a side point - subscribing so LWN was worth every penny. How I love to have civil conversations with intelligent people.)

LCA: How to destroy your community

Posted Jan 24, 2010 17:14 UTC (Sun) by zooko (guest, #2589) [Link] (4 responses)

Yeah, I was guessing that git is actually network-bound or disk-bound often enough that the CPU hit doesn't matter, but I'm not sure. (According to http://cryptopp.com/benchmarks-amd64.html , SHA-1 is 192 MB/s, SHA-256 is 139 MB/s. Faster than either your disk or your network?)

If git holds out for SHA-3 then hopefully SHA-3 will turn out to be faster than SHA-256. There's even a chance that it will turn out to be faster than SHA-1 on modern CPUs!

LCA: How to destroy your community

Posted Jan 25, 2010 0:32 UTC (Mon) by johnflux (guest, #58833) [Link]

LCA: How to destroy your community

Posted Jan 25, 2010 3:56 UTC (Mon) by njs (subscriber, #40338) [Link] (1 responses)

SHA-1 can definitely be a bottleneck in some situations. The most extreme -- though I'm not sure whether git does this -- is verifying hashes on an initial clone. (The idea is to prevent one person's disk corruption on an old file or whatever from spreading throughout the network of clones.) Here the disk and network cost is proportional to the size of the delta-compressed repository, but the SHA-1 cost is proportional to the size of the uncompressed repository, which can easily be in the terabyte range.

It can also easily be the bottleneck on, say, committing a large merge (many modified files, all in cache because they were just written).

LCA: How to destroy your community

Posted Dec 29, 2015 18:11 UTC (Tue) by Spitfire19 (guest, #106038) [Link]

I feel that you would also have a big hit when you are performing CI tasks, as for some scenarios you may want your automated tool to delete everything in given directory before pulling and checking out the latest commit.

LCA: How to destroy your community

Posted Jan 27, 2010 11:58 UTC (Wed) by broonie (subscriber, #7078) [Link]

In normal workflows you end up with the files you're accessing in cache so there's no I/O to physical devices and most operations do get CPU bound, especially read only ones. For performance purposes git pretty much assumes that most of the time you're running from hot cache.


Copyright © 2025, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds