Centralized vs decentralized [LWN.net]

Centralized vs decentralized

Posted Feb 24, 2004 15:26 UTC (Tue) by lm (guest, #6402)
In reply to: subversion 1.0 is released by veelo
Parent article: subversion 1.0 is released

With respect to Mr Hudson's comments about the decentralized model versus a centralized model, I have a few comments.

1) As Mr Hudson pointed out, the BitKeeper model is a superset of the Subversion model. In other words, you can use BitKeeper the same way you might use Subversion, it has that capability, but you can't use Subversion the way you might use Bitkeeper, Subversion doesn't have that power. I'm not making this up, read the Subversion mailing lists, they have freely admitted that SVN will never do what BK can do.

2) Given that Mr Hudson works on Subversion, his statements are far from unbiased. In fact, they could be compared to a buggy manufacturer touting the advantages of the horse and buggy when presented with the automobile. Of course he thinks his product is better but that doesn't mean it is. The same can be said of my comments, so salt heavily. The difference is he's working on the buggy, we're working on the car. Take your pick.

3) His claims that the BitKeeper model as used by the Linux kernel team is inferior are simply not substantiated. Since switching to BitKeeper, the Linux development pace has gone up by more than 2x, no matter how you measure it. Mr Hudson claims that the model used by Apache, for example, is better than the Linux model. Hmm. When the Apache team can review and integrate changes at the rate of 18,500 patches a year (more than 50 a day) then perhaps he might have a point. Until then we are left to wonder if maybe the Apache team is working on a far easier problem.

4) Mr Hudson might want to stop to consider that every time there is parallel development with a tool like SVN information is lost. If you have N way parallel development you lose N-1 events in SVN. You lose none in BitKeeper. Anyone who has had to "unmerge" their work from some previously integrated bad code knows what I mean.

Let's review. The Subversion team admits that BitKeeper can do what Subversion can but Subversion can't do what BitKeeper can do. The kernel uses BitKeeper and the development pace doubles. This leads us to the obvious conclusion that the BitKeeper model is somehow inferior for open source work. Maybe it's just me, but I'm not following the logic.

(Log in to post comments)

Bitkeeper, the universe, and everything

Posted Feb 24, 2004 19:20 UTC (Tue) by lm (guest, #6402) [Link]

Hi Dirk,

I can't do the comparison to Arch objectively because we produce a product with a similar model. I suspect, however, that an objective comparison would come up with things like:

BK is 6 years old and has fulltime talented staff working on it. Hence it is more mature, better docs, better platform support, etc.
BK scales better. Try and import 38,000 changesets in Arch and watch what happens. That's not to say that BK is zippy, it needs be better, but it can at handle the kernel in the free version and upcoming commercial versions are much faster.
BK merges better, both automerges and hand merges.
BK is easier to use. We add a new user who syncs with bkbits.net every 45 minutes. More than 50,000 users have figured out how to use BK without ever talking to us. I've designed and built two different SCM systems, I'm well versed in at least 8 others, and I have a tough time with Arch. SVN wins that battle hands down.
BK has far more infrastructure built up in it to handle all those little tasks that come up when doing SCM things. BK can import/export CVS trees, for example. One customer said "we rolled out BK and threw away the 6,000 lines of perl we had wrapped around CVS, BK does all that". I can't emphasize this point enough. We can all argue which model is better in theory but even if you have the perfect model there are 1000 details that have to be handled that have nothing to do with the model.
Better testing. 1/4th of the BK source base is regressions. We run regressions on all platforms all the time, it's a huge stability win.
Better platform support. We support Windows, MacOS, all the BSD/Linux and commercial Unux platforms equally.
GUIs. We have some pretty pleasant GUIs and they are getting better, see bkexplorer for example. Not exactly eye candy but it gets the job done. And we offer Visual Studio integration, etc.
More workflows. See this description of using BK on a submarine or this talk on various BK workflows.
Infrastructure you can use. We have a free hosting service at bkbits.net which hosts the kernel, mysql and a pile of other projects. No ads, no nonsense, just a place to put your stuff. And it's a ton faster than sourceforge.
Business model. We have a business model that pays for BK development. No outside investors, hence no stupid business decisions which hurt the end users. Contrast with Arch where the primary developer is forced to beg for handouts or SVN where HP paid for the development. You may or may not like our model but SCM tools are annoyingly complex (it took SVN 3.5 years to get to a CVS clone) and you need some development effort that isn't going away.

Personally, I am starting to think it is great that Arch et al exist. They get people away from the licensing discussions and get them focussed on the model. We like the distributed model, it clearly works better, and if you have to use open source tools we'd rather have you learning about the model we think is better.

The difference between Arch and BK is going to be a lot like the difference between GCC and Microsoft's compiler. I just timed compiling BK on windows with gcc vs Microsoft's compiler: 10m41s vs 4m40s. They both do the same thing at some level, one just works better. You pays your money and takes your choice and the old adage you get what you pay for still applies.

Bitkeeper, the universe, and everything

Posted Feb 24, 2004 21:06 UTC (Tue) by ballombe (subscriber, #9523) [Link]

You know better than comparing compilation time of compilers.

Bitkeeper, the universe, and everything

Posted Feb 24, 2004 21:26 UTC (Tue) by khim (subscriber, #9252) [Link]

I just timed compiling BK on windows with gcc vs Microsoft's compiler: 10m41s vs 4m40s. They both do the same thing at some level, one just works better.

Have you tested it with Intel Compiler ? Last time I checked Intel compiled required few times more then even gcc. And it's still cost extras and it is better. It's stupid to compare compiler on speed.

Bitkeeper, the universe, and everything

Posted Feb 24, 2004 23:10 UTC (Tue) by piman (subscriber, #8957) [Link]

Hi Larry,

You put the rest of us in something of an unfair position. The BitKeeper License (as I find it at http://www.bitkeeper.com/bkl.txt) clause 3c prevents those developing products similar to BK (i.e. other SCMs or SCM-related software) from using BitKeeper under its free license. Thus, the Subversion, Arch, or Monotone developers can't try out BitKeeper to compare features, speed, and so on. Whereas since those are all released under a free software license, you are free to try them out and evaluate them even though you are competing with them.

Thus, no free SCM developers can provide a fully informed (if biased) critique of BitKeeper in the same way you can provide a fully informed (if biased) critique of Arch or Subversion. If you are so sure about BitKeeper's capabilities compared to the competition, perhaps you should remove clause 3c.

Your baiting comment about compilers is disappointing; given the amount of software development you've done, you should know better than to provide such a useless benchmark.

Bitkeeper, the universe, and everything

Posted Feb 24, 2004 23:26 UTC (Tue) by aya (guest, #19767) [Link]

Hi!

It's convenient that you mention speed, Robert Collins has been working on code lately to speed up arch by cutting down on system calls for file info. This probably won't be merged until 1.3, though. (The current release is 1.2-pre2.)

I guess I'm not qualified to say anything about BK's merge capabilities; I'm not going to use the free version, since I may hack arch in the future if I have time, and I'd rather not budget a commercial license. (Please note, this isn't a license flame. It's your software, it's your license, and I'm fine with that. In any case, I know better than to try to resurrect a horse that's been kicked to a million deaths already ... Besides, I'm glad the kernel team is using something better than the previous process.)

I do know that I don't really have a handle on arch's merging features, though, so at the very least your comment about documentation rings true. UI and documentation are commonly cited as GNU Arch shortcomings. Still, from what I've seen, arch's merge operations aren't *that* far off from BK's, though I suspect BK's are easier to use overall. The general use pattern is the same; develop in a local branch/repository of a project, filter changes into an integration branch, and have the project lead merge from your integration branch.

I will say this about the license, though: the BKL doesn't seem to be linked from any page on bitmover.com anywhere anymore. To read it, you seem to have to download and run Bitkeeper. I realize people bitch about the thing all the time, but that's not good for people who want to read the license to see if they can use BK before downloading it ...

Bitkeeper, the universe, and everything

Posted Mar 30, 2004 2:57 UTC (Tue) by ghudson (guest, #20530) [Link]

I'm not sure where Larry got the idea that "HP paid for the development" of Subversion; it sounds like he thinks some big company felt charitable one day and decided to dump a few EFTs into open source SCM development.

Subversion development is hosted and partly funded by CollabNet (which I have no financial association with), which as I understand it is a startup company whose business model is to sell a commercial product which either does now or will soon integrate Subversion, among other components. By making Subversion open source, I believe they hope to be better able to compete with the heavies like IBM/Rational than they could either by integrating CVS or by developing a proprietary SCM system.

Centralized vs decentralized

Posted Feb 27, 2004 4:12 UTC (Fri) by brouhaha (subscriber, #1698) [Link]

Subversion can in fact do one thing that BitKeeper can't. It can be used for development of new and improved free software (or open source) version control systems.

On multiple occasions the ability to patch CVS and Subversion has been useful to me. And in general, I try to run as little closed source software on my computers as possible.

If I really needed the decentralized model, I think I'd investigate GNU Arch before switching to BitKeeper.

Years ago, I attended a talk you (L.M.) gave at SVLUG about BitKeeper. I was fully prepared to dislike it due to its being based (at that time) on SCCS. But you did such a good job of explaining the reason for that decision, the advantages of it, and the advantages of the decentralized model in general, that I left the talk feeling like I'd been fully converted to the BitKeeper religion.

But when I tried to push BitKeeper at my job, I immediately ran into trouble. Emailed requests for pricing and other licensing information went unanswered. And the per-seat pricing information that was available on the web site was ludicrous. The per-seat price went UP as the number of developers increased. My company was near one of the quantity thresholds, and we were concerned that if we added just a few more developers, we would cross that threshold and suddenly have to pay not just a higher price for the new seats, but possibly to "upgrade" the old seats as well. Yet no one at BitMover cared to even answer questions about these policies.

By comparison, when I investigated Perforce, I had no difficulty finding out about their pricing and licensing.

Ultimately the company chose to stick with CVS. Were the decision to be made today, Subversion probably would be a serious contender. In fact, I'm trying to encourage my current employer to switch from CVS to Subversion.

I fully recognize that BitKeeper is a much more powerful system, and I even agree with your arguments as to why free software isn't likely to match it any time soon. But eventually free software will catch up. It wasn't that many years ago that people didn't think free software would match Excel, and I'm not sure it does 100% even now, but it's close enough.

Centralized vs decentralized

Posted Mar 30, 2004 2:25 UTC (Tue) by ghudson (guest, #20530) [Link]

I just found this comment, and want to raise a few objections.

(1) I wouldn't say Subversion developers "have freely admitted that SVN will never do what BK can do." Some Subversion developers are very keen on having good support for distributed development. (I'm not one of those people, and for all I know, Larry may be right that a tool like SVN can never evolve into a tool as good as Bitkeeper for distributed development. But I don't think it's a foregone conclusion.)

Moreover, the logic here seems a bit simplistic. I could probably name some feature of Subversion that Bitkeeper can't currently match (exporting a repository for read-write via WebDAV, just as a wild guess) and make the same claim in reverse, but that's only important to people who want that feature.

(2) I certainly have my biases (perhaps there should be a disclaimer in the essay in question; I'll think about it), but I have no vested interest in Subversion and work on it only out of personal interest. I would argue that my conclusions about version control systems have led me to contribute to Subversion, not the other way around. I'm also not a very big Subversion evangelist, despite my role in its development; at work, we still use CVS.

(3) "Since switching to BitKeeper, the Linux development pace has gone up by more than 2x, no matter how you measure it" only establishes that BitKeeper is better than manual patch management for projects already organized around the pyramid system. I have no doubt that this is true, but it misses the thrust of my essay. (Also, though my essay mentions the Apache model, there are projects like Mozilla and the *BSDs which have a similar pace of development to Linux, and they do alright with centralized SCM.)

I'll confess to not fully understanding point (4); if it's just about a lack of merge history tracking, that's a known (and hopefully temporary) weakness of Subversion. Merge tracking is not directly related to distributed functionality or the pyramid model, although there are some tie-ins.

The point of my essay is not to denigrate Bitkeeper's quality--on the contrary, everything I hear suggests that it's a very polished product ("polish" being one of those intangible, subjective, and crucially important features of any piece of software). I only question its necessity for open source projects.

Centralized vs decentralized

Posted Mar 30, 2004 2:41 UTC (Tue) by kfogel (guest, #20531) [Link]

This mostly fails to address the points Greg Hudson was making.

Point number (1) above, and the final paragraph, are not particularly compelling. Being a formal superset is not the same as being a *convenient* superset.

Imagine a discussion about the relative merits of two programming languages, where someone says "Well, language X is Turing-complete, and so is language Y, therefore Y clearly can't claim any advantage." Anyone who's tried to write large, modular applications in (say) both Lisp and C would probably disagree -- sure, they are formally equivalent, and you could even paint C as a "superset" of Lisp, in that one is much more likely to implement Lisp efficiently in C than vice versa. Yet many people prefer Lisp, for the kinds of tasks they face.

Likewise, Greg is making the twofold claim that a) a certain kind of development process is more likely to succeed, and b) BitKeeper is not as well-suited to that process as centralized-repository systems.

The comparison of Linux's pre-BitKeeper and post-BitKeeper change incorporation rates is somewhat spurious. Greg isn't arguing that BitKeeper wouldn't help a single integrator do his job better; he's arguing that that's not a good model in the first place. I'm not saying he's right or wrong, just asking that his argument be addressed, not evaded. A more useful analysis would have rebutted Greg by saying that the way the Linux kernel project uses BitKeeper is not the way Greg claimed they would (if that's the case -- I don't know).

Greg's essay was written in such a way as to make quality disagreement easy. He provides obvious hooks on which to hang competing claims; he makes clear, specific statements which could turn out to be wrong or incomplete, and he left doors open for that eventuality. It was well-written and thoughtful, and deserved a response in kind. I don't feel it got one here :-(.