LWN: Comments on "Bazaar on the slow track"

Bazaar on the slow track

jelmer — Sun, 09 Dec 2012 20:16:26 +0000

You could argue that Canonical should give up on Bazaar because that's what the majority of FOSS projects is using nowadays, but not that they started Bazaar for the sake of it. Bazaar predates Git.

Git and Mercurial were both started after the BitKeeper fiasco sometime around March 2005. bzr was already well underway in February of that year; http://sourcefrog.net/projects/bazaar-ng/doc/news.html

Bazaar on the slow track

fw — Wed, 03 Oct 2012 10:07:49 +0000

I did another test, and the numbers are now:

bzr: 41s, 27.7 MiB
git: 35s, 24.9 MiB

So bzr and git are roughly in the same ballpark, at least for this test (pulling changes into a local repository which hasn't seen any local changes).

Bazaar on the slow track

smurf — Wed, 26 Sep 2012 14:38:27 +0000

I beg to differ; there are a couple of other good reasons why systemd is a good idea.

Before: "Owch, the job failed to start. Now which syslog file did it log its error message to? Oops, it was stderr, so hack the initscript; lather,rinse,repeat".
After: "systemctl status NAME.service"

Before: "This job needs to auto-restart when it dies for whatever reason. Write a hacky wrapper which utterly fails to *not* restart the thing when it *should* die."
After: "This job needs to auto-restart? Add a single well-documented line to the .service file."

And so on.

Bazaar on the slow track

makomk — Wed, 26 Sep 2012 10:00:38 +0000

Ubuntu are hardly the first distro to go off on their own with their init system, though. The days Debian's using a weird dependency-driven system (apparently derived from OpenSuse?), and Gentoo's pretty much always had its own init system (though I think some of its tools are Debian-derived). Once you've got something that's actually dependency-driven, switching to systemd suddenly starts to look a lot less worthwhile.

Bazaar on the slow track

makomk — Wed, 26 Sep 2012 09:34:04 +0000

git rebase should always save the previous state of the branch being rebased in ORIG_HEAD, so if you screw it up and get the branch names the wrong way around you should just be able to do "git reset --hard ORIG_HEAD" so long as you realise immediately (or "git rebase --abort" if the rebase failed partway through).

Mercurial mostly written in Perl?

robinst — Sat, 22 Sep 2012 13:02:00 +0000

It's pretty funny that the Ohloh comparison shows Mercurial to be "Mostly written in Perl"! The same is also shown on the language details for Mercurial.

Turns out that Mercurial uses something similar to shell scripts with a .t extension for test cases, which the language detector categorizes as Perl. Luckily Ohcount, the library which does the counting, is on GitHub, so I submitted a pull request to change that.

Bazaar on the slow track

fw — Fri, 21 Sep 2012 19:37:14 +0000

Git had much more history to pull. I tried again with the last few days (both incremental this time), and the speed is comparable (15 seconds each). Git still transfers slightly less data (3.8 MB vs 4.3 MB). I should probably do this comparison again after two weeks.

Bazaar on the slow track

nix — Wed, 19 Sep 2012 21:52:23 +0000

Quite. I just pulled the last five days of Emacs development with bzr 2.5. It pulled down 16905Kb at wildly varying rates all well below line speed, and re-evaluated its infuriating 'estimate' of amount to be transferred five times. Does anyone believe that Emacs actually had 17Mb of changes in the last week? You'd be right.

Getting a diff of the changes on trunk -- entirely in core -- took four seconds (not too shabby but still terribly slow compared to git, where in-core diffs often run at rates in excess of 30,000 lines per second), and revealed

311 files changed, 3025 insertions(+), 2658 deletions(-)

A wc of the output says

12952 55987 447812

so "bzr pull" had to receive roughly thirty times as much data as was actually changed, and that's despite the fact that patches contain lots of redundant context info that hasn't actually changed in any way at all.

I'm sorry, but no matter which way you slice it, bzr is still hilariously inefficient. It may not be *unusably* inefficient anymore, but that's damning with very faint praise.

Bazaar on the slow track

smurf — Wed, 19 Sep 2012 19:52:25 +0000

What he said.

A "clean" history (meaning "to the best of my knowledge, every change transforms program X1 into a strictly better program X2") means that you can take advantage of one of git's main features when you do find a regression.

Bisecting.

If you do break something, "git bisect" requires ten compile-test-run cycles to find the culprit, among a thousand changes. Or twenty cycles if you have a million changes. (OK, more like 13 and 30, because history isn't linear, but you get the idea.) If you try to keep track of that manually you'd go bonkers.

Of course this isn't restricted to git. bzr and hg also implemented the command. The idea was too good not to. ;-)
I don't know how well they do in finding reasonable bisection points in a complex revision graph; git's algorithm is very good these days.

Bazaar code metrics

bronson — Wed, 19 Sep 2012 16:49:26 +0000

Interesting! It looks like Bazaar is seeing a lot of churn?

                 bzr             hg              git
Lines Added	 658,329 lines	 107,171 lines	 830,470 lines
Lines Removed	 332,498 lines	 39,803 lines	 268,636 lines

Also, the 30 day numbers are currently telling a different story than the 12 month numbers.

Still, overall, that's impressive work per capita by the bzr developers.

Bazaar code metrics

jermar — Wed, 19 Sep 2012 15:45:21 +0000

Ohloh provides a useful comparison between Bazaar, Mercurial and Git based on accurate code metrics. From there, it follows that Bazaar has had the fewest developers in total and also during the past 12 months. But surprisingly, the code activity (2589 commits) produced by its 35 contributors last year is much higher than the activity produced by Mercurial's 115. It's rather closer to what Git's 200 contributors did during the past 12 months. The trend of lines-of-code is kind of logarithmic, but so is Git's. From these statistics, there appears to be a drop in number of developers and commits, but given the absolute numbers above, it's probably nothing dramatic.

Bazaar on the slow track

dlang — Wed, 19 Sep 2012 15:31:36 +0000

one of the features of git is the ability to recreate and combine patches before pushing them upstream.

Yes, this can be abused to combine a huge amount of work into one monster patch.

But it can be used sanely to re-order and combine patches from a line of development into a clean series of logical patches.

When you are developing something, if you check it in frequently as you go along, you are going to have cases where you introduce a bug at one point and don't find and fix it for several commits. You are also going to have things that you do at some point that you find were not the best way to do something and that you change back later (but want to keep other work you have done in the meantime)

you now have the option of either pushing this history, including all your false starts, bugs, etc.

Or you can clean the history up, combining the bugfixes with the earlier patches that introduced the bug, eliminating the false starts, etc and push the result.

The first approach has the advantage that everything is visible, but it has the disadvantage that there are a lot of commits in the history where things just don't work.

If the project in question encourages the use of bisect to track down problems, having lots of commits where things just don't work makes it really hard for users trying to help the developers track down the bugs.

As a result, many projects encourage the developers to take the second approach.

Now, many developers misunderstand this to mean that they are encouraged to rebase their entire development effort into one monster patch relative to the latest head, but that's actually a bad thing to do.

And in any case, the history is still available to the developer, they are just choosing not to share that history with the outside world.

Bazaar on the slow track

pboddie — Wed, 19 Sep 2012 14:22:12 +0000

note that git doesn't force you to throw away the history.

I think that the intention may have been to describe the apparently common practice, particularly amongst git-using projects, of aggressively rebasing everything and asking people to collapse their contributions into a single patch so that the project history is kept "clean".

Bazaar on the slow track

hummassa — Wed, 19 Sep 2012 11:17:54 +0000

After re-reading, I *think* the "9.5 months" mean that the "entire git mirror" refers to more development time... (maybe twenty or so years?) can you clarify this? And as you stated, full mirroring is really fast...

Bazaar on the slow track

hummassa — Wed, 19 Sep 2012 11:03:46 +0000

> I pulled about 9.5 months of Emacs development using bzr 2.6.0dev2. It took 5.5 minutes and roughly 170 MB of network traffic. Cloning the entire Git mirror took 14 minutes and 690 MB

Care to elaborate that? You just stated that git was 3x slower than bzr, so I'm confused.

QotW material

man_ls — Tue, 18 Sep 2012 22:04:19 +0000

I would like to nominate your comment for "Quote of the week" for distributions, if it is not too late. It is sarcastic, it has the LOL factor, and it is true as life.

Bazaar on the slow track -- Montone gets too little attention

jackb — Tue, 18 Sep 2012 21:16:45 +0000

I think there are two components necessary to build a web of trust that real people will actually use. First is automated in-person key signing that I described in an eariler post. The second part is an online database of facts about a particular identity.

The database would consist of one table that associates arbitrary text strings with public key IDs, and another table containing cryptographically-signed affirmations or refutations of the entries in the first table.

An example of an arbitrary text string could be a legal name, an email address, "inventor of the Linxu kernel", "CEO of Acme, Inc.", etc.

Everybody is free to claim anything they want, and everyone else is free to confirm or refute it. A suitable algorithm would be used to sort out these statements based on the user's location in the web of trust to estimate the veracity of any particular statement.

The value of the web of trust depends on getting people to actually use it so the tools for managing it would need to be enjoyable to work with instead of painful. That's one reason I think making the user interface similar to a social network because the emperical evidence suggests that people like using Facebook more than they like using GPG or OpenSSL. The other reason is that social networks better model how people actually interact in real life so making the web of trust operate that way is more intuitive.

Bazaar on the slow track

fw — Tue, 18 Sep 2012 20:36:41 +0000

Just for testing, I pulled about 9.5 months of Emacs development using bzr 2.6.0dev2. It took 5.5 minutes and roughly 170 MB of network traffic. Cloning the entire Git mirror took 14 minutes and 690 MB. Now cloning from scratch is always more efficient, but these numbers still don't look very good for bzr.

Bazaar on the slow track -- Montone gets too little attention

Cyberax — Tue, 18 Sep 2012 20:25:58 +0000

PKI is a failure on all levels, starting from technical and going up to the social/management level.

For example, is there anybody here who can claim enough of ASN.1 knowledge to parse encoded certificates and keys? I certainly don't, every time I need to generate a CSR or a key, I go to Google and search for the required command-line to make OpenSSL spit out the magic binhex block.

Then there's a problem with lack of delegation. It's not possible to create a master cert for "mydomain.com" which I then can use to sign "host1.mydomain.com" and "host2.mydomain.com".

And so on. I'd gladly help a project to replace all this morass with clean JSON-based certificates with clear human-readable encoding.

Bazaar on the slow track -- Montone gets too little attention

graydon — Tue, 18 Sep 2012 20:23:28 +0000

I agree. My hunch (currently exploring in code) is that a more useful model involves defining trust in reference to cross-validation between multiple private small-group communication-histories. Put another way: identity should adhere to evidence concerning communication-capability (and the active verification thereof), not evidence of decrypting long-lived keys. Keys should always be ephemeral. They'll be broken, lost or stolen anyways; best to treat them as such.

(Keep in mind how much online-verification comes out in the details of evaluating trust in our key-oriented PKI system anyways. And how often "denying a centralized / findable verification service" features in attack scenarios. Surprise surprise.)

So, I also expect this will require -- or at least greatly benefit from -- a degree of "going around" current network infrastructure. Or at least a willingness to run verification traffic over a comfortable mixture of channels, to resist whole-network-controlling MITMs (as the current incarnation of the internet seems to have become).

But lucky for our future, communication bandwidth grows faster than everything else, and most new devices have plenty of unusual radios.

Bazaar on the slow track -- Montone gets too little attention

jackb — Tue, 18 Sep 2012 20:05:24 +0000

Now that people are carrying mobile, internet-connected computers around with them basically all the time key signing can be automated.

The question of "does the person standing in front of me control a particular private key" can be answered by having each person's smartphone sign a challenge and exchange keys via QR codes (bluetooth, NFC, etc). This step should require very little human interaction.

That question, however, does not establish an identity as we humans understand it. Identity between social creatures is a set of shared experiences. The way that you "know" your friends is because of your memories of interacting with them.

Key signing should be done in person and mostly handled by an automated process. Identity formation is done by having the users verify facts about other people based on their shared experiences.

If properly implemented the end result would look a lot like a social network that just happens to produce a cryptographic web of trust as a side effect.

Bazaar on the slow track -- Montone gets too little attention

hummassa — Tue, 18 Sep 2012 19:43:21 +0000

> The fraction of the population that really understands what it means to assign cryptographic trust to a key is statistically indistinguishable from "no one". Maybe the reason that the web of trust we've been promised since the 90s hasn't appeared yet is because the model itself is broken.

Ok, but... what is the alternative?

Bazaar on the slow track -- Montone gets too little attention

jackb — Tue, 18 Sep 2012 18:59:02 +0000

All that's a distraction though, at this stage. Git won; but there's more to do. I agree with you that the residual/next/larger issue is PKI and naming. Or rather, getting _rid_ of PKI-as-we-have-tried-it and deploying something pragmatic, decentralized and scalable in its place for managing names-and-trust. The current system of expressing trust through x.509 PKI is a joke in poor taste, and git (rightly) rejects most of that in favour of the three weaker more-functional models: the "DNS and soon-to-be-PKI DNSSEC+DANE" model of global-name disambiguation, the "manual ssh key-exchange with sticky-key-fingerprints" model of endpoint transport security, and the (imo strictly _worse_) "GPG web of trust" model for long-lived audit-trails. The three of these systems serve as modest backstops to one another but I still feel there's productive work to do exploring the socio-technical nexus of trust-and-naming as a more integrated, simplified, decentralized and less random, more holistic level (RFCs 2693 and 4255 aside). There are still too many orthogonal failure modes, discontinuities and security skeuomorphisms; the experience of naming things, and trusting the names you exchange, at a global scale, still retains far too much of the sensation of pulling teeth. We wind up on IRC with old friends pasting SHA-256 fingerprints of things back and forth and saying "this one? no? maybe this one?" far too often.

My theory is that PKI doesn't work because it is based on a flawed understanding of what identity actually means.

The fraction of the population that really understands what it means to assign cryptographic trust to a key is statistically indistinguishable from "no one". Maybe the reason that the web of trust we've been promised since the 90s hasn't appeared yet is because the model itself is broken.

Bazaar on the slow track -- Montone gets too little attention

graydon — Tue, 18 Sep 2012 15:25:48 +0000

I don't think it was just a matter of a missing nail in a horseshoe. If I _had_ to point to a single matter, it would center on identity-tracking (that is, not "just dumb content tracking"). We initially just did content-tracking alone -- which was orders of magnitude faster -- and found that we were stapling together too many ad-hoc merge "algorithms" to reconstruct events like file and directory renames, and java users were complaining about the inaccuracy of those, so we wound up building a large (and as it turns out, computationally expensive) secondary layer of logic concerned with file and directory object lifecycles. That's probably the source of the lion's share of the costs; but even if we hadn't done that, I'm sure the amount of in-memory transformations and verification, data re-parsing, crypto, and simple buffer-copying / sqlite-IO would probably have doomed us going up against kernel engineers. They think in a much closer to "zero copies and never calculate anything twice" mode. Very hard to compete with, given my background and coding style. I'm happy to yield defeat on implementation here; git _flies_. Very impressive implementation (though I do wish it'd integrate rolling-checksum fragment-consolidation in its packfiles, a la bup).

All that's a distraction though, at this stage. Git won; but there's more to do. I agree with you that the residual/next/larger issue is PKI and naming. Or rather, getting _rid_ of PKI-as-we-have-tried-it and deploying something pragmatic, decentralized and scalable in its place for managing names-and-trust. The current system of expressing trust through x.509 PKI is a joke in poor taste, and git (rightly) rejects most of that in favour of the three weaker more-functional models: the "DNS and soon-to-be-PKI DNSSEC+DANE" model of global-name disambiguation, the "manual ssh key-exchange with sticky-key-fingerprints" model of endpoint transport security, and the (imo strictly _worse_) "GPG web of trust" model for long-lived audit-trails. The three of these systems serve as modest backstops to one another but I still feel there's productive work to do exploring the socio-technical nexus of trust-and-naming as a more integrated, simplified, decentralized and less random, more holistic level (RFCs 2693 and 4255 aside). There are still too many orthogonal failure modes, discontinuities and security skeuomorphisms; the experience of naming things, and trusting the names you exchange, at a global scale, still retains far too much of the sensation of pulling teeth. We wind up on IRC with old friends pasting SHA-256 fingerprints of things back and forth and saying "this one? no? maybe this one?" far too often.

Bazaar on the slow track

pboddie — Tue, 18 Sep 2012 15:19:11 +0000

Maybe the sentiment was that Canonical could not wait for appropriately usable solutions to emerge or become dominant in the areas of version control (Bazaar, in its different forms) and project collaboration (Launchpad), and so they set out to make their own solutions as part of a larger endeavour.

Unfortunately, with Unity being the other prominent example, they don't have a great record of getting people on board or outperforming the other projects in those fields. And so the gamble of striking out on their own hasn't paid off as much as might have been expected.

Bazaar on the slow track

faassen — Tue, 18 Sep 2012 11:31:12 +0000

Nice bit of info here "Also, Launchpad and Bazaar have never really been meant as products in their own right, but more as facilities to build Ubuntu."

Canonical tried pretty hard to get projects in my little community (Zope) to join Launchpad, and also tried pretty hard to get the Zope developers to convert their repository from SVN to Bazaar. So is this "never really meant as products in their own right" a nice bit of historical revisionism, or was this the plan all along and nobody told us?

LP-bzr vs. LP-git?

smurf — Mon, 17 Sep 2012 23:25:15 +0000

Actually, I'd assume that replacing bzr with git in Launchpad would be reasonably straightforward.

Ah yes

jspaleta — Mon, 17 Sep 2012 20:18:25 +0000

There's a critical point here I think needs to be stressed.

Bzr is primarily used as a tool to interact with launchpad.net. It's never really garnered significant adoption as a standalone tool. Yes some people do use it outside of launchpad, I'm not saying there is zero interest in bzr sans launchpad integration, just that i don't see critical mass in bzr community if launchpad integration were lost.

Because of the intimately tied nature with a Canonical controlled service (launchpad) there is less overall incentive for an external to try to create and lead a competing fork for fear of creating an incompatibility with the launchpad.net service requirements. It's forkable, the license on the code assures that is a possible future, but I doubt anyone is going to ever seriously do it.

It's a deep tangle.

I've yet to hear of someone attempting to take the launchpad.net sources and spinning in an alternative site. If someone did that, forked the launchpad.net codebase and spun up a implementation outside of Canonical's control, then a fork of bzr would be an obvious part of that much more involved effort. In for a penny in for a pound.

The irony of course being that the deep integration with launchpad is probably the only reason keeping bzr alive as a project at all at this point. If Canonical threw away launchpad entirely and started from scratch today... I'd wager they'd pick up git as part of the scaffolding. bzr is just a long lived piece of technical debt at this point for the launchpad team inside Canonical.

-jef

Bazaar on the slow track

luto — Mon, 17 Sep 2012 18:31:37 +0000

I used to agree, except that I've shot myself in the foot numerous times using Mercurial in ways that would be impossible in git. Git never throws away commits until you ask it to. Mercurial throws them away in the process of rebasing, and it even requires you to strip things before you rebase their ancestors. This means that, if the rebase screws up [1] [2], then you lose data.

[1] Mercurial also seems to screw up more often. When it finishes embracing the three-way merge, it'll work better. The Mercurial tools certainly are prettier, though.

[2] Mercurial is worse at doing bizarre merges than git. At some point I'll dig up the example that broke Mercurial and file a bug. At least this failure mode doesn't eat my data. (Basically, the git "recursive" algorithm, while imperfect, is better than Mercurial's approach of choosing an arbitrary base from the set of bases with no possibility of overriding the choice.)

Bazaar on the slow track -- Montone gets too little attention

zooko — Mon, 17 Sep 2012 15:16:54 +0000

If I recall correctly on the day (weekend?) that Linus tried monotone, the then-current release of monotone had some diagnostic/debugging/profiling code compiled in which caused it to have superlinear runtime for some computation or other. Correct me if I'm wrong, Graydon, as I think what I'm recalling is from something you wrote shortly thereafter.

It's one of those "for want of a nail the horseshoe was lost" kinds of moments in history -- if monotone had been fast enough for Linus to use at that time then presumably he never would have invented git.

And while *most* of the good stuff that the world has learned from git is stuff that git learned from monotone, I do feel a bit of relief that we have git's current branch naming scheme. Git's approach is basically to not try to solve it, and make it Someone Else's Problem. That sucks, it leads to ad-hoc reliance on DNS/PKI, and it probably contributes to centralization e.g. github, but at least there is an obvious spot where something better could be plugged in to replace it. If we had monotone's deeper integration into DNS/PKI (http://www.monotone.ca/docs/Branches.html), it might be harder for people to understand what the problem is and how to change it.

Bazaar on the slow track -- history notes

douglasbagnall — Mon, 17 Sep 2012 11:52:13 +0000

> I am actually surprised that the Bazaar people haven't traded their core engine for git.

The other day I heard a Canonical employee advocating bzr as a git front-end. The argument was that nobody suffers if you use Bazaar locally and Git remotely, so bzr people should just do that and stop fussing. As you suggest, they may have been glossing over incompatibilities in the models, or perhaps they haven't hit them in practice.

Bazaar on the slow track

pboddie — Mon, 17 Sep 2012 09:11:07 +0000

Take git. Linus developed the thing for the kernel. git supported *large* source repositories quite well, right from the start. All the others were "OK it works for ten files and ten revisions, I'm done with the basics. 10000 files and 1000 revisions? Oops, need to take our lunch break now, hopefully it'll be done when I get back."

I'm pretty sure Mercurial was developed for working with the kernel sources.

Bazaar on the slow track

dlang — Sun, 16 Sep 2012 13:04:52 +0000

I ran across this post some time back, it seems to fit what I know on the subject quite well

http://xentac.net/2012/01/19/the-real-difference-between-...

Bazaar on the slow track

kleptog — Sun, 16 Sep 2012 12:44:43 +0000

I'm a git user "forced" to learn Mercurial. While I agree that Mercurial has many of the same features as git, I feel git works much better for my workflow. The index, "git add/remove -p" and "git rebase -i" are my very close friends.

The first thing I noticed is Mercurial's rigid adherence to "committed is unchangeable". For me a commit is more a checkpoint, but it's not necessarily something finished. Usually I develop something as a series of patches, commit various bug fixes and use rebase to fold the bugfixes into the appropriate patch.

I was relieved to find the MQ extension which gives you much the functionality but with a very obtuse UI. The phases you point to seem to be a further step in the right direction. Though I feel their painting themselves into a corner since now a review tool like Gerrit becomes impossible: you push to the review tool which would make your patch immutable, while the whole point of the review is to be able to fix the patch!

Other rough edges: that the "pager" extension is not standard, there is no justification for "hg log" on the terminal filling your scrollback buffer with the entire history of your project. The "color" extension could also be better advertised.

My feeling is that git is a tool for people who deal with large numbers of branches and patches daily, and Mercurial is for people push a few patches around occasionally.

Bazaar on the slow track

bronson — Sun, 16 Sep 2012 03:16:02 +0000

Oh, ok. Can't disagree with that!

Bazaar on the slow track -- Montone gets too little attention

cmccabe — Sat, 15 Sep 2012 20:54:37 +0000

I always get a chuckle when people try to compare the speed of most DVCSes to git. It's like hearing someone say his bicycle can out-perform your ferrari-- if there's a good tail-wind, and the road is downhill, and maybe if Lance Armstrong is the rider... Kudos to you for being realistic about the issue. And monotone was cool, back in the day.

Bazaar on the slow track

paulj — Fri, 14 Sep 2012 20:07:28 +0000

FWIW, I'm not taking a side. Just noting that both ways are possible. :)

Bazaar on the slow track

bronson — Fri, 14 Sep 2012 18:27:11 +0000

Oo, hey! A micro vs. monolithic argument! Let's all voice our meaningless opinions.

Bazaar on the slow track

paulj — Fri, 14 Sep 2012 17:51:37 +0000

Well, no, you can trivially have other process managers in between init and your service processes. You could have one that was dedicated to network services, hey it could even handle opening the sockets, watching them and only launching the actual services if there was activity (sound familiar?). You could have another that was specialised to watch TTY lines (Novell UnixWare had one of those I think). You could have another one for whatever.

Or you can have a kitchen-sink system, where you put all this into init, and it has to support every possible need any kind of service will ever have.

Bazaar on the slow track

apoelstra — Fri, 14 Sep 2012 16:23:23 +0000

> Though, there's no pressing reason why that job must be done by init…

Well, init is the job's parent, so it's uniquely positioned to notice when a job crashes -- and since init started the job, it's also uniquely qualified to /re/start it.