User: Password:
|
|
Subscribe / Log in / New account

Katz: The Future of CouchDB

CouchDB creator Damien Katz appears to be forking the project with the creation of "Couchbase." "It's not that we think CouchDB isn't awesome. It's that we are creating the successor to it: Couchbase Server. A product and project with similar capabilities and goals, but more faster, more scalable, more customer and developer focused. And definitely not part of Apache."
(Log in to post comments)

Katz: The Future of CouchDB

Posted Jan 5, 2012 18:08 UTC (Thu) by erinnlooneytriggs (guest, #24665) [Link]

"more faster", ugh. I always want my car to go more faster, so I more increase pressure on the gas pedal.

Katz: The Future of CouchDB

Posted Jan 5, 2012 18:50 UTC (Thu) by cmccabe (guest, #60281) [Link]

Another pet peeve: referring to "C/C++" as though it were a single language.

Besides the rewrite in C/C++ so it can go more faster, are there any other design changes planned here? All joking aside, I'd like to hear more about the internals from someone knowledgeable...

Katz: The Future of CouchDB

Posted Jan 5, 2012 18:41 UTC (Thu) by Cyberax (✭ supporter ✭, #52523) [Link]

I've actually started a small project with exactly the same purpose: https://github.com/Cyberax/SofaDb (license is GPL for now). Interesting.

Katz: The Future of CouchDB

Posted Jan 5, 2012 20:05 UTC (Thu) by ceplm (subscriber, #41334) [Link]

So, everything is great, Apache is awesome, Erlang is perfect, and therefore the original co-author of CouchDB leaves the project and forks it! Does it make any sense to anybody?

Katz: The Future of CouchDB

Posted Jan 5, 2012 20:36 UTC (Thu) by Cyberax (✭ supporter ✭, #52523) [Link]

Erlang/CouchDB has several problems.

1) It's slow. My SofaDB database can manage 200000 non-batched simple inserts per second using a local API. CouchDB can do about 20000 per second using its own local API (https://github.com/jchris/hovercraft).

2) It's not very suitable for embedding.

3) CouchDB source code is somewhat obfuscated. In places _very_ obfuscated (unintentionally).

Katz: The Future of CouchDB

Posted Jan 5, 2012 20:59 UTC (Thu) by ceplm (subscriber, #41334) [Link]

a) I was more baffled by the style of the original post, which was completely opaque about the reasons of fork,
b) concerning 3), I really shouldn't mention http://www.joelonsoftware.com/articles/fog0000000069.html, right? I think especially about “It's important to remember that when you start from scratch there is absolutely no reason to believe that you are going to do a better job than you did the first time.”

Katz: The Future of CouchDB

Posted Jan 5, 2012 21:40 UTC (Thu) by Cyberax (✭ supporter ✭, #52523) [Link]

Well, I'm not a CouchDB developer.

In fact, my code is completely independent and I use CouchDB sources only to clarify fine details of replication (though I'm planning to steal the whole Futon management interface wholesale. It's just too nice).

In general, second system syndrome is very real. But it still sometimes makes sense to rewrite something from scratch. Especially since CouchDB is not that large to make rewrite impossible.

Katz: The Future of CouchDB

Posted Jan 6, 2012 6:53 UTC (Fri) by bronson (subscriber, #4806) [Link]

> when you start from scratch there is absolutely no reason to believe that you are going to do a better job than you did the first time.

Sure there is. If you previously chose poor infrastructure, you'll presumably make better choices this time around.

Just beware the second system effect! Don't try to fix *everything*...

Katz: The Future of CouchDB

Posted Jan 6, 2012 9:50 UTC (Fri) by ceplm (subscriber, #41334) [Link]

No, there isn't. What you are looking for is called refactoring, not rewrite.

No, I am not a friend of obscure langauges (living for a long time in XMPP and made me tired of the only choice between broken server in Java, or other ones in Erlang or Lua), and yes I understand that sneaking out of Erlang to some more mainstream language is a great undertaking, but looking around for history of big rewrites, I believe I would still stick with refactoring.

Katz: The Future of CouchDB

Posted Jan 6, 2012 18:23 UTC (Fri) by bronson (subscriber, #4806) [Link]

I'm talking about changing infrastructure, which usually requires a significant portion of your app must be rewritten. It's true that the definition of refactor has become pretty loose lately but I don't think it's THAT loose yet.

Katz: The Future of CouchDB

Posted Jan 6, 2012 11:39 UTC (Fri) by niner (subscriber, #26151) [Link]

Joel certainly got a good point there, but it's equally certain not the only truth. It's good advice but not always the perfect decision. Heeding Joel's advice, I use a rewrite only as last resort to solve problems. But even so, they are sometimes neccessary. And when I did them, they have always involved some change of base technology (like different implementation languages, frameworks, data storage) and have always been very successful solving longstanding problems that prevented further company success.

Joel's article itself gives a good example: the Mozilla rewrite. Back when Netscape 6 finally came out late and buggy, pretty much everyone thought it was a failure. Nowadays with Firefox having more than 50 % market share in countries like Germany, Microsoft really trying to adhere to standards and the web finally moving forward again, I think it's fair to call it the best thing those engineers could have done.

Katz: The Future of CouchDB

Posted Jan 6, 2012 14:12 UTC (Fri) by tialaramex (subscriber, #21167) [Link]

We did a big re-write of our big in-house application a few years back which on the whole I think was a success. Perhaps because:

• The application was previously developed by an external contractor as a cost saving measure, now it was to come in-house. So most people (except the one hired away from the external contractor for continuity) would have no existing in depth knowledge of the prior system and thus the re-write also served to educate them about the inevitable quirks that don't make it into any specification.

• The application had evolved with the business from its earliest days, being repeatedly refactored to reflect the sort of acrobatic changes possible in a young startup. The rewrite did not face such a turbulent environment, and it could eliminate the scars from that refactoring without facing the possibility that in a year's time it too would have its entire purpose changed in a single meeting.

• We had new UI designs as a results of having people sit down and talk to users about what they wanted/expected it to do, and what frustrations they experienced. So the UI would have been largely a re-write even if we overall decided just to re-factor.

• Our architectural vision for the new system involved many small modules with very simple interfaces, which meant developers working on modules could be almost autonomous thereby reducing the collaborative cost that leads to Brooks's law. The old system was not designed this way, so initial re-factoring would have involve co-ordination of all team members at every step.

There were of course obstacles, and some years on all those "small modules" have grown in size, and their "simple interfaces" have sprouted curious lumps and bumps, but the general approach seems to have worked. It delivered more or less on time, the user feedback was positive, the costs involved were manageable.

Katz: The Future of CouchDB

Posted Jan 6, 2012 17:53 UTC (Fri) by erwbgy (subscriber, #4104) [Link]

Insightful comments like this based on real experience are why I love reading LWN. Thanks.

Katz: The Future of CouchDB

Posted Jan 7, 2012 13:39 UTC (Sat) by ceplm (subscriber, #41334) [Link]

Sure, Mozilla got saved by skunkworks and a miracle of Firefox (http://web.archive.org/web/20110623034401/http://weblogs....), but it doesn’t mean that refactoring of Netscape 3 codebase (suggested by jwz among others) wouldn’t give them relevant browser sooner and with bigger impact than Firefox gave them.

Katz: The Future of CouchDB

Posted Jan 8, 2012 8:25 UTC (Sun) by ssmith32 (subscriber, #72404) [Link]

I agree with (1) it is baffling he doesn't explain the reason.

I also agree with (3) - you shouldn't mention it.

It would be better if you reference the original source of the idea:

http://en.wikipedia.org/wiki/The_Mythical_Man-Month

One that captures the subtle distinctions necessary to apply the principle correctly, and doesn't reduce to web blog punditry.

It's not that I mind that Joel has ripped off Brook's ideas as his own (in this and other blogs) without giving credit - given that I doubt Brooks would care either way if he was name-dropped in the post. What annoys me is that Joel rips off the idea badly.

Katz: The Future of CouchDB

Posted Jan 8, 2012 20:02 UTC (Sun) by ceplm (subscriber, #41334) [Link]

Right, one more book I need to read. Oh well :(

Katz: The Future of CouchDB

Posted Jan 12, 2012 17:43 UTC (Thu) by dgm (subscriber, #49227) [Link]

Don't worry, it's very light read and very entertaining.

Just my personal opinion, but if you have to dump some other book to make place for TMMM, do so on the spot without thinking twice. It will change the way you think about software projects, developers, and their interactions.

Katz: The Future of CouchDB

Posted Jan 13, 2012 0:27 UTC (Fri) by jschrod (subscriber, #1646) [Link]

s/need to/must/

Having read TMMM should be a principle precondition for anyone who is allowed near management of software projects, IMNSHO. As well as discussions of newer methods (agile, UX focused, etc.), and their restrictions in real-life projects.

Katz: The Future of CouchDB

Posted Jan 9, 2012 12:47 UTC (Mon) by tialaramex (subscriber, #21167) [Link]

Actually I think when Joel says "that old mantra" he's not trying to avoid citing Brooks, but instead assuming his audience are already familiar with that work.

I don't think The Second-System Effect, which is the chapter of Brooks most relevant to this discussion, comes close to listing all the problems Joel hits on, and its examples are inevitably dated. The dusty copy on my shelf proudly states that it is the "20th anniversary edition".

For example, in the chapter's section describing why OS/360 is "a pile" Brooks offers the example of 26 bytes he sees as being "wasted" on handling the correct behaviour of the new year in leap years, insisting that this situation should instead be handled by the operator. So, every four years Brooks proposes that the inevitable skeleton staff left on duty while others party should undertake a tricky manual change to the system precisely at midnight, in order to reduce the size of the system by 26 bytes. A modern student encountering this paragraph would be right to snort at the lack of foresight in such an assertion and perhaps to raise an eyebrow at the apparent implication that Brooks was in favour of the sort of shortcuts that led to the "Millennium Bug".

Katz: The Future of CouchDB

Posted Jan 6, 2012 10:51 UTC (Fri) by imgx64 (guest, #78590) [Link]

I'm usually skeptical when developers decide to do a complete rewrite of their codebase, but I think they did the right thing by clearly making it a fork instead of simply calling it "CouchDB 2" and subsequently breaking things for all their users.

GNOME developers (and many others) could learn a lesson or two here.

Katz: The Future of CouchDB

Posted Jan 8, 2012 22:06 UTC (Sun) by rahulsundaram (subscriber, #21946) [Link]

GNOME didn't do a total rewrite of anything. Its mostly the same GNOME with just a new shell interface.

Katz: The Future of CouchDB

Posted Feb 1, 2012 22:27 UTC (Wed) by mfedyk (guest, #55303) [Link]

I'd just like to point out that the Apache couchdb is still there and going strong, this is simply one contributor that has moved on to another project.


Copyright © 2012, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds