Sleepycat and CollabNet Open Source Collaboration [LWN.net]

From:		Noah Veltman <noah-AT-pageonepr.com>
To:		lwn-AT-lwn.net
Subject:		Open Source Collaboration
Date:		Tue, 7 Feb 2006 12:27:54 -0800

--News Release--

SLEEPYCAT HARDENS SUBVERSION VERSION CONTROL SYSTEM WITH HIGHLY  
RELIABLE DATABASE

Berkeley DB ensures Subversion data integrity and provides automatic  
recovery
  in the event of system failure

Emeryville and Brisbane, Calif.  February 7, 2006  Sleepycat  
Software, makers of the Berkeley DB family of open source developer  
databases, and CollabNet, the leading provider of on-demand  
distributed software development solutions and primary sponsor of the  
Subversion open source project, today announced that the two  
companies have collaborated, along with the Subversion open source  
community, to optimize the integration between Berkeley DB and  
Subversion, ensuring that data is stored reliably and with full,  
automatic recoverability after a system or application failure.

Subversion is the preeminent open source version control system for  
managing source code, documentation, and other files. The latest  
version of Berkeley DB v4.4 is being integrated into the upcoming  
Subversion 1.3.1 as a repository for files and their previous  
versions. Berkeley DB provides Subversion with a number of features,  
among them guaranteed data integrity, automatic recovery, and hot or  
cold backups.

Subversion has been using Sleepycat Berkeley DB since its inception,  
both for Berkeley DBs reliability and its features, said Karl  
Fogel, a founding developer of Subversion. Sleepycat has been an  
important contributor to Subversion by first enhancing Berkeley DB  
and then working with CollabNet and the broader Subversion open  
source community to update Subversion to take advantage of those  
added features. We have consistently received outstanding support  
from the Sleepycat engineers, and feel this close collaboration has  
helped make Subversion one of the most popular configuration  
management solutions today.

Sleepycat is as committed to the open source community as to our  
commercial customers, both of whom use Berkeley DB in mission- 
critical applications, said Keith Bostic, co-founder and vice  
president of product development at Sleepycat Software. Subversion  
is a critical tool for developers, and as developers ourselves, we  
can appreciate the importance of a bullet-proof repository for source  
code.

Berkeley DB is already used by hundreds of major open source  
projects, including all components of the LAMP stack, which consists  
of the Linux operating system, Apache web server, MySQL database, and  
the scripting languages, Perl, Python, and PHP. Other open source  
projects using Berkeley DB include BSD UNIX, OpenLDAP, Kerberos,  
JBossCache, Sendmail, OpenOffice, and many others.

Additional information on the use of Berkeley DB in open source  
projects is available at: http://www.sleepycat.com/customers/ 
opensource.html. Information on the use of Berkeley DB in commercial  
projects is available at: http://www.sleepycat.com/customers/ 
customerlist.html

Availability
The optimized Subversion and Berkeley DB integration is available in  
the Subversion development tree and is undergoing testing now. It is  
expected to be generally available as part of the upcoming Subversion  
1.3.1 release. More information is available at: http:// 
subversion.tigris.org/

About CollabNet
CollabNet develops and delivers innovative on-demand distributed  
software development solutions and is the primary sponsor of the  
award-winning Subversion open source project. Founded upon open  
source principles, CollabNet enables organizations to take advantage  
of global resources by delivering a web-based development platform as  
a managed service.

CollabNet Enterprise Edition and CollabNet Subversion help companies  
eliminate the inefficiencies and limitations inherent in traditional  
development platforms by enabling on-demand development for  
geographically dispersed software development teams. The company is  
transforming traditional software development processes by allowing  
businesses to more efficiently and cost-effectively drive software  
projects through the application lifecycle, and collaborate securely  
with key business partners, contractors and offshore development  
firms across multiple locations. Using CollabNet solutions and  
services, organizations can realize the true value of their software  
assets to create reusable applications and re-architect for SOA  
deployment, ensure regulatory compliance, and create dynamic global  
development environments that deliver software projects on-time, on- 
target, and on-budget.

More than 700,000 developers and IT project managers at global 2000  
and industry-leading enterprise companies use CollabNet solutions.  
For more information: http://www.collab.net/.

About Sleepycat Software
Sleepycat Software (www.sleepycat.com) makes Berkeley DB, the most  
widely used open source developer database in the world with over 200  
million deployments. Customers such as Amazon.com, AOL, Cisco  
Systems, EMC, Google, Hitachi, HP, Motorola, RSA Security, Sun  
Microsystems, TIBCO and VERITAS also rely on Berkeley DB for fast,  
scalable, reliable and cost-effective data management for their  
mission-critical applications. Profitable since it was founded in  
1996, Sleepycat is a privately held company with offices in  
California, Massachusetts and the United Kingdom.

Sleepycat Software and Berkeley DB are trademarks of Sleepycat  
Software, Inc. CollabNet is a trademark or registered trademark of  
CollabNet, Inc. in the United States and other countries. All other  
trademarks or registered trademarks are the property of their  
respective holders.

For more information, please contact:
		
Tim Cloonan		  		 	
Director of Marketing 	 		    	
CollabNet		  		  	
650-228-2584		  			    	
tim@collab.net	      	

Matt Otepka	
104° West Partners
PR for CollabNet
206-568-5142
matt.otepka@104degreeswest.com

Craig Oda
Sleepycat Software
650-565-9800 x102
coda@sleepycat.com

(Log in to post comments)

Sleepycat and CollabNet Open Source Collaboration

Posted Feb 8, 2006 1:38 UTC (Wed) by elanthis (guest, #6227) [Link]

Do they break API? I've been under the impression that the API is pretty darn stable. Even the ABI is stable, as I understand.

It's the database format that keeps breaking.

Sleepycat and CollabNet Open Source Collaboration

Posted Feb 8, 2006 13:54 UTC (Wed) by cartman (subscriber, #11404) [Link]

Yeah you got me ;)

Sleepycat and CollabNet Open Source Collaboration

Posted Feb 11, 2006 4:09 UTC (Sat) by zblaxell (subscriber, #26385) [Link]

It's more like every minor release has an *entirely new* ABI, complete with C preprocessor tricks to add the current version number to every symbol in the library. On a good day this means you can link multiple versions dynamically into your executable. On a bad day this means you absolutely cannot link against some random shared library version, only the specific version your application was compiled with.

Sleepycat and CollabNet Open Source Collaboration

Posted Feb 8, 2006 6:27 UTC (Wed) by ctg (subscriber, #3459) [Link]

Yeah, I can only think it means "automatic" in the sense of "whenever
anything goes wrong, the system administrator will automatically go
through a full recovery process because he/she knows there isn't any
point in trying to troubleshoot the system".

db4 is the most unreliable DB system I've ever come across - not that it
has failures, but that the smallest problem seems to cause the most major
impact. Impossible to work what is going wrong - everything hangs -
sometimes shutting the system down and doing a db_recover doesn't always
work - and a full dump and restore is required. I've switched to sqlite3
- which is a bit slower but seems to work - working/slow is better than
fast/broken.

The annoying part of this announcement is that we are in the process of
switching from CVS to SVN. I'll have to find out what this means w.r.t.
the impact on reliability.

Sleepycat and CollabNet Open Source Collaboration

Posted Feb 8, 2006 6:42 UTC (Wed) by Dom2 (guest, #458) [Link]

To be honest, I'd still choose the FSFS over the DB4 backend. The main advantage is that because
it's bundled with subversion, something else upgrading your DB4 libraries won't suddenly stop your
version control system from working. Unfortunately, this did happen to me. I switched to FSFS
when subversion 1.1 came out and I haven't looked back...

But I still love subversion. :-)

-Dom

Sleepycat and CollabNet Open Source Collaboration

Posted Feb 9, 2006 5:08 UTC (Thu) by Peter (guest, #1127) [Link]

Yeah, I can only think it means "automatic" in the sense of "whenever anything goes wrong, the system administrator will automatically go through a full recovery process

While I appreciate the cynicism, they really do mean automatically. This code was tested by doing normal subversion stuff but also having a thread running whose only purpose was to start a transaction then get kill -9'd. Once per second, or was it 10 times per second? Anyway, the ability for an application to avoid wedging the DB even when dying unexpectedly is new to DB 4.4, so your previous experiences aren't completely valid.

The annoying part of this announcement is that we are in the process of switching from CVS to SVN. I'll have to find out what this means w.r.t. the impact on reliability.

DB isn't the default backend. Most people recommend the FSFS backend instead, which is why it's the default. It's a little slower for some operations, and creates a huge directory (one file per revision, just like arch, so if you have 25000 revisions you get a dir with 25000 files, which doesn't perform well with certain filesystems), but it has the very nice property of not changing your old data - think of it as an append-only system. And it works over NFS, not that we'd necessarily recommend it.

Sleepycat and CollabNet Open Source Collaboration

Posted Feb 9, 2006 8:51 UTC (Thu) by georgm (guest, #19574) [Link]

Ok, but what about file stability over releases?
I don't want to always get into trouble when switching to a newer db-Version. I had these problems for 2 or 3 times, then I switched to fsfs to avoid this problem.

Sleepycat and CollabNet Open Source Collaboration

Posted Feb 9, 2006 4:58 UTC (Thu) by Peter (guest, #1127) [Link]

Wait, how the HECK do you get automatic recovery from BDB?

That's what this news release is about! Using new features of BDB 4.4, it's now possible for the app to do this. Previously it was not. The Subversion BDB backend was recently hacked on for awhile (the bdb-fixes branch) mainly to add this enhancement.

By the way, I believe there's an error in the press release. They say the new stuff will be in Subversion 1.3.1, but last I heard, that's not the plan. bdb-fixes has been merged back to the development trunk, but the developers are leaning toward not merging it into the 1.3.x branch until after 1.3.1 - so probably in 1.3.2 or so. It's still new and needs a bit of time to "season" first.