|
|
Subscribe / Log in / New account

Getting the (Share)Point About Document Formats

November 13, 2007

This article was contributed by Glyn Moody

The OpenDocument Foundation was formed in 2005, with the mission "to provide a conduit for funding and support for individual contributors to participate in ODF development" at the standards body OASIS. So, at a time when backing for the ODF format seems to be gaining in strength around the world, eyebrows were naturally raised when Sam Hiser, the Foundation's Vice President and Director of Business Affairs, wrote on October 16 that it was no longer supporting ODF:

We at the OpenDocument Foundation have been displeased with the direction of ODF development this year. We find that ODF is not the open format with the open process we thought it was or originally intended it to be.

Microsoft's Jason Matusow naturally allowed himself a little Schadenfreude, Mary Jo Foley waxed apocalyptic, speculating that "the ODF camp might unravel before Microsoft's rival Office Open XML (OOXML) comes up for final international vote early next year," and IBM's Rob Weir provided a characteristically witty point-by-point criticism of the group's reasoning behind its move, dubbing the OpenDocument Foundation "two guys without a garage", in a nod to the "mythology of Silicon Valley" and its history of "two guys in a garage founding great enterprises."

Meanwhile, in an attempt to understand what was going on, standards expert Andy Updegrove tried applying Occam's Razor:

The simplest explanation would appear to be simply that when the Foundation's founders decided to turn out the lights, they decided to poke a sharp stick in the eye of those that had rejected their approach.

That seems a little hard to believe, though, given the years of hard work put in by Foundation members in support of ODF. For example, Hiser notes that he worked on the OpenOffice.org project "from 2001 through its 20-millionth download. I was OpenOffice.org's Marketing Project Lead...back when people said 'Open What?'" Moreover, if the Foundation had really wanted to wield a sharp stick, it could have done so far more effectively by announcing its break earlier. As Hiser points out: "I was supportive of ODF into the summer in order to avoid negative attention for ODF leading up to the September ISO vote on OOXML."

The roots of the Foundation's decision to abandon the ODF format in favour of the little-known Compound Document Formats (CDF) from the W3C go back to one of the most fraught and painful episodes in the history of open source and open standards: the attempt by the Commonwealth of Massachusetts to adopt the ODF format. Hiser was in the thick of it:

I was the outside consultant working with Mass ITD [Information Technology Division] between Nov 2005 & June 2006 on their Pilot of ODF-ready software. (We are bound by NDA so I can't go into details.) What you do know already is that the Pilot ended with [Massachusetts CIO] Louis Gutierrez putting out a Request For Information ("RFi") for an ODF Plugin for MS Office. That tells you what? That ODF-ready software like OpenOffice.org worked splendidly in ITD? The RFi was a cry for help to the free software community to get what Sun and others decided not to provide: interoperabilidad!

Sun's Chief Open Source Officer, Simon Phipps, begs to differ:

Sun sees interoperability for OpenOffice.org as hugely important and has contributed vast amounts of code to the community to make it possible. Most of the interoperability support to date has been implemented by Sun engineers, and Sun continues to invest heavily in this essential capability, seeking to get it as close to 100% as is technically possible.

The main bone of contention between the OpenDocument Foundation and the rest of the community supporting ODF comes down to the issue of whether "100%", full-fidelity interoperability with Microsoft Office is achievable or even desirable. For example, Weir wrote:

I would not claim a priori that all customers require lossless, 100% fidelity conversions. Remember, we do not see 100% fidelity even when upgrading from Office 2003 to Office 2007, but this appears to be adequate. What is required is that the total return from changing document formats exceeds any other profitable use of capital available to the enterprise.

The Foundation's position was explained by its President, Gary Edwards:

We had to have perfect fidelity because there was no reasonable expectation of ever successfully migrating those business processes to a Microsoft Office alternative like ODF-ready OpenOffice.org, StarOffice, WorkPlace or Novell Office. Such a re-engineering of the business processes would be costly and beyond disruptive.

If however we could achieve full fidelity conversions of legacy Microsoft binary documents to ODF, and were able to guarantee the roundtrip process of these newly christened ODF documents in a mixed desktop environment, one comprised of ODF-enabled Microsoft Office, OpenOffice.org, Novell Office, WorkPlace, and KOffice, the existing MSOffice bound business processes could continue being used even as new ODF ready workstations were added to the workflow. Massachusetts could migrate to non-Microsoft Office software in manageable phases, restoring competition -- and sanity -- to the Commonwealth's software procurement program.

If on the other hand, there is no full fidelity conversion to ODF of legacy documents available at the head point of the migration -- Microsoft Office -- then the business process will break under the weight of users having to stop everything to fix and repair the artifacts of lossy file conversions. What Massachusetts discovered is that users will immediately revert to a Microsoft-only process wherever the business process system breaks down due to conversion fidelity problems. It is a productivity killer and a show stopper for migration to ODF-supporting software.

This line of thinking probably explains the widespread incomprehension that greeted the Foundation's decision to abandon ODF. Supporters of the latter believe that it is by far the best document format, one that provides numerous benefits to users, notably freedom from lock-in. Hiser couldn't agree more: "We don't want OOXML to ever see the light of day, and certainly we feel deeply that it needs to be rejected by ISO finally and conclusively." But he adds:

Whatever happens at ISO, though, the market is where acceptance of OOXML is inevitable. The clock is ticking as the major governments that are trying to adopt ODF are finding it quite taxing on a practical level (Mass, Denmark, Belgium). Each one is drifting from ODF-only policies to ODF + OOXML. This is because OpenOffice.org installation is not enough to overcome the sticky business processes in workgroups across the extended enterprise.

In companies running the Microsoft enterprise stack, those "sticky business processes" are defined and stored in a program that has a surprisingly low profile, but which may well turn out to be the biggest emerging threat to open source: Sharepoint. One perceptive observer, Alfresco's Matt Asay, had already spotted that threat 18 months ago, and is just as worried today:

The more content you put in - whatever the individual file formats - the harder it is to get it out, because you're locking it into both a closed repository *and* a closed Microsoft ecosystem. Even if you manage to get your content out of Sharepoint it's still set to work on SQL Server with IIS/ActiveDirectory, etc. Closed. Closed. Closed.

Even worse, this same lock-in applies to any ODF documents the user might have. As Asay explains:

Let's assume you store data in ODF in a Sharepoint repository. It doesn't matter that ODF is an open format. The repository holding it is proprietary, and that proprietary lock-in is doubled by the fact that the enterprise will build (proprietary, non-standard) workflows to manage that content which keeps content a prisoner to Microsoft.

In other words, the lock-in occurs not at the document level - the one the ODF community is most focused on - but at the level of the workflow. If companies can't export their documents with ease and with perfect fidelity, the Foundation believes, they will simply opt for the default Microsoft solution - Sharepoint - and become trapped by workflow lock-in. This is why the Hiser and his colleagues have shifted their support away from ODF to CDF, which they think could allow companies to export Microsoft Office documents with perfect fidelity into other, truly open workflow systems. Asay's explains the difference with reference to his own company's product:

Alfresco is open source, open standards, and open ecosystem. If you choose to leave Alfresco, your content easily goes with you and you can take it to whatever repository you want. We run on any database, any application server, any directory service, any security protocol, etc. Our customers get to choose best of breed components, rather than being forced into a closed ecosystem.

The Foundation believes that the ODF format could have addressed this issue had it added certain extensions to the standard to provide perfect interoperability with Microsoft's Office documents. Some, like Weir, doubt this:

If I thought, as the Foundation claims, that 5 simple changes to ODF would make it perfectly compatible with MS Office, then I would be 100% behind their proposal. I'd have no hesitation. However, I don't think their claims hold water.

Given the reluctance of the ODF community to take that route, the Foundation hopes to blunt the Sharepoint threat by combining the CDF format with some code it had already written for use with ODF files, called the "da Vinci" plugin. Hiser explains where the name came from:

The thing about the plugin is we've cracked the secret of Microsoft's format - there's something we call Secret RTF. You know how MS has had dozens and dozens of formats - how do you think you would keep your head straight? You have one format you check in on, and it checks in on Secret RTF. We've cracked the code, which is why we've called this da Vinci."

It is the da Vinci plugin, the Foundation claims, that will allow perfect interoperability with Microsoft's Office files, which in turn will allow such documents to be taken out of the Microsoft ecosystem into open workflow software without users seeing any loss of fidelity. Partly because of the grand claims made for it, the plugin has been the cause of much of the skepticism surrounding the Foundation's ideas and future plans. As Weir wrote:

Why isn't [da Vinci] open source? Are we to follow the Foundation's claim of 100% interoperability, based on blind faith, without seeing some proof in the form of working code? I've been working on document conversions and document file formats of one kind or another for almost 20 years. I've never seen 100% fidelity conversions of anything but trivial formats. Extraordinary claims require extraordinary evidence. But we have nothing here, just white papers.

Hiser explains that in fact they had intended to release it as open source: "we actually agreed to open source it, and Massachusetts said, 'oh, great, OK'. And then the vendors that Louis [Gutierrez] asked to fund it walked away. When Louis resigned, we stopped doing work."

With the start of the CDF project, Hiser and Edwards have new priorities: "The reason for not opening sourcing it [now] is not because we're going to make a million dollars, although that's one of our goals too. We are business people: we need to fund the business that sells products, and we have to do that in about that magnitude to sustain our developing capabilities, and to feed our families." They will also be coming up with a new name for the now-defunct OpenDocument Foundation: "It takes time to do a total brand and corporate make-over," Hiser says, "but that's underway. I'm leaning toward 'Two Guys Without a Garage LLC'."

Whether or not CDF is the right format, whether the da Vinci plugin can provide 100% interoperability, and whether the new company of Hiser and Edwards will flourish, remains to be seen. In any case, the dramatic decision to break with the ODF community, and the attendant publicity it has garnered, may have already achieved something beneficial, for it has helped to direct attention towards the hitherto largely under-appreciated threat that Microsoft's Sharepoint represents for open source, open documents and open standards in the enterprise.

Asay has some thoughts on what must be done to meet that threat:

We need open-source companies and projects tightly integrating with each other. Alfresco + Jboss Portal + Red Hat Linux + JasperSoft would be a pretty compelling alternative to Sharepoint, as would other combinations. Alfresco plans to continue building out (and exceeding) Sharepoint functionality, and we'll start to message against Sharepoint in 2008. But it really requires that a community engage against Sharepoint, and not just one company.

Weir agrees, and even sees an downside to Microsoft's huge installed base of Office users:

From the FOSS perspective I think our greatest strength is that we do not have the legacy base that Microsoft has. Sure, this is a revenue stream for Microsoft, but it is also a huge burden. Microsoft cannot move as fast or innovate as fast as they would like to, since they have so many legacy documents and legacy users to worry about. I think FOSS can and should try to out-innovate Microsoft.

Clearly, then, now is a good time for two guys with - or even without - a garage to get coding, and found that great open source enterprise.

Glyn Moody writes about open source at opendotdotdot.

Index entries for this article
GuestArticlesMoody, Glyn


to post comments

Getting the (Share)Point About Document Formats

Posted Nov 13, 2007 17:54 UTC (Tue) by flewellyn (subscriber, #5047) [Link] (1 responses)

One does not get perfect fidelity when converting between different versions of Microsoft Office, so how on Earth can we expect it when converting between Office and ODF? More to the point, if the "hiccups" involved in converting between Office versions (repeatedly!) have been acceptable up to now, why would it be unacceptable to have to do so once with ODF?

I sense excuse-making for their new product.

Getting the (Share)Point About Document Formats

Posted Nov 14, 2007 2:25 UTC (Wed) by jzbiciak (guest, #5246) [Link]

Heck, you don't even get perfect fidelity within MS Office if you change your target printer.
I've had many papers blow up on me with out changing print margins or paper size just by
changing the target printer.

Getting the (Share)Point About Document Formats

Posted Nov 13, 2007 18:55 UTC (Tue) by clugstj (subscriber, #4020) [Link] (4 responses)

Functionality of Sharepoint?  Does it have any?  We are forced to use Sharepoint here at work
and it SUCKS!  Maybe it is just the cluelessness of our IT, but other than saving old
revisions of files, I see no advantage to it over a shared directory structure on a file
server.

Getting the (Share)Point About Document Formats

Posted Nov 14, 2007 0:02 UTC (Wed) by drag (guest, #31333) [Link]

Well Microsoft never got anywere by making their software better then everybody else's.  From
early versions of DOS, to Apple, to OS/2, to Novell, everything they made was done better by
other people.

They got 95+% of the PC market, business and personal, by making their products more
attractive, cheaper, and more profitable then other people's. Good advertising, correct
feature targetting, effective FUD, agressive salesmenship, manipulating other businesses,
giving big kick-backs to ISVs for convincing their customers to move to Microsoft, etc etc
etc. They are/were better businessmen.


In this case most businesses already massively use and depend on Windows, MS Office, Exchange,
Microsoft Server, Active Directory, and other Windows-only proprietary software. Sharepoint
integrates with all of that cheaply. It's a no-brainer to try it out and gradually use it more
and more as it matures. 

Linux and open source software, on the other hand, while being much more technically superior
(especially in the web arena) require much higher amounts of investment in employees (which
are treated as a liability, not a asset, in the financial books) and changes to their
infrastructure to use.

Getting the (Share)Point About Document Formats

Posted Nov 14, 2007 5:20 UTC (Wed) by dmarti (subscriber, #11625) [Link]

The problem is when a vertical application depends on Sharepoint instead of some other kind of
back end storage.  Makes it harder for whoever built the application to do a version for
Linux.

Don't be complacent about SharePoint

Posted Nov 14, 2007 7:27 UTC (Wed) by Cato (guest, #7643) [Link]

I use MediaWiki and Sharepoint at work currently, and have done a lot with TWiki in the past,
including developing features and fixes.  I don't really like Sharepoint and find it clunky to
use compared to TWiki or MediaWiki - but it's very dangerous to write it off.  Sharepoint has
a long list of features, exceeding TWiki's in some cases (e.g. granular access control and
in-place Office document editing).  Microsoft is investing a lot in Sharepoint including
adding wiki features, so it's quite likely it will be more competitive with TWiki soon.  

In many companies, Sharepoint is being mandated by IT (even one major bank where TWiki was
pioneered), so the competing Wikis have to be a LOT better simply to survive against this
mandate.  Currently they are better in some areas, and weaker in others.

See https://twiki.cern.ch/twiki/bin/view/DESgroup/TWikiVShare... for a good comparison by
CERN, http://twiki.org/cgi-bin/view/Codev/SharePointVsTWiki for the TWiki.org discussion, and
http://slashdot.org/comments.pl?sid=186678&cid=15404016 for another brief comparison.  There
are many other Wikis of course but TWiki is the one that's closest to matching Sharepoint -
MediaWiki is also nice, and very fast, but isn't really oriented to the enterprise in the way
it does access control, attachments, etc.  

If you want to help FOSS compete with Sharepoint, which is important to FOSS competing on the
desktop with Microsoft as has been discussed, I would suggest you install TWiki.  Try using
it, and maybe contribute some documentation updates, and if possible bug fixes and plugins.
TWiki has 200+ plugins but could really do with some that match Sharepoint features.

Getting the (Share)Point About Document Formats

Posted Nov 18, 2007 18:41 UTC (Sun) by man_ls (guest, #15091) [Link]

[...] other than saving old revisions of files, I see no advantage to it over a shared directory structure on a file server.
For saving old revisions of files, my first option would be a shared directory over HTTP on Subversion. Works great and it is completely transparent to your users. Every time you save your document you get your new revision. And it is free.

Still, people keep making their IT choices by looking at the list of features provided by the vendor. Classic mistake from the nineties, people!

the simple answer

Posted Nov 14, 2007 22:12 UTC (Wed) by qu1j0t3 (guest, #25786) [Link]

Don't use SharePoint.

From day zero it was a lock-in play. For God's sake, it even ignores the concept of URLs.

If you're not smart enough to avoid it (and every single other MS product on the face of the
planet), then you deserve what you get.

SharePoint and lock-in

Posted Nov 15, 2007 10:20 UTC (Thu) by dion (guest, #2764) [Link] (1 responses)

It seems that everything MS makes these days is first and foremost about lock-in.

It makes me sick that IT departments lack the ability to see the problem with vendor lock-in.

Although it would be glorious for customers to dump MS and choose OS solutions exclusively,
perhaps the only way for us to get rid of MS is for MS to selfdestruct.

There are signs of MS failing, like the bungled Vista release, but I think it's much worse for
MS that they are seen as a dangerous competitor by some their own customers, ie. everybody who
develops software.

If MS keeps competing with everybody who bases their software on Windows then the smarter
developers will try to avoid being 100% tied to Windows, some would say this has already
happened.

The same thing happened a while back ('92) when Pepsi bought Taco Bell, KFC and Pizza Hut,
that meant that many restaurants started to see buying Pepsi as a way to support their own
competition:
http://query.nytimes.com/gst/fullpage.html?res=9E0CE2D816...

Another example of this mechanism happened in software where SAP brought out SAP DB to avoid
being dependent on databases from Oracle, IBM and MS who all compete directly against SAP,
whether that one worked or not is debatable.

This turned out to be quite rambly, sorry, perhaps I should write a blog in  stead.

SharePoint and lock-in

Posted Nov 16, 2007 3:13 UTC (Fri) by N0NB (guest, #3407) [Link]

I don't think that IT departments fail to see the vendor lock-in of MS, rather that they want
and embrace such lock-in.  Many IT professionals have spent their careers getting various MS
certifications, are in a comfort zone, have tied their personal fortune to MS, and are loathe
to learn anything else.  Since company management defers to their "judgment", IT staff is able
to put the pieces in place that will guarantee their long-term employment in their comfort
zone.

In a lot of cases, I think it's as simple as the above scenario as to why obviously inferior
MS products gain significant market share.  Above all, politics rule the day in the Enterprise
Data Center.

Getting the (Share)Point About Document Formats

Posted Jan 10, 2009 16:19 UTC (Sat) by opensource2.0 (guest, #56065) [Link]

There is an open Java open source multi-platform alternative to Sharepoint, ExtenXLS 360(created by Extentech), a document automation/collaboration server, includes an open source AJAX Web spreadsheet with REST API and Java app server. Web app administrators retain full control of their data by storing document information in the database of their choice.


Copyright © 2007, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds