When Is a Standard Truly Open? - When It's Universal, Reflections on Massachusetts and Microsoft's XML [LWN.net]

November 30, 2005

By Pamela Jones, Editor of Groklaw

There are standards and there are standards. They are not all born equal.

What makes one standard open and not another? Massachusetts, when deciding to use the OpenDocument Format, as set forth in its Enterprise Information Technology Architecture (ETRM) document [PDF], set the bar here:

[Secretary of Administration & Finance for the Commonwealth of Massachusetts Eric] Kriss emphasized, however, that the state is not moving to open standards for economic reasons but to protect the right of the public to open and free access to public documents for the foreseeable future. "What we've backed away from at this point is the use of a proprietary standard and we want standards that are published and free of legal encumbrances, and we dont want two standards," Kriss said.

A recent statement by the Governor's office in Massachusetts, expressing optimistic hopes that Microsoft's Office Open XML document formats will meet their standard for an "open format" someday raises two questions: Is the Microsoft covenant not to sue, assuming it is someday offered for their new version of XML schemas, and their plans to submit their XML to standards bodies ECMA and ISO sufficient to meet the Massachusetts requirements for openness? If so, what are the implications for the Internet? If not, will Massachusetts decide their bar was set too high, in order to include Microsoft? If the bar was set too high, which part shall we lop off?

Shall we say the public has no right to open and free access to documents their tax money paid for?

No. Massachusetts has a Public Records Law [PDF] that mandates [PDF] that "all people have an absolute right of access to public information". The law doesn't distinguish between paper documents and digital documents. The disabled have a right of access, but so do the rest of us.

Are proprietary standards acceptable if nonproprietary, universally available standards are available?

Can a government dictate which operating system you have to use to access those documents? More pointedly, can it favor one proprietary vendor and compel citizens to spend money on a proprietary system, when they already have a perfectly functional operating system on their computers already?

We already saw in the Katrina disaster what happens when a government agency enables Windows-only access. I wrote about that in "When Open Standards Really Matter: the Katrina Factor", which begins like this, trying to explain why standards matter:

If you have any doubts about the direction Massachusetts is following in requiring open standards for all government documents, consider what happened when Hurricane Katrina knocked out almost all communications except the Internet. Cell phones and walkie talkies failed, once again, just as they did in 9/11, as David Kirkpatrick tells us in an article in Fortune:

In the immediate aftermath of the hurricane, much of the regions communication systems failed or didnt work properly. Water and wind knocked out power, toppled phone lines, and destroyed cellphone towers. What systems remained were quickly overwhelmed. When rescue workers did have working equipment, like walkie-talkies, they often couldnt connect with others on different communication systems.

Catch that? "On different communication systems." The same thing happened after the tsunami disaster in Thailand, as a report just released by the ePolicy Group reports:

"Responding agencies and nongovernmental groups are unable to share information vital to the rescue effort," the report recalls of the government in Thailand in the tsunami's immediate aftermath. "Each uses different data and document formats. Relief is slowed; coordination is complicated. The need for common, open standards for disaster management was never more stark or compelling."

Isn't it time, after so much suffering, to recognize that keeping people alive is more important than allowing private companies to lock in customers into proprietary systems that don't then work in an emergency? And why does the Internet always work, no matter who you are or what operating system you use? Because it was built, not on proprietary standards, but entirely on open standards. That's why you can send an email to me, even if you are using Microsoft Outlook. I don't run any Microsoft products currently, but because of open standards, I can still read your email, and in an emergency, we will not be disconnected because we are on "different communication systems."

Accepting Microsoft's proprietary XML *with its proprietary extensions* as the language of the Internet would certainly alter the openness and universality of the Internet. Proprietary, by definition, means it isn't universal. Is that what we want? If Massachusetts accepts proprietary extensions, it will inevitably be excluding some of its citizens, I think.

Can any government, including Massachusetts, in the Internet age, justify telling its citizens, by the decisions it makes, that they must stop using their operating system of choice -- one used by millions all over the world -- and instead purchase a proprietary vendor's product instead if they wish to interact with their government? In any case, in an emergency, it may not be possible to quickly buy a Windows operating system. People use what they have. Sometimes it's all they have access to; sometimes it's all they can afford. On what basis would you argue a government should do that? More pointedly, on what *legal* basis would you argue they can?

Should Massachusetts wait for a future, yet-to-be-developed proprietary standard, not yet published, on terms that are not yet fully known, controlled by a single vendor, when you have one that is vendor-neutral and is already available? On what basis could the Commonwealth justify such favoritism to a single company? In the FAQ on the Commonwealth's ETRM, they state this:

The Final ETRM Version 3.5 does not require that agencies use only one office product. To the contrary, it offers agencies many choices. Agencies may choose to retain their existing MS Office licenses, as long as they use a method to save documents in Open Document Format. They may also use one of the many office tools that support Open Document Format in native format--- OpenOffice, StarOffice, KOffice, Abiword, eZ publish, IBM Workplace, Knomos case management, Scribus DTP, TextMaker and Visioo Writer. Because the Open Document Format is an open standard, it increases the vendor pool available to state agencies by encouraging and permitting vendors not already in this field to develop products that support the standard. Adoption of the Final ETRM Version 3.5 will greatly increase competition among vendors for the sale of office applications to agencies.

What about legal encumbrances?

There are still questions about legal encumbrances on the MS XML and related licenses and EULAs. Is that going to be the part Massachusetts should wink at? They'd likely be sued if they tried, I think, and I doubt they will try, because it would be impossible to justify it in public, for one thing. That's the kind of thing that only works in back rooms, and nothing about this process will be hidden from the public's gaze.

On the issue of license clarity, obviously, if you can't understand the license, you won't dare to use the standard. I don't understand all the legalities of Microsoft's covenant not to sue and I don't know anyone who does. The danger is in establishing a standard that would, in effect, block Linux and GNU/Linux developers from participation. Do you imagine that could never happen? Consider your history.

Remember the SenderID flap? Who was to be left out in the EU Commissioner-MS deal? Do you really want to set up a world where Linux could never happen again, a world where the Linux we have is exiled to the backwaters of technology? Just being an ECMA standard doesn't mean it is open enough for *government* use. ECMA standards, by their own self-description, are for industry use:

Ecma is driven by industry to meet the needs of industry, generating a healthy competitive landscape based on differentiation of products and services, rather than technology models, generating confidence among vendors and users of new technology.

The needs of government are not identical to the needs of industry. It's up to industry to meet the needs of government, not the other way around. The government is the customer, after all, so they get to state what they need in a product and to choose whatever meets their needs best.

Finally, shall we have two standards?

That seems to be what they are leaning toward. The Commonwealth has already stated that the disabled can continue to use Microsoft's products, if there are no better options they like, so that might be one area of carve-out. From what I've heard, however, that will be a solved problem for ODF by the time of the 2007 rollout. So again, the issue is going to be: should a government favor a proprietary solution, when there is a universally available solution that isn't proprietary and excludes no one? Tim Bray has made a sensible suggestion:

The ideal outcome would be a common shared office-XML dialect for the basicsand it should be ODF (or a subset), since thats been designed and debuggedthen another extended vocabulary to support Microsoft features , whether theyre cool new whizzy features or mouldy old legacy features (XML Namespaces are designed to support exactly this kind of thing). That way, if you stayed with the basic stuff youd never need to worry about software lock-in; the difference between portable and proprietary would be crystal-clear. And, for the basic stuff that everybody uses, thered be only one set of tags.

This outcome is technically feasible. Who could possibly be against it?

Let's think about what we should look for in a standard, what elements justify even calling it a standard. I'd like to redefine the issue, if I may. I think the sine qua non for a standard isn't whether it's open or proprietary. If, for example, Microsoft can open up its XML to match ODF's, that's fine with me. The real issue isn't openness alone. It's universality. Let me explain.

In trying to understand fully the standards issue in Massachusetts, I came across a helpful list on Bob Sutor's blog dating back to September. I hadn't read it, and perhaps you didn't either, so I'd like to provide here the danger signs that a standard isn't as open as it needs to be. I think it's pertinent, because Microsoft folks are telling the world that by applying to ECMA, the conversation about whether their XML is open enough should be over. It's only just beginning, actually, as they will discover.

Consequently, let's list the elements Sutor provides that indicate there is a problem with a standard and you can judge for yourself:

In practice, I think it is probably easiest to tell when a standards effort is less open than you think it should be. Here are some danger signs:

control by a single vendor,
overly complicated license agreements,
license agreements that reserve certain special rights to individuals or vendors,
license agreements that prevent some kinds of implementations,
overly complicated procedural rules that can allow people to be less democratic than they should,
a history of disregard for backward compatibility,
costs of participation that exclude individuals or small organizations,
high costs of obtaining copies of the standards,
standard specifications not being openly available online, and
for XML-based standards, allowance for proprietary, vendor-specific extensions.

I see at least 6 items that would apply to Microsoft's XML. I'm sure I don't have to underline for you that the last is the deal-breaker, proprietary extensions. It's not a standard unless it's universally available and usable by everyone. The whole point of a standard is interoperability. Can everyone use it and have it work? Proprietary extensions, by definition, hamper interoperability.

The real question to me isn't just whether a standard is open. That's a continuum anyway, with various degrees of openness. The real question that matters is: is it universal? Can everyone freely use it? If not, should it be a standard?

XML Standards Impact the Internet, Not Just Massachusetts

When we're talking about XML, we're talking the Internet, not just about Massachusetts, because XML is the language of the future Internet. HTML was fine as the universal language for the Internet for its babyhood, but the future is XML, and so any discussion about standards for XML involve the Internet too. It's the next step, because it enables collaborative computing and universal data exchange.

No doubt Microsoft wants its proprietary version of XML to be adopted as a standard. But seriously, do you want the Internet's common language to belong, in essence, to one US company? And what a company! Ask yourself one question, and let your natural, first instinctive response answer: Do you trust Microsoft?

If you do, kindly tell me why, in light of their history. This is a company twice found guilty of antitrust violations, on two continents. What should that tell you? They had a version of the Internet too, remember? Happily no one was foolish enough to accept it. The theme of the Internet is universality, not proprietary walled gardens. Do you remember how Microsoft added proprietary extensions to HTML and degraded performance for competing browsers?

In the ETRM document, referenced above, the Commonwealth pointed out something very important:

Initiatives such as Homeland Security rely upon all parties adhering to Community of Interest XML specifications, defined by open standards bodies comprised of representatives from Government, Business and Technology Communities. Open formats for data files ensure that government records remain independent of underlying systems and applications thereby preserving their accessibility over very long periods of time.

Do you really think the Internet language should be determined by just one US proprietary company motivated by their own profit? The rest of the world will not go along with that, even if Massachusetts were to think so. And then how will they interchange data with countries and governments abroad? Massachusetts will be odd man out. Accepting Microsoft's version of XML with its proprietary extensions as the standard will thus result in serious issues ahead.

The Internet only works well if *everyone* uses the same language, and frankly, the whole world isn't going to accept Microsoft's proprietary XML as that common language. Can you imagine Europe agreeing to that? Or China? Or South America? FOSS developers? The next question is: Do we want two standards? Two Internets, in effect? That destroys the very purpose of the Internet, which is universality. Can there be justification for degrading the performance of something as important to us as utilities are for the sole benefit of a single proprietary vendor? I understand why Microsoft wants that, but why should you?

And there is a choice. ODF is here right now. There is no pie in the sky about it. It's not a year away. It's a universal standard, not vendor-controlled, which everyone can use, including Microsoft, without having to open up in any way themselves.

We're not just talking about Microsoft and Massachusetts, then. We're talking about the Internet, which doesn't belong to any company or any country. It's for everyone, and everyone uses it. Because it's universally available and usable, shouldn't the language of the Internet be universal too?

(Log in to post comments)