When Is a Standard Truly Open? - When It's Universal, Reflections on Massachusetts and Microsoft's XML
What makes one standard open and not another? Massachusetts, when deciding to use the OpenDocument Format, as set forth in its Enterprise Information Technology Architecture (ETRM) document [PDF], set the bar here:
A recent statement by the Governor's office in Massachusetts, expressing optimistic hopes that Microsoft's Office Open XML document formats will meet their standard for an "open format" someday raises two questions: Is the Microsoft covenant not to sue, assuming it is someday offered for their new version of XML schemas, and their plans to submit their XML to standards bodies ECMA and ISO sufficient to meet the Massachusetts requirements for openness? If so, what are the implications for the Internet? If not, will Massachusetts decide their bar was set too high, in order to include Microsoft? If the bar was set too high, which part shall we lop off?
Shall we say the public has no right to open and free access to documents their tax money paid for?
No. Massachusetts has a Public Records Law [PDF] that mandates [PDF] that "all people have an absolute right of access to public information". The law doesn't distinguish between paper documents and digital documents. The disabled have a right of access, but so do the rest of us.
Are proprietary standards acceptable if nonproprietary, universally available standards are available?
Can a government dictate which operating system you have to use to access those documents? More pointedly, can it favor one proprietary vendor and compel citizens to spend money on a proprietary system, when they already have a perfectly functional operating system on their computers already?We already saw in the Katrina disaster what happens when a government agency enables Windows-only access. I wrote about that in "When Open Standards Really Matter: the Katrina Factor", which begins like this, trying to explain why standards matter:
Catch that? "On different communication systems." The same thing happened after the tsunami disaster in Thailand, as a report just released by the ePolicy Group reports:
Isn't it time, after so much suffering, to recognize that keeping people alive is more important than allowing private companies to lock in customers into proprietary systems that don't then work in an emergency? And why does the Internet always work, no matter who you are or what operating system you use? Because it was built, not on proprietary standards, but entirely on open standards. That's why you can send an email to me, even if you are using Microsoft Outlook. I don't run any Microsoft products currently, but because of open standards, I can still read your email, and in an emergency, we will not be disconnected because we are on "different communication systems."
Accepting Microsoft's proprietary XML *with its proprietary extensions* as the language of the Internet would certainly alter the openness and universality of the Internet. Proprietary, by definition, means it isn't universal. Is that what we want? If Massachusetts accepts proprietary extensions, it will inevitably be excluding some of its citizens, I think.
Can any government, including Massachusetts, in the Internet age, justify telling its citizens, by the decisions it makes, that they must stop using their operating system of choice -- one used by millions all over the world -- and instead purchase a proprietary vendor's product instead if they wish to interact with their government? In any case, in an emergency, it may not be possible to quickly buy a Windows operating system. People use what they have. Sometimes it's all they have access to; sometimes it's all they can afford. On what basis would you argue a government should do that? More pointedly, on what *legal* basis would you argue they can?
Should Massachusetts wait for a future, yet-to-be-developed proprietary standard, not yet published, on terms that are not yet fully known, controlled by a single vendor, when you have one that is vendor-neutral and is already available? On what basis could the Commonwealth justify such favoritism to a single company? In the FAQ on the Commonwealth's ETRM, they state this:
What about legal encumbrances?
There are still questions about legal encumbrances on the MS XML and related licenses and EULAs. Is that going to be the part Massachusetts should wink at? They'd likely be sued if they tried, I think, and I doubt they will try, because it would be impossible to justify it in public, for one thing. That's the kind of thing that only works in back rooms, and nothing about this process will be hidden from the public's gaze.On the issue of license clarity, obviously, if you can't understand the license, you won't dare to use the standard. I don't understand all the legalities of Microsoft's covenant not to sue and I don't know anyone who does. The danger is in establishing a standard that would, in effect, block Linux and GNU/Linux developers from participation. Do you imagine that could never happen? Consider your history.
Remember the SenderID flap? Who was to be left out in the EU Commissioner-MS deal? Do you really want to set up a world where Linux could never happen again, a world where the Linux we have is exiled to the backwaters of technology? Just being an ECMA standard doesn't mean it is open enough for *government* use. ECMA standards, by their own self-description, are for industry use:
The needs of government are not identical to the needs of industry. It's up to industry to meet the needs of government, not the other way around. The government is the customer, after all, so they get to state what they need in a product and to choose whatever meets their needs best.
Finally, shall we have two standards?
That seems to be what they are leaning toward. The Commonwealth has already stated that the disabled can continue to use Microsoft's products, if there are no better options they like, so that might be one area of carve-out. From what I've heard, however, that will be a solved problem for ODF by the time of the 2007 rollout. So again, the issue is going to be: should a government favor a proprietary solution, when there is a universally available solution that isn't proprietary and excludes no one? Tim Bray has made a sensible suggestion:This outcome is technically feasible. Who could possibly be against it?
Let's think about what we should look for in a standard, what elements justify even calling it a standard. I'd like to redefine the issue, if I may. I think the sine qua non for a standard isn't whether it's open or proprietary. If, for example, Microsoft can open up its XML to match ODF's, that's fine with me. The real issue isn't openness alone. It's universality. Let me explain.
In trying to understand fully the standards issue in Massachusetts, I came across a helpful list on Bob Sutor's blog dating back to September. I hadn't read it, and perhaps you didn't either, so I'd like to provide here the danger signs that a standard isn't as open as it needs to be. I think it's pertinent, because Microsoft folks are telling the world that by applying to ECMA, the conversation about whether their XML is open enough should be over. It's only just beginning, actually, as they will discover.
Consequently, let's list the elements Sutor provides that indicate there is a problem with a standard and you can judge for yourself:
- control by a single vendor,
- overly complicated license agreements,
- license agreements that reserve certain special rights to individuals or vendors,
- license agreements that prevent some kinds of implementations,
- overly complicated procedural rules that can allow people to be less democratic than they should,
- a history of disregard for backward compatibility,
- costs of participation that exclude individuals or small organizations,
- high costs of obtaining copies of the standards,
- standard specifications not being openly available online, and
- for XML-based standards, allowance for proprietary, vendor-specific extensions.
I see at least 6 items that would apply to Microsoft's XML. I'm sure I don't have to underline for you that the last is the deal-breaker, proprietary extensions. It's not a standard unless it's universally available and usable by everyone. The whole point of a standard is interoperability. Can everyone use it and have it work? Proprietary extensions, by definition, hamper interoperability.
The real question to me isn't just whether a standard is open. That's a continuum anyway, with various degrees of openness. The real question that matters is: is it universal? Can everyone freely use it? If not, should it be a standard?
XML Standards Impact the Internet, Not Just Massachusetts
When we're talking about XML, we're talking the Internet, not just about Massachusetts, because XML is the language of the future Internet. HTML was fine as the universal language for the Internet for its babyhood, but the future is XML, and so any discussion about standards for XML involve the Internet too. It's the next step, because it enables collaborative computing and universal data exchange.No doubt Microsoft wants its proprietary version of XML to be adopted as a standard. But seriously, do you want the Internet's common language to belong, in essence, to one US company? And what a company! Ask yourself one question, and let your natural, first instinctive response answer: Do you trust Microsoft?
If you do, kindly tell me why, in light of their history. This is a company twice found guilty of antitrust violations, on two continents. What should that tell you? They had a version of the Internet too, remember? Happily no one was foolish enough to accept it. The theme of the Internet is universality, not proprietary walled gardens. Do you remember how Microsoft added proprietary extensions to HTML and degraded performance for competing browsers?
In the ETRM document, referenced above, the Commonwealth pointed out something very important:
Do you really think the Internet language should be determined by just one US proprietary company motivated by their own profit? The rest of the world will not go along with that, even if Massachusetts were to think so. And then how will they interchange data with countries and governments abroad? Massachusetts will be odd man out. Accepting Microsoft's version of XML with its proprietary extensions as the standard will thus result in serious issues ahead.
The Internet only works well if *everyone* uses the same language, and frankly, the whole world isn't going to accept Microsoft's proprietary XML as that common language. Can you imagine Europe agreeing to that? Or China? Or South America? FOSS developers? The next question is: Do we want two standards? Two Internets, in effect? That destroys the very purpose of the Internet, which is universality. Can there be justification for degrading the performance of something as important to us as utilities are for the sole benefit of a single proprietary vendor? I understand why Microsoft wants that, but why should you?
And there is a choice. ODF is here right now. There is no pie in the sky about it. It's not a year away. It's a universal standard, not vendor-controlled, which everyone can use, including Microsoft, without having to open up in any way themselves.
We're not just talking about Microsoft and
Massachusetts, then. We're talking about the Internet, which doesn't
belong to any company or any country. It's for everyone, and everyone
uses it. Because it's universally available and usable, shouldn't the
language of the Internet be universal too?
| Index entries for this article | |
|---|---|
| GuestArticles | Jones, Pamela |
Posted Dec 1, 2005 9:55 UTC (Thu)
by wingo (guest, #26929)
[Link] (2 responses)
While Jones is correct in her assessment of the viability of MS' XML, the ODF grassroots appear to me to be made of OO.o astroturf.
Corrections welcome of course -- I'm pretty ignorant about the topic.
Posted Dec 1, 2005 11:31 UTC (Thu)
by nathan (subscriber, #3559)
[Link] (1 responses)
Posted Dec 2, 2005 20:43 UTC (Fri)
by pimlott (guest, #1535)
[Link]
Both MS Office and OpenOffice are gigantic, hyper-featureful systems with over a decade of history. It's hardly surprising that their document formats, even newer XML-based formats, reflect their labyrinthine internal structures. It's going to take some hard study to determine which is a better "open" office format, or whether neither is really suitable.
Why is it that as soon as XML enters the picture, complex and difficult issues of representation and semantics suddently evaporate, and all we have to argue about is whether or not they used mixed-content?
Posted Dec 1, 2005 14:18 UTC (Thu)
by kevinbsmith (guest, #4778)
[Link] (3 responses)
It's not a perfect analogy, but XML is more like an alphabet than a language. Imagine that all the world's human languages shared a single writing script, such as the Latin letters used in English. Microsoft might control the English language (office doc schema) that is built with these letters, but that has no direct impact on the French or Thai languages (other XML schemas) built with the same letters. There are thousands of uses of XML that have nothing to do with the office doc XML schema.
Unless Microsoft is planning to modify the rules of XML itself (the alphabet), this debate really is only about office docs. The outcome will *indirectly* affect similar discussions in other domains, but would have no direct technical effect.
Posted Dec 1, 2005 14:39 UTC (Thu)
by pjgrok (guest, #17472)
[Link] (2 responses)
Posted Dec 1, 2005 23:21 UTC (Thu)
by giraffedata (guest, #1954)
[Link] (1 responses)
XML is not a language for writing documents, like HTML is. But I see a lot of people use the term "XML" as if it is. I suppose it's the same kind of mistake that leads people to use "IDE" (the technology of packaging a disk controller with the disk drive) as if it were a protocol for attaching a disk device to a computer. (In particular, they're thinking of the protocol called "ATA", one of many protocols that are implemented by IDE disk devices).
So does anyone know: Is there some XML-based document standard that Massachusetts is considering adopting? Does it have a name? Is there a Microsoft alternative, and does it have name?
Posted Dec 2, 2005 13:39 UTC (Fri)
by kevinbsmith (guest, #4778)
[Link]
Any product that reads or writes office documents is affected by this. And of course if MS wins this battle, it will be harder for anyone to push for governments to use other open formats like OGG or non-proprietary calendaring. This is an important decision, and it will have broad effects throughout the US and elsewhere, whether MS wins or loses.
But, to my knowlege, in the context of the Massachusetts/Open Document Format debate, MS is not saying they will mangle XML itself (the underlying rules of tags, attributes, etc) in any way that would have a technical effect on any XML format or app outside the office suite domain. So XHTML, the XML format used by all modern web pages, seems unlikely to be affected by the outcome of this battle. Likewise, web service XML formats, including blog feeds and SOAP transactions should be safe from harm here.
Posted Dec 1, 2005 15:40 UTC (Thu)
by Baylink (guest, #755)
[Link] (1 responses)
> Catch that? "On different communication systems."
is evidence of a lack of understanding of interagency emergency communications; it's not a good analogy for the situation at hand, in most current cases.
The em-comms issue is one of assigned and provisioned communications frequencies. Field em-comm equipment is *not* frequency-agile; it's channelised to match the municipality's license.
So, even though most of those radios are narrow-band FM (APCO 25 digital is up and coming, but not there yet that *I* know of, and besides, it, too, is standard for precisely this reason), they still can't talk to each other, unless it's pre-planned and agreed.
But the reasons are administrative.
The *real* reason is that while interagency channels and coordination plans do exist, none of them were designed for a disaster on the wide scale of Kat/rita. Ad, frankly, most of those people who need to talk on them have only the most basic instruction on how to handle a communications net with that many people on it.
This, on the other hand, is why we train and licence hams... though the integration of those hams into the emergency response plan was also, I gather, somewhat catch-as-catch-can. Hams *do* train for precisely that sort of situation (traffic triage; 1000 people on the same frequency, etc).
I'm sure we haven't heard all the stories yet.
Posted Dec 12, 2005 3:16 UTC (Mon)
by pjgrok (guest, #17472)
[Link]
I love you too, but it isn't just a bandwidth-type of problem. Different systems were also a problem, which you can verify by reading a Berkman Center white paper (http://cyber.law.harvard.edu/epolicy/roadmap.pdf) and this Washington Post article.
I knew about it because a Groklaw member who is an expert in the field told me. He was in discussions about the interoperability problem with FEMA before the hurricane hit. They were asking him about open standards and Open Source, but unfortunately the storm hit before everything could get fixed and ready. The same problem showed up in the Tsunami disaster and on 9/11, something I knew because I was in that state when it happened, and it is something you never forget. Here is a snip from the Berkman paper, quoted in a news article, on how communications broke down there in the tsunami crisis:
"Responding agencies and non-governmental groups are unable to share information vital to the rescue effort. Each uses different data and document formats. Relief is slowed; coordination is complicated. The need for common, open standards for disaster management was never more stark or compelling. The Royal Thai Government responded by creating a common website for registering missing persons and making open file formats in particular an immediate national priority."
This quote, taken from the opening of the white paper, "Roadmap for Open ICT Systems" by the Berkman Center for Internet and Society, Harvard Law School shows how open systems affect more than the business world ... The issue you raise is a problem too. But the analogy is based on reality.
Posted Dec 1, 2005 15:57 UTC (Thu)
by N0NB (guest, #3407)
[Link] (1 responses)
Based on some of the pronouncements comin out of MA during the past week it appears the vapors are having their intoxicating effects. MS is now "welcomed" and "thanked" by high government officials. Meanwhile, those that have championed true competition in MA are now suddenly subjected to audits and other investigations.
Politics has always been a dirty vocation, but there is no question that American politics has become especially nasty over the past quarter century. There is no doubt that MS sees not only much money but much perceived prestige on the line in MA and they are fighting like a junkyard dog--to the death if need be. They know that they cannot allow MA to mandate ODF as it seals their doom.
We are fighting MS with logic and reason which is often no match for the nuclear weapon MS wields--money--lots of money. Money is being funneled into the appropriate accounts, back pockets, and palms of politicians and IT pundits. This is a full-on frontal assault. MS is playing for keeps. Are we?
Posted Dec 2, 2005 10:10 UTC (Fri)
by kleptog (subscriber, #1183)
[Link]
His answer was along the lines of: Well, the annoying thing is that you don't get anywhere near as far with money because the whole campaign financing angle simply doesn't exist".
The presenter asked surprised: How so, annoying?
He said: Well, you know, you have to put more work in to get the same result. You have to convince them with arguments.
To which my only thought was: Good! You should work for your money.
Pity I can't find the reference anywhere or his name, but I'll remember the conversation for a long time.
Posted Dec 8, 2005 13:49 UTC (Thu)
by wolfrider (guest, #3105)
[Link]
From what I've heard from some developers of Abiword and Gnumeric, the ODF formats end up being reflections of openoffice.org's internal structure rather than well-designed formats. One even said that Microsoft's spreadsheet format was better designed (licensing issues aside of course).odf as a format
Whilst no specificly addressing spreadsheet requirements, this Format comparison between ODF and MS XML shows fairly clearly to me which format is better.
odf as a format
Let me get this straight: You conclude which is a better spreadsheet format, from a comparison which, you acknowledge, doesn't address spreadsheet requirements, but rather focuses on superficial matters such as stylistic conventions and affinity to other standards? And you poo-poo the statements of actual developers of office software? That makes no sense.
odf as a format
This is mostly a good article, with some interesting ways of looking at the issue. However, I wasn't impressed with the last section, where somehow the XML format (roughly speaking, a schema) used for office documents gets confused with the XML "schemas" used for web pages and web services."The language of the internet"
I think it will have the same type of effect that MS's "The language of the internet"
proprietary extensions to HTML had, because of its
dominance.
kevinbsmith's point is that nobody is talking about Microsoft extensions to XML; the article is flawed. They're talking about Microsoft extending something else. I would like to know what that is, and if it is a standard that has a name.
"The language of the internet"
My understanding is that there is a non-proprietary XML-based document format named "Open Document Format", used by several office apps (word processors, spreadsheets, etc). Microsoft has their own similar (but proprietary) format named "Microsoft Office Open XML". Apparently MS is proposing turning their format into an ECMA standard, but they would still have proprietary extensions (presumably schema extensions, with new tags or new meanings for existing tags) that could only be reliably written and read by their own products.Microsoft Extensions
While I love Pam to death, Interoperability - a bad analogy
Hi,Interoperability - nope, it's right on point
A thirty-foot-high wall of water - a tsunami - slams into the famed resort islands off Thailand's southern coast. In one tragic moment, thousands of lives are lost, and thousands more are missing. In the race to identify victims and assist survivors, Thailand's government hits its own wall.
Beware the Vapor!When Is a Standard Truly Open? - When It's Universal, Reflections on Massachusetts and Microsoft's XML
This reminds me of a news segment on here on Dutch TV where they were talking about EU lobbyists and what they should and shouldn't do and whether they should be registered. They were interviewing one of the firms that did lobbying and he was asked "What's the difference between lobbying in US vs in the EU?".EU vs US
This is the best front page news I've seen on LWN. :) Good article.When Is a Standard Truly Open? - When It's Universal, Reflections on Massachusetts and Microsoft's XML
