If you've been paying attention then you'll have noticed that the UK Government has had a fairly torrid time of trying to implement a policy for open standards.
It eventually won that fight despite efforts from numerous groups such as the BSA, large US companies apparently attempting to influence the process, questions raised over the role of one of the chairman and software patent heavyweights piling into public meetings. At the end of several rounds of shenanigans and lobbying, the UK Gov was finally able to announce a comprehensive policy towards open standards. It was finally able to do what the only respected peer reviewed academic paper on the subject said was a good thing to do - adopt the policy.
However, this fight was just to get the policy approved. We should of course expect many more fights when it comes to specific open standards. Why? Well, it has all to do with money or more importantly the ability of certain companies to help themselves to large amounts of your and my money.
To explain why, let us first understand why we need standards. One of the fundamental purposes of standards is to provide you with choice. If two systems use the same standards then you can switch between both and this switching encourages competition. The reason why UK Gov has a preference for open standards is that it also has a preference for a fair and competitive marketplaces i.e. if the standard is owned by someone or encumbered with patents then that person can limit competition in the market.
Shouldn't we just therefore make everything into a standard? No. There is a cost with standardisation in that it limits innovation at the interface of the standard, hence the implementation of standards is only suitable for activities that have become commonplace and mature.
But surely the marketplace as it matures will define its own standards? Yes. These are called the defacto standards and in most cases they should be adopted. However, in a mature market where the defacto doesn't meet some user need or encourage competition then Governments might need to introduce standardisation to create a competitive market.
Can you give me examples? Certainly. Two examples spring to mind.
1) In the case of cloud computing you have an evolving market where defacto standards are developing. As an evolving market it is too soon for any Government to attempt to implement standards.
2) In the case of documents and word processing systems (like Open Office, Libre Office and Microsoft word) then you have a highly mature and established market. However, in this case despite there being a defacto there is not easy switching between a competitive market of products and this is most clearly shown when people talk of the cost of transfer from the defacto to another.
So why should I care as a vendor? If you are the defacto provider for a mature marketplace and there is a significant cost for people to transfer off your system then you've in effect created a captured market and this can be very lucrative. You won't experience the normal competitive pressures of a free market, you can charge high prices knowing that most won't wish to incur a cost of transfer. This actually creates an incentive for you to lock people further into your system and you'll try and stop anything which threatens this such as open standards.
So why should I care as a customer? Well, if it's a mature market with a defacto and there are limited alternatives unless you incur a high cost of transfer then you're likely to be paying well over the odds. What's operating is not a free market but a captured one and your exit cost from the existing defacto is simply a long term liability which will keep on increasing. In such cases, you should be in favour of open standards because that's the only way you'll get a fair and competitive (known as 'free') market and limit future liabilities.
So that gives some basic background on open standards. The real question is why do I bother mentioning this now? Well the UK Government has just launched a process to select open standards for government documents. I fully expect the lobbying machine to be out in full on this one as what's at stake is oodles of cash and control of a market.
Microsoft Office is obviously the defacto in a well established and mature market which has little differential between product offerings but also high costs of transition. The market has all the appearance of a captured market with weak competition.
However, doesn't Microsoft Office 2010 uses .docx which is Microsoft Office Open XML Format and therefore an open standard? Alas no.
Microsoft Office 2010 provides a number of formats (docx, pptx, xlsx) that collectively are called Office Open XML format (OOXML or OpenXML). This file format was submitted to ISO after intensive lobbying including accusations of rigging and it known as the OpenXML standard (ISO/IEC 29500 adopted in 2007).
Alas, this standard was broken into two parts. There is the standard itself, which is known as strict OpenXML and was accepted by ISO (international standards organisation) and then there is transitional OpenXML which was supposed to be a transitional file format for Microsoft to ease the removal of some of the past closed source legacy from their file format. Of course, Microsoft Office 2010 implemented transitional OpenXML but only reading of strict OpenXML.
So when you use Microsoft Office 2010 to send a document in an OpenXML format such as .docx to someone who uses Libre Office which reads and writes strict OpenXML (as defined by ISO) and they edit the document and send you back a .docx file, then what you see is often a corrupted file or something which can only be read and has to be converted.
You probably think that something is wrong with their software. Well, this problem isn't because of their word processor but instead Microsoft Office 2010 didn't implement strict OpenXML except for reading. It certainly claims that .docx is OpenXML, and that OpenXML is an open standard approved by ISO but the version of OpenXML that ISO approved (strict) is not the same version of OpenXML that Microsoft Office 2010 .docx uses (transitional). This means every other system has to reverse engineer the transitional OpenXML format that the default .docx uses and that's never perfect.
So let me be crystal clear, Microsoft Office 2010 didn't even fully implement the open standard it created and was approved by ISO after its own intensive lobbying. It did promise to provide reading / writing of this format in the future.
Hang on that's old news .... doesn't Microsoft Office 2013 now implement reading / writing of strict OpenXML? Well, the latest release of Microsoft Office does now implement strict OpenXML, unfortunately the default .docx is still transitional OpenXML, you have to specifically select strict OpenXML when you save the file (which is buried in the options) and of course, in order to use Microsoft Office 2013 you need to be running Windows 7 or Windows 8.
Hang on that's old news .... doesn't Microsoft Office 2013 now implement reading / writing of strict OpenXML? Well, the latest release of Microsoft Office does now implement strict OpenXML, unfortunately the default .docx is still transitional OpenXML, you have to specifically select strict OpenXML when you save the file (which is buried in the options) and of course, in order to use Microsoft Office 2013 you need to be running Windows 7 or Windows 8.
So basically, yes you can have the open ISO standard strict OpenXML if you upgrade your operating system, buy the latest version of Microsoft Office 2013 and remember to save in the right format. It must be noted that whilst Microsoft Office 2013 does now provide support for strict OpenXML and even includes group policies for this, the default is .docx which is using Microsoft's own version of an open standard (transitional OpenXML) but without actually being that open standard (strict OpenXML). It's all a bit messy and confusing.
So what does this mean in practice? Well, let us assume that the UK Gov chooses the ISO approved strict OpenXML as an open document standard. First, that means Gov departments would need to upgrade to Microsoft Office 2013 which means upgrading to Windows 7 or 8 and all the changes needed to applications (NB. Microsoft XP is out of support in April 2014). They'd also have to set group policies so that the default was strict OpenXML.
However, let us suppose you create a document, a .docx file and sent it to someone else who happens to be using Microsoft Office 2010. Unfortunately though they could read the .docx file (Microsoft Office 2010 provides read support for strict OpenXML) they couldn't write / edit such a document without changing it to another type of .docx (the default transitional OpenXML). Hence you might send out strict OpenXML but you could easily get back transitional OpenXML though both files are called .docx.
Now given that some companies / organisation are still just in the midst of rolling out Microsoft Office 2010 then you're going to have all sorts of problems trying to introduce use of strict OpenXML as the .docx format and in practice the default transitional OpenXML format will rule for many many years to come. At best, you're going to end up with a messy estate of .docx files some of which are strict format and some of which are transitional format but all are called .docx.
Fortunately if you're using Microsoft Office 2013 this is ok and the more messy the environment, the more incentive there is to all upgrade (including the Operating System). Alas, there are only a limited number of alternative companies providing word processing capable of reading / writing Microsoft Office .docx (transitional format) as it currently stands because of the effort needed to reverse engineer transitional OpenXML (remember strict is the standard).
For example Libre Office does provide the ability to save files as two different .docx formats - one for "Microsoft Word 2007/2010 XML (.docx)" which is the reverse engineered transitional OpenXML format and one for "Office Open XML (.docx)" which is the strict OpenXML format that is the ISO standard.
The upshot of this, for many it'll just seem easier to stick with Microsoft, upgrade the OS and Office package, accept a messy estate and in all probability stick with transitional OpenXML as the .docx format. So much for a competitive market and open standards then.
However, there is an alternative - ODF.
So what is ODF? It stands for Open Document Format, it covers a number of office formats (.odt, .ods, .odp) and is an ISO/IEC standard 26300 adopted in 2006. It is supported by multiple technology providers including AbiWord, Adobe Buzzword, OpenOffice, Atlantis Word Processor, Calligra Suite, Corel WordPerfect Office X6, Evince, Google Drive, Gnumeric, IBM Lotus Symphony, Inkscape exports, KOffice, LibreOffice, Microsoft Office 2010 onwards, NeoOffice, Okular, OpenOffice, SoftMaker Office and Zoho Office Suite.
Microsoft also provides support for ODF however as it says "Formatting might be lost when users save and open .odt files". Brilliant.
One of the big advantages of word processing / spreadsheet and presentations systems like Libre Office is they have close to feature parity with Microsoft Office but they also aren't tied to a specific operating system i.e. you can get Libre Office for Ubuntu, MacOSx and Windows.
Microsoft also provides support for ODF however as it says "Formatting might be lost when users save and open .odt files". Brilliant.
One of the big advantages of word processing / spreadsheet and presentations systems like Libre Office is they have close to feature parity with Microsoft Office but they also aren't tied to a specific operating system i.e. you can get Libre Office for Ubuntu, MacOSx and Windows.
So why doesn't Microsoft just adopt ODF? In a nutshell, control of a market. Why simply give up a captured market if you don't have to and especially if you can persuade people that the exit costs that your product has created aren't actually an ongoing liability. If you can use those exit costs to persuade people against moving, explain that your product really is 'open' with OpenXML, add in a complex mess of strict and transitional formats then you can hopefully can get people to stay put and just upgrade both Office and the operating system. You get to keep the captured market intact!
Oh, and by the way, those exit costs are significant. Microsoft's own people estimated that adoption by UK Government of ODF as the standard would cost in excess of £500 million. What that means is we're currently locked into an environment which it will cost £500 million to escape from (assuming the figure is correct) but what they fail to mention is that 'liability' is unlikely to decrease over time. It's that 'liability' which keeps us paying for new versions of Office, new versions of OS, application upgrades and also prevents a truly competitive market for what is fundamentally a common and well defined activity - writing documents etc.
Oh, and by the way, those exit costs are significant. Microsoft's own people estimated that adoption by UK Government of ODF as the standard would cost in excess of £500 million. What that means is we're currently locked into an environment which it will cost £500 million to escape from (assuming the figure is correct) but what they fail to mention is that 'liability' is unlikely to decrease over time. It's that 'liability' which keeps us paying for new versions of Office, new versions of OS, application upgrades and also prevents a truly competitive market for what is fundamentally a common and well defined activity - writing documents etc.
This is where Government really has to step in because it has power and influence in the market.
This is clearly a mature but captured market which the introduction of an open standard will encourage greater competition, reduce long term liabilities such as exit costs and therefore benefit users. There is no reason why Microsoft can't upgrade previous versions of Office to write strict OpenXML (they've had six years) and change the default .docx to this, however it's probably just not in their interests to do so. The pragmatic choice for Government would therefore seem to be ODF.
If you select ODF then there is also no reason why Microsoft cannot sell office products based upon ODF (which it supports) and if not Microsoft then there's lots of other potential vendors (see list above) including open source solutions who will. Since Portugal and other Governments have already gone down a route of establishing ODF as the standard then it also doesn't make much sense for the UK Gov to create its own island of technology standards.
If you select ODF then there is also no reason why Microsoft cannot sell office products based upon ODF (which it supports) and if not Microsoft then there's lots of other potential vendors (see list above) including open source solutions who will. Since Portugal and other Governments have already gone down a route of establishing ODF as the standard then it also doesn't make much sense for the UK Gov to create its own island of technology standards.
But then again, standards are rarely about pragmatism and more about vested interests. Now this is the reason why there will be a fight because you're talking of an established defacto with a captured market potentially being forced into giving up that control and competing in a free market. I expect to see lots of intense lobbying over why OpenXML should be the standard despite the default .docx of Microsoft Office 2010 and 2013 being transitional OpenXML and not the approved ISO open standard strict OpenXML.
At the end of the day this is about word processing, spreadsheets and presentations. We don't need umpteen file formats for this and certainly not two different formats for .docx such as strict and transitional. This will be claimed to be a necessity for reasons of transition but we've had six years and no end in sight to this mess. In reality this is all about prolonging control of a captured market and almost nothing about end user needs.
This is why Governments must act but of course, it'll need you to help counter the well funded lobbyists that are likely to appear - just like last time.
Before I go, I thought I'd better clear one last thing up. I happen to love Microsoft Office. I use it all the time on MacOSX in particular Excel. However, I'd rather see Microsoft compete on providing me with a better experience / product which was based on ODF (an open standard) in a competitive market than what is happening today. I do understand the temptation to claim something is an open standard when it is not, I do understand the gameplay and how standards can be used to control the market.
I hope, maybe with a change of leadership that Microsoft learns it can compete purely on having better products and not by attempting to create and exploit lock-in for what is a common activity. I fear that this standards debate will just become another example of Microsoft rallying troops into the fray with the usual cries of being "open", concerns over exit costs that ignore long term liabilities and funded "white papers" masquerading as academic work etc.
Hence in the next post, I'll talk about some of the common tactics we're likely to see.
At the end of the day this is about word processing, spreadsheets and presentations. We don't need umpteen file formats for this and certainly not two different formats for .docx such as strict and transitional. This will be claimed to be a necessity for reasons of transition but we've had six years and no end in sight to this mess. In reality this is all about prolonging control of a captured market and almost nothing about end user needs.
This is why Governments must act but of course, it'll need you to help counter the well funded lobbyists that are likely to appear - just like last time.
Before I go, I thought I'd better clear one last thing up. I happen to love Microsoft Office. I use it all the time on MacOSX in particular Excel. However, I'd rather see Microsoft compete on providing me with a better experience / product which was based on ODF (an open standard) in a competitive market than what is happening today. I do understand the temptation to claim something is an open standard when it is not, I do understand the gameplay and how standards can be used to control the market.
I hope, maybe with a change of leadership that Microsoft learns it can compete purely on having better products and not by attempting to create and exploit lock-in for what is a common activity. I fear that this standards debate will just become another example of Microsoft rallying troops into the fray with the usual cries of being "open", concerns over exit costs that ignore long term liabilities and funded "white papers" masquerading as academic work etc.
Hence in the next post, I'll talk about some of the common tactics we're likely to see.
