Thursday, September 03, 2009

The cloud computing standards war

Over the last couple of years, I've consistently talked about the necessity for standards in the cloud computing space and the oncoming war that this will create. This was not some insightful prediction but simply the re-application of old lessons learned from the many industries which have undergone a transformation to a service world.

That standards war is now in full swing.

The principle arguments behind standards comes from componentisation theory (the acceleration of innovation through the use of standardised subsystems) and the need for marketplaces with portability between providers (solving the lack of second sourcing options and competitive pricing pressures). The two main combatants at the infrastructure layer of the stack are shaping up to be Amazon with the EC2 API and VMware with vCloud API.

Much of the debate seems to be focused about how "open" the standards are, however, there's a big gotcha' in this space. Whilst open standards are necessary for portability and the formation of markets, they are not sufficient. What we really need are standards represented through open source reference models, i.e. running code.

The basic considerations are :-

  • A specification can be controlled, influenced and directed more easily than an open source project.
  • A specification can easily be exceeded providing mechanisms of lock-in whilst still retaining compliance to a 'standard'.
  • A specification needs to be implemented and depending upon the size and complexity of the 'standard' this can create significant adoption barriers to having multiple implementations.
  • Open source reference models provide a rapid means of implementing a 'standard' and hence encourage adoption.
  • Open source reference models provide a mechanism for testing the compliance of any proprietary re-implementation.
  • Adoption and becoming de facto are key to winning this war.

So, in the war of standards whilst the vCloud API has sought approval from the DMTF and formed a consortium of providers, the Amazon EC2 API has widespread usage, a thriving ecosystem and multiple open source implementations (Eucalyptus, Nimbus and Open Nebula).

There appears to be a lot of FUD over the intellectual property rights around APIs and a lot of noise over vCloud adoption. You should expect this to heat up over the next few months because these early battles are all about mindshare and will heavily influence the outcome.

However, whilst VMWare has struck boldly it has exposed a possible achilles heel. The only way of currently implementing vCloud is with VMWare technology, there is no open source reference model. If Reuven is right and Amazon does 'open up' the API, then Amazon have a quick footed route to IETF approval (multiple implementations) and can turn the tables on vCloud by labeling it as a "proprietary" only solution.

Of course VMWare could pre-empt this and go for an open source route or even attempt to co-opt the various open source clouds into adopting their standard. I'd be surprised if they weren't already trying to do this.

This space is going to get very interesting, very quickly.

11 comments:

Anonymous said...

Also have a look at http://www.cloud-standards.org

jake said...

I don't think standards are a bad thing, but I do believe they need to not be created to benefit the large cloud providers at the expense of innovation. www.workxpress.com

M. Behrens said...

Check out the Open Cloud Computing Interface (OCCI) standards work which is just getting started within the Open Grid Forum (OGF) standards body:

http://www.occi-wg.org

Christofer Hoff said...

Where does this statement come from? "The only way of currently implementing vCloud is with VMWare technology, there is no open source reference model"

That's absolutely inaccurate. The API has abstracted away the notion of VMware dependency; I even discussed this @ VMworld with Simon Crosby. Anyone may implement the API given the reference model currently defined as far as I can tell.

I assume you've read the API specifications?

/Hoff

swardley said...

@jake: There is this idea that standards somehow inhibit innovation. Componentisation would suggest otherwise, it will accelerate it.

@M.Behrens: thanks for the link. I'm well aware of the OCCI and various groups working on standards. I'm a great believer that the market will choose.

@Christopher Hoff: As for reading the specification, yes I have but I now wait with baited breath for you to provide a link to an open source cloud system that currently provides the vCloud API. Whilst Eucalyptus, Open Nebula an Globus Nimbus are all open sourced cloud systems, they implement the EC2 API.

I'm absolutely busting to see the open source cloud system which you claim implement the vCloud API.

Roger said...

Amazon should definitely open up EC2 APIs. That will be a very wise move in order to undermine VMWare´s vCloud initiative.

Christofer Hoff said...

@swardley:

You said: "@ChristoFer Hoff: As for reading the specification, yes I have but I now wait with baited breath for you to provide a link to an open source cloud system that currently provides the vCloud API. Whilst Eucalyptus, Open Nebula an Globus Nimbus are all open sourced cloud systems, they implement the EC2 API.

I'm absolutely busting to see the open source cloud system which you claim implement the vCloud API."

...perhaps I misinterpreted your comments here, but it seems to me like you are implying that the API somehow constrains a vendor who might choose to utilize the API to only utilize VMware.

It's certainly true that those who are developing against the API THAT WE KNOW OF are using VMware to do so, but are not limited as such.

I'd argue given the level of abstraction provided for in the API, it does the opposite and is not tied to a specific implementation, unlike the AWS API which allows one to manage only...?

I see you did use the word "currently," but I was reading between the lines. Perhaps this was impertinent.

/Hoff

swardley said...

@beaker (Christofer Hoff - huge apologies for the terrible spelling).

From my post "The only way of currently implementing vCloud is with VMWare technology", this does not imply that the vCloud API is constrained to VMWare Technology only that the current methods of implementation use VMWare technology.

This is the source of misunderstanding.

Mark said...

Hmm.. you failed to point out that it's really all about control. Openness doesn't matter. DeJure vs DeFacto doesn't matter. The winner is the one that gets adoption and yet still maintains change control over the API.

So, call it an API war, but it is certainly not a standards war. It's a war between one company controlling the API vs. a vendor neutral body controlling the API.

Which one benefits the customers?

Which one will actually be widely adopted in the end?

swardley said...

@Mark: The greatest benefit to all users is gained from the formation of a competitive marketplace with portability between providers based upon standards which are not tied in or controlled by a vendor. The latter rarely happens and we normally end up with some form of government intervention and control.

As you say the winner is the one with the greatest adoption (that, by the way, is the meaning of de facto) rather than that blessed by some committee or otherwise (the de jure).

Your point about it's an API war not a standards war is excellent. The end result is to become the de facto standard but at this moment it's API vs API.

Cohen said...

also have look this
http://bygsoft.wordpress.com/2010/01/09/cloudy-combo-google-app-engine-and-amazon-s3-combo-pack/