Tuesday, May 06, 2014

What if ... co-opting Amazon APIs

Back at OSCON in 2007, I talked about the necessity of creating a competitive market of multiple cloud infrastructure providers based upon the same open source technology in order to create a functioning free (as in unconstrained) market. This required co-opting the Amazon APIs.

I repeated the same advice to Eucalyptus before they commercialised in 2008 and to Rackspace before the OpenStack project was created. Alas in the case of Eucalyptus they temporarily embarked on an open core route and OpenStack decided to differentiate (which led to the current prisoner dilemma and other issues).

Alas, it's impossible to roll the clock back and determine exactly what would have happened. However let us suppose that OpenStack had decided to launch as an AWS clone and that the vendors had realised that co-option was their route forward, what could have happened?

First, it's worth remember that AWS services have a long tail i.e. there are core services such as EC2, S3, EBS which are heavily used and then other services which are less well used. Hence in producing an  effective AWS clone you'd only have to produce the most commonly provided services - something which customers can tell you.

If, in early 2010, the main 'players' - e.g. IBM, Dell, HP, Cisco, Rackspace, Google, PayPal, AT&T, Yahoo etc had got behind an open source AWS clone effort with significant investment - say $200M each in the first year, doubling each year - then by end 2011 you would have likely seen the first public AWS clones. 

Ok, they wouldn't have all the features of AWS but it would be enough to start to form a competitive market and some authoritative mechanism for ensuring semantic and syntactic interoperability between providers. By end of 2012, the market would have had enough internal investment - with a run rate of $8Bn+ - to start to initiate a price war.  The purpose of which would be to increase demand (compute is elastic) beyond the ability of one provider to supply (supply is constrained by building data centres) and to naturally fragment the market.

This wouldn't have stopped Amazon's growth but simply accelerated the growth of the market, overcoming concerns over single supplier options and providing a vibrant market. By the end of 2014, the AWS clone market should have become larger than AWS itself with all the main players having significant public providers and a reasonable investment of $3Bn+ each year and a total AWS clone market investing $30 Bn+ a year.

The AWS clone market would in effect start to control the market and take ownership of the APIs i.e. if the clone market decided to move the APIs in a particular direction then Amazon itself would be under pressure to follow.

Of course, none of this happened. 

One of the reasons why, and probably the most ludicrous and naive reasons, is because of concerns over legal ownership of the APIs and concerns that a legal challenge could be mounted. First, APIs are principles not expressions and even if Oracle succeeded in changing this (with their battle with Google) by the time this would have happened the AWS clone market would have in effect controlled the infrastructure API space and could easily as a  single group changed this API base. 

Amazon, is a smart player and was always more likely to want a significant chunk of a huge pie rather than get itself cut out of the market. Had the AWS clone market grown, there is unlikely to have been any challenge on API ownership. 

Alas none of this happened. The numpties who raised concerns over legal status of APIs and the desire to differentiate won the day. No competitive market formed. OpenStack is suffering from a collective prisoner dilemma. The players are all a tiny fraction of the juggernaut that Amazon has created.

It didn't have to be this way but the past happened. However, there are some lessons for the future including :-

1) Never, ever rely on strategic advice from a lawyer especially an IP lawyer. Remember IP is just a tool, a tactic and not the end goal. It must be balanced with an understanding of the landscape and competing concerns.

2) Never, ever try and differentiate in a commoditising market. Accept it is commoditising and exploit this including co-option where needed. Avoid the epic CEO fails.

3) Learn to effectively use Open as a weapon.

As for anyone asking 'Is it all too late, has Amazon won it all?' - well, of course not. There are many ways that Amazon can still be outcompeted. However, those paths are complex and beyond the capabilities of most companies / execs given that even simple plays (such as adoption and co-option) have confounded many. So, they're not even worth mentioning.

Instead focus on moving the battle into the platform space (i.e. co-opt and build around CloudFoundry) and learn to play substitution games for the lower order systems.