Tuesday, March 05, 2013

Punctuated Equilibriums and the Coup De Grace

It's never genesis (the innovation of the novel and new) which dramatically changes our society but commoditisation of pre-existing activities (from nuts and bolts to electricity to computing infrastructure).

Commoditisation (nee evolution of activities) to standard components allows for higher order systems to rapidly form (standard nut and bolts beget more machinery, standard electricity beget consumer electronics etc). However, it's the interplay of inertia to change (due to past success and co-evolution of practice) and evolution  (due to consumer and supplier competition) which creates different economic states of competition.

These states are peace (a time of relative competition), war (a fight for survival of past industries) and wonder (the mad scramble to establish new sources of future value). These states can be seen at both micro and macro-economic scales, at the macro scale we call them ages or k-waves.

Details on the above can be found in my series of posts on strategy but that's not the point of this post. What I want to discuss and focus on is the shift from the state of peace to the state of war, something which is happening in IT driven by commoditisation of a range of activities that are collectively called cloud computing.

The problem faced by past suppliers (vendors) in this changing environment is several fold :-
  • They suffer inertia due to past success.
  • Their consumers suffer inertia due to co-evolution of practice
  • The forces of evolution (consumer and supplier competition) are relentless and evolution to a more commodity / utility stage is inevitable.
  • The benefits of evolution - higher efficiency, increased agility in building higher order systems, new source of worth - are powerful and consumers of computing infrastructure will have to adapt to the new world just to keep their relative positions with their own competitors (Red Queen Effect)
  • The pace of change during the peace state is relatively slow creating an illusion that change is slow. Computing infrastructure for example has been in a state of relatively peaceful competition for 30 odd years.
It's easy for a past suppliers to think they have time and to be lulled into providing what the consumer is asking for (remembering that those consumers have their own inertia) rather than what the consumer actually needs. Alas, this change is inevitable and many past suppliers will end up being disrupted by an entirely predictable market change.

The quickening pace of change during the "war" is what often catches companies out. For example, take computing infrastructure and the growth of Amazon. In figure 1, I've plotted the median estimate of the growth of Amazon Web Services taken from a handful of "cloud watchers" statements which compares to my own much earlier analysis at Canonical because Amazon naturally won't reveal the actual figure. The sample is tiny, so take it with one huge big pinch of salt and the figures are based upon forward annual revenue based upon quarter 4 of each year (the one quarter that I can seem to be able to extract a reasonable revenue figure for). What this means, is that for each year after 2012, I'm expecting AWS to clock almost $2 billion in revenue. Sounds crazy? Just wait.

If (and it is a big if) the median estimate is close then Amazon Web Services currently generates a forward revenue equivalent to around 3% of the Worldwide server sales. Of course, AWS is more than just servers but I'll use this as my benchmark.

Figure 1 - Estimated Growth and Forward Revenue

The actual percentage is however fairly irrelevant, what matters is the rate of change. If I told you that it has taken Amazon six years to reach the equivalent of 3% of worldwide server sales and then asked you how do you think they will do in the next 4 years - you might well respond 5% or maybe 6%?

Alas, when activities shift from a state of peace to war we often experience a punctuated equilibrium, a time of rapid change where past "species" or in this case suppliers are significantly reduced or even eliminated.  

If I take the past growth rates and extrapolate forward, assuming (and it is a huge assumption) that Amazon's growth isn't checked by competitor actions and the estimates in anyway reflect reality and the starting conditions were even close, then by the end of that four years then Amazon should represent around 50% of the Worldwide future server sales (see figure 2).  Let us be clear hear, what I'm saying is AWS will have a revenue of almost $30bn for 2017 and onwards.

Before you got "that's mad" and we get into an argument over numbers, whether it's 30% or 50% isn't the important thing. What matters is that it'll be vastly higher than what most people would expect.

Figure 2 - Extrapolated Forward

We often get caught out by such rapid (and exponential change) and often deny it is happening due to the past state of peaceful and gradual change along with any inertia we might have. This is no different with other fields from climate change to biological evolution. When we see the past growth we want to extrapolate in a linear fashion. Alas, punctuated equilibriums exist and past performance is no guarantee for the future.

However, this time of rapid change also creates issues for the new suppliers along with the obvious negative effects for past suppliers.

Take for example Amazon EC2 which would appear to operate at high margin compared to equivalent highly commoditised private systems. The most efficient (and rare) examples of private clouds that I'm aware of run at between $100-$200 per year per EC2 equivalent instance which is vastly lower than Amazon's public price. So, assuming if Amazon can gain such efficiency why doesn't it just reduce price?

The problem is Jevon's paradox or in the case, the elastic nature of demand with supply. If Amazon reduces price (and it has done so some twenty times over the last six years) then demand goes up. Now, if you're in effect doubling already then too drastic a price reductions can push demand beyond your ability to supply because there are constraints such as buying land and building data centres. These constraint are capital intensive and time consuming. They are not instantly resolved. Hence price reductions have to be managed.

Of course, the same constraints could (and should) have been used against Amazon. The shift to cloud computing was signposted in 1966 in Parkhill's marvellous book and between 2002-2004 the signals were screaming "now".  By 2008, the past suppliers should have been well prepared and flooded the market with low cost alternatives forcing an increase in demand and fragmenting the market. The CEOs of the those tech companies should have been well aware of the storm that was approaching and the inertia in their organisations (due to past success). They had plenty of time to prepare.

Given this didn't happen and Amazon was allowed to steal the market, is quite startling. Certainly there has been a lot of recent noise from IBM's "big bet" on OpenStack to VMWare exec's shaking their fists but it's all extremely late in the game. It does look like some CEOs have been snoozing at the wheel.

To compound matters, Amazon has not only built rapidly growing services but also created a huge advantage through the ecosystem it has grown. Such ecosystems, if operated under an ILC (Innovate - Leverage - Commoditise) model enable a company to appear almost super linear for innovation, customer focus and efficiency. This does require a constant eating of the ecosystem and provision of new components and it's a careful game to play. Amazon certainly seems to be playing it.

Still, the battle isn't quite over yet.

The weakness is that it's just Amazon. You can compete against this by building a larger ecosystem around a market of suppliers (as per Android vs IOS). However, such a market needs semantic interoperability which requires an open source reference model unless we wish to create a captured market (dependent upon one supplier able to exert a tax on the entire market). It will also need an assurance body to limit the natural tendency for suppliers in the market to differentiate on function and hence weakening the market through a collective prisoner's dilemma. This is all old stuff but it's worth reiterating.

The reality is today we barely have any form of competitive market in the infrastructure space and that which exists lacks any assurance body and major scale investment (by which I mean the many billions needed). Communities are growing around OpenStack and CloudStack for example but the problem of course is that Amazon, through its ecosystem, is accelerating.

Given Amazon's dominance and growth, a model of direct competition through a market based around an open source reference model and alternative ecosystem seems a long lost hope. Instead a slower burn model of co-opting of the Amazon ecosystem appears the likely way forward i.e. build open source AWS clones with functional differentiation held back to a later date.  What the market needs to "differentiate" in the mid term should be the simple fact that it is a market not a single supplier. Alas, the slow burn model will take well over a decade but it is this approach I tend to favour and hence my interest in the Apache CloudStack effort. 

Amazon can still deliver its coup de grace by open sourcing its technology, the value of course being in the service and ecosystem and not the code. For Amazon, it probably has its eyes set on building a two factor market e.g. becoming the exchange for multiple providers and consumers. It probably has some concerns that no-one (bar possibly Google) is looking like a real competitor. 

This year should be interesting. For some it's probably the last gulp in the last chance saloon before their future in this industry becomes set.

--- 12th Feb 2015

Still amazed people don't understand punctuated equilibriums in the commercial world. Oh well.

--- 29th Feb 2016

We finally have some real figures. AWS closed $7.8 Bn for 2015. Now, I had the forward revenue rate at the end of 2014 to be $8Bn, so I was expecting that AWS would exceed $8Bn in 2015. It didn't but it was close. I won't give some half hearted excuse about the original analysis dating from my Canonical times (2008-2010) and ...  blah, blah ... wrong is wrong. No cigar.