The abuse of innovation.

Innovation is a term which is widely abused and this abuse prevents us from seeing patterns in how business activities evolve.

It is difficult to see what changes when everything is called an innovation in the same manner that it's difficult to see the difference between commodification (assignment of economic value) vs commoditisation (shift from imperfect to perfect undifferentiated competition) because of the catch-all nature of the term commodification (i.e. it's used to mean both).

Take for example the utility provision of computing infrastructure (as per Amazon) - is it an innovation?

When it comes to computing infrastructure, the innovation of modern computing probably started with the Z3 in 1941. This act of innovation created an entirely new class of activity - computing infrastructure - which has evolved over time through various stages with custom built examples (LEO etc), products (IBM 650 and onwards) and eventually led to commodity and utility provision. For reference, the full cycle is innovation, custom built, product (with rental services) and commodity (with utility services).

Two things should be noted, firstly that the pathway of evolution is common for activities (and knowledge) though it's not a time based sequence. Secondly, the innovation of the Z3 created a new class of activity rather than evolved an existing class (as with the first phone, the first radio, the first ...)

When it comes to the shift from products to utility, this simply represents an evolution of an activity and not the creation of a new form i.e. infrastructure existed before Amazon. However, it is perfectly true to say that this evolution enables (through creative destruction) and accelerates (through componentisation) the innovation of higher order systems i.e. as infrastructure has evolved we've seen an explosion of innovation in big data, mash-ups etc. This is perfectly normal as commoditisation (the common term used to describe this evolution) creates a cycle with innovation.

So, we have a difference between innovation of a new activity and evolution of an existing activity - both of which we unfortunately call innovation.

To complicate matters there's also the consumer and provider perspective. Whilst electricity is a commodity provided through utility services to consumers, behind the interface (the plug) has been a world of innovation of novel activities (wind farms, solar power, geothermal etc) aiming to create some form of operational advantage. However, it is worth noting that this provider innovation doesn't suddenly turn a consumer commodity into an innovation.

Finally we have terms like sustaining and disruptive innovation. As an activity evolves, in many cases changes to the activity (such as feature differentiation in the product stage) are sustaining and occasionally they are disruptive.

When an activity evolves across a boundary i.e. shifts from products to utility services (as with cloud) then this shift is generally disruptive because the incumbents have huge inertia to the change caused by their past success in the previous stage of evolution (i.e. product or rental vendors).

So the pattern we have is :-

1. Innovation of a genuinely new activity which is distinct from the evolution it enables.
2. Evolution of an activity to custom-built, product (rental) to commodity (utility services). This process is commonly called commoditisation.
3. Sustaining changes dominating within domains (i.e. product)
4. Disruptive changes dominating between domains causing a discontinuity with the past (i.e. product to utility services)
5. Enablement and acceleration of the innovation of higher order systems through commoditisation of lower order subsystems (i.e. creative destruction and componentisation)
6. A difference between consumer and provider perspective.

Now, the problem with the abuse of the term innovation is we end up with :-

1. Breakthrough Innovation
2. Feature, Product and Service Innovation
3. Sustaining Innovation
4. Disruptive Innovation
5. Loads of Innovation (paradigm shift etc)
6. It's my product, of course it's an Innovation ...

We normally shorten this to Innovation, Innovation, Innovation, Innovation, Innovation and Innovation.

Or in other words Innovation.

You have no hope with spotting the pattern under such circumstances and it's no wonder that people get confused with this subject. This has severe impacts on management practices but that's a post for another day.

As for Amazon's EC2, it represents an evolution of an existing activity which is disruptive, will enable breakthrough innovation of higher order systems and for the provider has probably involved a mix of different types of innovative pursuits in operations.

I hate to give up on words, however "innovation" has become so widely abused as to be meaningless. For the future I'm tempted to use the word "Genesis" to describe the creation of a new activity and to put "innovation" in my book of pointless words along with "Cloud" etc.

Hosting Con Keynote

I was very fortunate to be asked to give the opening keynote at Hosting Con 2011 covering commoditisation, business evolution, leadership and what the various tactical plays in the cloud computing space mean to hosting companies. The audience was fantastic, I had a great time and despite using excessive numbers of slides, no-one was hurt in the process.

Continuing on the theme from my OSCON tutorial, I've uploaded a summary set of slides which are highly condensed but give a taster to what we covered.

Alas, there's no video and as per usual I'm six years into writing my book and around 30% of the way there. The subject matter keeps on giving me more areas of interest to explore, so don't hold your breath for me to finish any time soon.

Summary of Hosting Con 2011 keynote

View more presentations from swardley

OSCON Tutorial

I gave a three hour tutorial at OSCON on innovation, commoditisation, business evolution, organisation, leadership and various tactical plays in the cloud computing space. The talk was a blast, I really enjoyed it and judging by the feedback it hit some home runs with many of the audience.

However, the presentation is 1,041 slides long and so - I'm not uploading that or creating a video. Instead I've made a summary presentation which covers the main points.

Be warned, it's highly condensed.

Situation Normal Everything Must Change

View more presentations from swardley.

Is Microsoft's biggest enemy … Microsoft?

Last year at OSCON, I examined mechanisms by which a company could use technology evolution to disrupt an existing player with minimal fear of retaliation. To quote myself :-

"it's the incumbents existing model which will protect you"

This year at OSCON, I'll be giving a three hour tutorial which will explore the entire subject of organisational warfare in far more detail. To give a taster of what is to come, I thought I'd expand upon some of the reasonings behind the above statement.

Anyone who has been following my public presentations over the last seven years or has been exposed to my exploration of this subject over the last decade+ will be well versed in much of this practice. For those uninitiated in this field, I'll start with some basics.

All business activities evolve through a common lifecycle and Cloud Computing is simply an example of this. Unfortunately, whilst we know how things will change, we cannot say when. The pattern of evolution is independent of time which is why the lifecycle graphs that I use have no time axis. But then they could never have a time axis, the future is an information barrier we cannot see past.

Fortunately, there are also barriers to the process of evolution and these give us a sense of when things will happen. In order for an activity to evolve from the domain of products to that of utility services then the following four factors are required - concept, suitability, technology and change in attitude. These factors are our "clue" that change will happen.

So we can predict how things will change, just not when - at least not with any great accuracy.

As any business activity evolves along its lifecycle its characteristics change from more chaotic (e.g. appearance of constantly changing, highly uncertain) to more linear (e.g. appearance of being defined, predictable, measurable). Using this change of characteristics we can develop organisational models which cope with evolution. An example of this is the Innovate-Leverage-Commoditise (ILC) pattern which can be found with many cloud companies.

So we can predict what will happen and though we can't predict precisely when, we can design an organisation around evolution.

There are two tactical plays that I'd like to discuss which can be used in such an environment. The first is around creative leadership, think Steve Jobs' Apple. The other is around disruptive leadership for which the best example would probably be Amazon.

Amazon is a company which rarely seems to create a new activity but instead specialises in commoditising activities and disrupting existing players. Infrastructure existed before EC2, book distributors existed before Amazon.com and the paperback existed well before the kindle. Amazon is extremely good at the disruption game.

To illustrate this disruption play further, figure 1 provides a rough tactical map of Salesforce. Whilst the incumbents are firmly entrenched in the product world for provision of CRM (& sales automation), Salesforce has commoditised this activity through provision of a standardised service. It also appears to be operating an ILC pattern i.e. core services around which a growing ecosystem is used to encourage innovation and identify new successful patterns which are then subsequently commoditised to core services - hence innovate, leverage and commoditise. This model is little different from Amazon's play in the infrastructure space.

Figure 1 - Tactical Map of Salesforce (click on image for higher resolution)

On closer inspection, Salesforce seems to be doing more than just commoditisation with an ILC pattern, as can be clearly seen from Radian's 6 acquisition. They also seem to be operating a tower and moat strategy, i.e. creating a tower of revenue (the service) around which is built a moat devoid of differential value with high barriers to entry. When their competitors finally wake up and realise that the future world of CRM is in this service space, they'll discover a new player dominating this space who has not only removed many of the opportunities to differentiate (e.g. social CRM, mobile CRM) but built a large ecosystem that creates high rates of new innovation. This should be a fairly fatal combination.

But, how is Salesforce able to get away with this? Why was it that Amazon and not a hosting company created this future world of infrastructure provision? The answer would appear to be … inertia.

The existing players in the CRM world are stifled by their very own success in the product world. This past success creates an inertia barrier to change. Whilst the causes of the inertia barrier can be traced back to the early stages of a companies formation, by the time a company is of a reasonable size it is often embedded in the organisation, culture and reinforced by external financial markets. Figure 2 provides an overview of this.

Figure 2 - Causes of Inertia (click on image for higher resolution)

So, back to the question about Microsoft. Whilst Microsoft is now making some strong moves into the service world, the problem for MSFT is this space is being even more aggressively commoditised through open plays. Whether it's VMware's necessary play into open source platform with CloudFoundry or the Openstack attempt to out-commoditise Amazon's out-commoditising of the existing industry.

Both efforts attempt to exploit the natural end state for ubiquitous and well defined IT activities i.e. good enough components provided through a marketplace of providers based upon common open source reference models. Both efforts will seek to create higher order revenue streams such as assurance, exchange, marketplaces and brokers alongside the normal business of being a service provider. 

Whilst MSFT has made much of a fanfare about its recent moves into the cloud, it was a probably a significant internal battle for MSFT just to make the change from products to services. However, this new world is likely to be rapidly commoditised to marketplaces based around open source and hence the real question becomes whether MSFT will be able to make the further change necessary to survive in that world?

Microsoft's future business should be intertwined with open source in the domain of utility services. Unfortunately, the last group of people who are usually willing to accept such a change are those who have built careers in the previous domain e.g. products. The bad news for Microsoft is that group probably includes a large chunk of its own organisation. Hence Microsoft itself is probably its own greatest threat to future survival.

Or as the great Bill Gates once noted:-

"Success is a lousy teacher."

That's one of those basic lessons which often gets forgotten in business. In this world of competition, there are two fronts to fight on. The external front includes those competitors who attempt to either gain a creative leadership position or to disrupt your existing model. The other front is internal and against your own past success.

Which is why in my latest research I've been looking into the web 2.0 world to see if we can't find techniques, strategies and methods for managing IT and the business more effectively in a continually changing world. Of course, I already know that there exists significant competitive advantage in organisational design and the application of cybernetic management as was demonstrated through my successes, failures and experimentation with Fotango during '01-'07. The real question for me is how widespread have equivalent practices become and who is pushing the envelope with culture, organisation, ecosystems and a complex adaptive approach?

-- Update 25 August 2013

Some two and bit years later, the New York Times has published an article on "Needed at Microsoft: A Catch-Up Artist".  The article talks about how “Microsoft does have a financial problem, and it’s been the fear of losing those massive profits from Windows and Office” i.e. inertia and goes on to propose they need a catch-up artist. 

When I wrote the above post in July 2011, these concepts and how to play them had been well established for many years (I personally had used the inertia of competitors as an advantage in Fotango in 2005 and Canonical in 2008).  Inertia is of course, highly problematic if a change is unpredictable (e.g. a change in value networks such as cable versus hydraulic excavators) and this will often lead to what is called "Disruptive Innovation".  However, corporate inertia is more than solvable if you are aware that a highly predictable and inevitable change is going to hit you. 

What I've subsequently discovered is that most companies don't seem to be aware of highly predictable changes and are often disrupted by things which shouldn't disrupt them. Yes, these changes get lumbered under the term "Disruptive Innovation" but in reality they are a class of highly predictable changes which were defendable against.  Cloud computing is an example of this.

Being disrupted by cloud (something which was predicted back in 1966) and was screaming loud in terms of weak signals in the early to mid 2000s is pretty shocking.  This is an issue beyond inertia (and denial) and it is better described as corporate blindness. 

From my experience, corporate blindness is fairly rife. The impact can be reduced through mapping of a landscape or other mechanisms to improve situational awareness especially when combined with some understanding of economic play.  Whilst I like the NYT article, in today's competitive landscape just being aware of inertia and that your own success inhibits your future survival isn't going to enable you to compete against some of the tough players out there.

Oh, and don't get me started on OpenStack.

-- Update 12th February 2015

Microsoft seems to be really turning the corner, they've increasingly adopted a more open route and seem to be overcoming inertia. This is fabulous to see. They've a top notch CEO in Satya Nadella.

Tim Cook has done a tremendous job with Apple, rebalancing it to a more ecosystem focused future. Truly exceptional CEO in my book and up there with Jeff Bezos.

Oh, and don't get me started on OpenStack. What a wasted opportunity.

Responsibility

I'm traveling on a research trip in the U.S. and whilst my topic is one of organisation and the impact of change, I cannot help but notice the current "confusion of responsibility" that appears to be occurring in the mainsteam media.

All business activities evolve and all organisations are in a constant competitive struggle. One of the impacts of this change is that previously successful models are often replaced, for example the current shift from software products to software services. Unfortunately, organisations who have built around the previous model often face internal barriers which create an internal inertia to making the change necessary for future survival. Without this inertia, external disruption would not occur and today's giants of industry would generally be tomorrow's giants of industry.

Disruption is simply a consequence of management failure, it is the responsibility of management to ensure the organisation not only survives and competes today but also tomorrow. However, management doesn't operate in a vacuum and the inertia is often exacerbated by the actions of the financial markets i.e. it becomes difficult to change a successful model even though that model has a limited lifespan precisely because the financial markets are short term and see only short term value.

The pressure that the financial markets create is immense for any large organisation and ultimately those financial markets share a heavy burden of responsibility for failures caused by the pressure they exert. However, the short term view of the markets is such that this is not seen as their responsibility and long term failure is solely put down to management.

Unfortunately, the financial markets are very adept at blaming others for what is ultimately their fault. The current economic crisis caused by irresponsible lending, a merry go-round of debt re-capitalisation and excessive exposure are all the responsibility of the financial markets. Whilst you can point fingers at consumers, that's like saying a toddler shouldn't have eaten the candy you gave it and blaming the government is like blaming a parent for not providing some rule to govern your irresponsible choice.

The financial markets are entirely responsible for the economic crisis. They're equally responsible for many company failures through short-term attitude. They're also responsible for acting in a dishonorable manner when it comes to the Fed Bank - through the exploitation of quantitative easing for financial gain to the using of Fed money to recapitalize rather than increase lending.

The U.S. Gov and Public should realise that the Financial markets act on a short term basis without consideration of the long term consequences. They act with the persona of an adolescent, always blaming others, threatening to run away, demanding less rules and pointing to a lack of rules as a reason for their own reckless behaviour. They discount the long term to a frightening degree.

Short of grounding the child (the fiscal equivalent of nationalisation), the Gov should understand that the financial markets are an offspring of society and despite the threats they have to operate within its rules. Those rules should and can be gamed to encourage more longer term responsible behaviour.

The financial markets won't magically learn responsibility, that behaviour has to be taught and encouraged. Left alone, they'll just get upto the same old tricks.

Open source as a tactical weapon, VMware's latest move.

All business activities evolve through a common lifecycle and we're currently witnessing a shift of many IT related activities from a product to a utility service world. This is commonly referred to as "the cloud". This transition brings benefits, risks, different methods of operating but also impacts the tactical plays in the great skirmish between companies. There are two models of tactical play which are particularly noteworthy - ILC and Tower & Moat.

In my previous LEF post, I discussed the innovate, leverage and commoditise (ILC) model that seems to be naturally appearing in companies such as Salesforce. To summarize, it is a technique by which a company uses a surrounding ecosystem to not only reduce the cost of innovation but to encourage innovation and rapidly identify success. By acquiring and providing such innovations as common services, a virtuous circle can be created.

A second model is the tower and moat. The principle here is to defend a revenue stream (the tower) by creating a moat devoid of differential value with high barriers to entry around it. By way of example, Salesforce created its own tower around provision of CRM as a utility service in a world where CRM was generally provided through customisable products or rental services. As barriers to entry into this new field were eroded (i.e. Amazon enabling widespread access to utility infrastructure) then new barriers were created through the acquisition of platform technology.

Whilst product based competitors attempt to differentiate themselves with activities such as social CRM, Salesforce acquired such activities with the view of providing common services. The net effect is this eliminates the differential value of social CRM and helps establish a moat. Salesforce has been extensively using its ecosystem (an ILC model) to identify and acquire a wide range of potential differentials and further strengthen its moat.

When competitors finally move to a cloud model then they will find the space inhabited by a large player with a large ecosystem and few opportunities to differentiate - a reasonably fatal combination. Both ILC and the Tower & Moat model are powerful tools which can also be used to counter competitors. They can be used together, or individually or combined with other tactical plays such as open source.

Take the case of Apple vs Android : whilst the iPhone is not one activity but a device describing many activities, Google has effectively created an ecosystem around Android which provides a means of identifying and accelerating innovation in this field whilst reducing costs. By providing the system as open source and creating a hardware ecosystem, then Android has effectively removed much of the differential value that Apple might have sort. Apple would appear to have been pushed into a high risk, stand alone innovation game against a broad ecosystem.

Take the case of cloud infrastructure : we've already seen Rackspace & NASA move to create open source software - the OpenStack project - to provide infrastructure as a service. Their vision is to create a competitive marketplace of computer utilities around OpenStack. Such a world plays to Rackspace's strength of service delivery as a utility provider but also fits with NASA's goals of increasing efficiency of infrastructure. The ecosystem around openstack should encourage rapid innovation and if successful will create the standard that a competitive marketplace depends upon. It will also drive out differential value in this space making it tough for new competitors or those with a proprietary offering. I say 'should' and 'if' because I have real concerns over the differentiation from Amazon idea.

Take the case of large scale infrastructure: into which Facebook has announced the OpenCompute project and in effect open sourced how to create large scale data centres. This should over time help eliminate differential value that such knowledge created and whilst beneficial to the future computer utility world it will also help to undermine those for whom such skills have acted as a barrier to entry into their industry - namely massive scale search engines and data processors.

Take the case of healthcare: which has seen the VA (Veterans' Association) create an open source electronic health record system from VistA. It seems clear that the VA are focused on encouraging innovation through ecosystem effects and creating a marketplace of competitive providers. Visions of a worldwide standard are not beyond the realm of reason.

Take the case of platform as service into which VMware has announced an integrated set of open source platform components known as CloudFoundry. If successful and there's every reason to believe it will be then VMware will succeed in creating a huge moat devoid of differential value in the platform space and a vast ecosystem driving this. Any would be competitors will face an uphill struggle to compete against VMware's effort. Those planning proprietary platform offerings should take note of this move.

But wait ... where's the tower?

The beauty of creating a competitive marketplace of utility service providers is that it opens up a huge range of opportunities from service provider, support, assurance, brokerage, exchange, marketplace and a dozen more. Being at the heart of this, which is where VMware will be, means they are well positioned to take advantage. It's a bold move, perfectly timed and well executed.

Of course, CloudFoundry has already been made to run on Amazon EC2 which means CloudFoundry on OpenStack built on an environment designed around OpenCompute can't be far behind.

The world of IT is changing and many IT activities have become suitable for provision through utility services. With this change comes tactical plays designed to take advantage of this shift. At the OSCON conference in July 2007, I stated that in this future utility world open source was the only way of effectively competing. Time will tell but the increasing drive towards open source and its use by major companies as a tactical weapon seems to be pointing that way.

Smart move by VMware, it'll certainly shake up the industry.

--- Update 5th May 2014

Most is proceeding as expected. Bizarrely SAP / Oracle just seem to be waking upto the threats ... a bit too late. Unfortunately also OpenStack continued its differentiation play and the market never formed. Cloud Foundry however is storming ahead. Apple is starting to look weak vesus Android whilst OpenCompute gathers momentum.

A question of standards.

I'm all in favour of standards for the "cloud" world because such standards should enable and accelerate innovation in IT through componentisation effects as well as encourage the formation of competitive markets of compute utilities

However, that said, I'm against standards committees and the concept that open standards (as in APIs & data formats) are enough to create the portability required for competitive markets.

In the former case, the market will decide the standard and the job of any standards body should be to rubber stamp what is an existing practice. Unfortunately, standards committees are often used as vehicles to promote specific vendor interests and in many cases their efforts are counter productive. For example, whilst IPX/SPX was the committee approved standard, it was TCP/IP which won the marketplace battle. The only impact that approving IPX/SPX as a standard had was to temporarily slow adoption of TCP/IP in some quarters and hence inhibit innovation.

Standards should have a positive effect but defacto has to precede dejeure. There are some exceptions to this but they are exceptions.

In the case of open standards, a competitive marketplace requires multiple providers, access to code and data (ideally with syntactic interoperability) and semantic interoperability of services. Whilst open standards provide part of the solution, it is critical for reasons of semantic interoperability that a common reference model (i.e. running code) is provided.

Since, we're talking about a world of service (and not feature) differentiation for activities which are fundamentally commodity by nature and hence suitable for utility service provision, then the obvious solution is an open source reference model as the standard. Potential examples of such would be the OpenStack effort.

Such open source reference models would ideally exploit the dual nature of GPLv3 which is both restrictive in the product world but simultaneously permissive in the service world to create a functioning marketplace. Unfortunately, many vendors promote open standards (as in APIs etc) as the solution to these problems of portability and hence describe their systems as open when they're quite clearly not.

So, in general :-

• Are standards good for the cloud?
  Absolutely, it'll encourage innovation through componentisation effects.


• Are standard committees good for the cloud?
  Generally no. At best they should rubber stamp market chosen approaches however in reality they're more likely to get in the way or slow progress by promoting vested interests.


• Is portability between providers important for the Cloud?
  Absolutely, it's the route to formation of competitive marketplaces and reducing outsourcing risks.


• Will open standards provide the portability needed?
  Only in the most trivial cases but not for the vast majority of activities. Open standards are necessary but they are not sufficient to provide the portability required. The idea that open standards alone will achieve this will inhibit the formation of competitive markets.


• Is open source essential for the cloud?
  Absolutely, the formation of competitive markets without loss of strategic control to a specific vendor depends upon the provision of open source reference models as the standard.

Preparing for war

Following on from my post on ecosystem wars, I thought I'd discuss some techniques and tactics to use. However, before I do, I need to first describe the organisation that I hope you work for. If this doesn't describe you and your organisation then the techniques and tactics probably won't make sense and I can save you the trouble of reading the next post.

Once again, my apologies to regular followers or attendees of my presentations, this post is just scene setting and so it runs the danger of teaching grandma to suck eggs.

I'm going to assume you're battle hardened, you realise that business is all about warfare and you've got a keen eye for competition. You've probably done a stint in the typical organisation, scratched your head over some of the practices and ultimately found yourself in a new sort of company (possibly created by you).

You understand what an organisation is. Rather than making the mistakes of the past and simply grouping people, activities and methodologies into generic structures such as IT, Finance, Marketing - you've taken the time to look. You may well have profiled your organisation (see figure 1) and identified those activities you consume, those activities you sell and those activities which act as barriers to entry into your industry.

Figure 1 - Profile (click on image for higher resolution)

You know the importance of ecosystem and how this can both accelerate innovation and reduce your risks. You might be using a model akin to ILC (innovate, leverage and commoditise) to grow and keep a vibrant ecosystem around your products and services (see figure 2).

Figure 2 - ILC model (click on image for higher resolution)

You know full well that methodologies have to change with lifecycle because characteristics do and how activities are interconnected (see figure 3 & 4). One size never fits all isn't effective hence you don't engage in the typical debates of this vs that, agile vs six sigma, push vs pull - you know you need both.

Figure 3 - How Characteristics Change (click on image for higher resolution)

Figure 4 - How Methodologies Change (click on image for higher resolution)

You also recognise that not everyone is the same, you have different types of people :-

• Your pioneers are feverishly imaginative, often chaotic in nature, constantly exploring and they create the crazy stuff. They work on a cadence of weeks, maybe months and they fail often and miserably but they're never afraid to experiment. They want to push things out there, they're infectious, they can even be hazardous. They're not a "safe" pair of hands but you don't want them to be. Every now and then they create your future source of competitive advantage, though you don't know it until it starts to be adopted by the wider ecosystem. They are your Da Vinci's.


• Your town planners build the core of your company and the components which your pioneers develop upon. They build the solid, useful and beautiful and they're obsessed with getting things better, faster, more efficient and more reliable. They're meticulous, methodological and geniuses who hide worlds of complexity behind standard interfaces. They're ultra reliable, your "safe" hands and they work on a cadence of months, even years. They provide the operational efficiency which you can use to bury your competitors. They are your rock, they are your Vitruvius'


• In between, you've got your settlers. They watch the landscape, your competitors and spot the patterns which are going to either disrupt an existing revenue stream or reduce some barrier to entry. They constantly take innovations away from the pioneers and force them to move onto the next thing. They'll drive that innovation into the wider ecosystem, spin it out or fail it fast where necessary. They adapt to the changing environment and adopt what's growing. They've always got their eye on pushing a growing trend towards the town planners. They listen to the ecosystem and whilst they don't create the future or build the cities, they play the games of tactical warfare which are so essential to survival. They are ruthless to your competitors, nurturing to your own ecosystem and you're just glad they work with you. They are your Machiavelli's.

You probably identify yourself with one group and you could probably have read down the list of your employees naming which group they belong to. You didn't have to, they told you.

You've long given up on trying to create departments with a mix of Da Vinci's, Machiavelli's and Vitruvius' and listening to the constant arguments within those departments about how to best run IT or whatever they're assigned to do. You understand they won't agree, they never will.

You understand that outsourcing a department may get rid of the arguments but you'll end up removing sources of competitive advantage. It's better to let the town planners outsource those activities that can be removed.

You've given up making speeches about the need for innovation or efficiency, you know you need both. You've also realised that you need to nurture both pioneers and town planners to do this and you have to let them use the tools they need to get the job done. You understand those tools are different. One size fits all seems a long distant and best forgotten memory.

Hence you've probably decided to structure yourself around change and have groups like pioneers, settlers and town planners - though you'll call them something else. You've probably discovered the alignment issues between your old groups was an artificial construct of how you organised yourself. You've probably also realised the importance of the settlers in managing change and wondered why you didn't do this before.

You've almost certainly realised you want the best Da Vinci's, Machiavelli's and Vitruvius' to compete and hence focus almost religiously on high levels of talent and acquiring the best, no matter where they are in the world. You remove any and all unnecessary vestiges of the old world - expenses, timesheets, standard company laptops and so forth and instead have instilled a culture of respect.

You're going into a war. You want your people fighting and you want them to have the tools they need. if you can't trust your people to fight on your side then you don't want them. Your management reflects this, your organisation reflects this, your people reflect this. It feels vastly different from the old ways you remember.

If this is you, then the next post on tactics will make sense and we can go on to explore some of the actions being taking by the big players in our ecosystems.

Ecosystem wars

Back in 2005, I started to talk publicly about lifecycle concepts and the importance of ecosystems. In the next two posts, I'd like to draw a line in the sand and cover those basics of ecosystem one last final time.

I'll take the liberty of assuming notions such as lifecycle, organisational profile, componentisation and consumerization are universally understood and start with an explication of what a business is.

What is a Business?

A business is a living thing, comprising a network of people, a mass of different activities, and reserves of capital including financial, physical, human and social. It consumes, it produces, it grows and it dies. Like all organisms, any business exists within a number of ecosystems in which it competes and co-operates with others; it’s shaped by and shapes its environment, and hence needs to adapt constantly merely to survive.

People come and go, activities change, and hence all firms are in a constant state of flux. In any industrial ecosystem, new activities (innovations) are a consequence of competition and those that are useful will diffuse throughout the ecosystem becoming more of a commodity. This constant change creates a paradox, identified by Salaman and Storey:

”Survival requires efficient exploration of current competencies and ‘coherence, coordination and stability’; whereas innovation requires discovery and development of new competencies and this requires the loosening and replacement of these erstwhile virtues.”

These two extremes of survival (today and tomorrow) have diametrically opposite concerns, and the techniques, tactics and methods needed to manage each are entirely different. Those who manage organizations are therefore caught on the twin horns of a dilemma: how is it possible to be standardized and efficient as well as innovative and new, without prejudicing your survival - either today or tomorrow?

The effects of this on business can be seen in the constant restructuring to cope with new paradigms, and in the yo-yoing of popular management theories between opposites in a scramble to maintain order. A more effective balance can be found through embracing both goals simultaneously.

How do we balance both goals?

This requires a rethinking of how we organize, and a realization that what really matters is not innovation or efficiency per se, but how we continuously manage the path between the two. To explain why this is the case, let us examine a typical profile of an organisation (see figure 1) and consider how we build systems for the future.

Figure 1 - Using profile to build systems (click on image for higher resolution)

First, let us take those cost of doing business activities which act as underlying components of other business activities such as payroll, computer infrastructure, authentication etc. All of these activities are linear, well defined, ubiquitous and suitable for provision as standard re-usable components through utility services. Such an approach will increase our agility, rate of innovation (componentisation effects) and reduce the cost of gambling for any innovative activities built upon these utility services.

However, it is important to understand that all innovations (i.e. those activities which are uncertain and rare) are a gamble and whilst we can reduce costs we can never eliminate it. The future value of something is inversely proportional to the certainty we have over it, we cannot avoid this information barrier any more than we can reliably predict the future. However, there is a means to maximise our advantage.

By making these utility services accessible through APIs, we not only benefit ourselves but we can open up these components to a wider ecosystem. If we can encourage innovation in that wider ecosystem then we do not incur the cost of gambling & failure for those new activities. Unfortunately, we do not enjoy the rewards of their success either

.

Fortunately, the ecosystem provides an early warning mechanism of success i.e. adoption. Hence by creating a large enough ecosystem, we can not only encourage a rapid rate of innovation but also leverage that ecosystem to identify success and then either copy (a weak ecosystem approach) or acquire (a strong ecosystem approach) that activity. This is how we maximise our advantage.

To capitalise on this, we simply drive our newly acquired activity towards utility service provision and create the next wave of innovation through further componentisation effects. In this manner we create a virtuous circle of encouraging innovation in the ecosystem, leveraging the ecosystem to identify the next pattern and commoditising the pattern to utility services in order to encourage the next wave.

In effect, by being at the heart of this ecosystem we manage to create the simultaneous appearance of being highly innovative (as others in the ecosystem do this for us), highly customer focused (by leveraging the ecosystem to identify rapid diffusion) and highly efficient (as we focus on commodity provision). Tom Peter's old adage of choose one is a busted flush in this brave new world.

I've summarised these concepts in figure 2

Figure 2 - ILC model (click on image for higher resolution)

What is critically important to note, is that in essence both innovation of new activities and efficiency of commodity provision can be outsourced. The innovation of activities can be outsourced to a surrounding ecosystem building upon our services, whilst those commodity activities we consume can be outsourced to a marketplace of utility providers. There are benefits to retaining some element of control in those areas but the critical area to focus on is the transitional phase and how you leverage the ecosystem. Our ability to exploit the link between innovation and commodity depends upon the size, composition, engagement, speed of information and overall level of activity within that ecosystem.

Whilst each company comes with its own personal ecosystem (its staff, its partners) growing an extended ecosystem and using that to manage change is a powerful tool and a competitive weapon. It's not businesses but ecosystems that collide in the commercial world and woe betide that organisation which has the weaker. We've seen this story repeated many times before at many different levels through all the usual examples of BetaMax vs VHS or TCP/IP vs IPX/SPX.

It's imperative to understand that a well formed and large ecosystem leads to lower costs of innovation, creates higher rates of innovation, improves the rates of successful adoption, encourages greater efficiencies and can be mined to detect future successes. However, there are cost to this especially in the form of data collection which is why utility based ecosystems (where information is derived from consumption of APIs) have an advantage over product based ecosystems (where information is derived from market research). 

If it is just you and your employees then you'll find it difficult to survive a direct onslaught from any of the modern day monsters with well developed and extended API driven ecosystems (such as Google, Amazon etc). Not impossible, just damn difficult.

For more details, my OSCON '10 talk provides a useful overview of these concepts. However this is enough of a base to begin with.

In the next few posts, I'll cover some of the consequences of this and the strategies which are commonly deployed. Again, I realise this is nothing new and apologies to all readers who've had to endure me harping on about this over the last five years but I'm just scene setting for much later posts.

Consumerization

The evolution of any business activity is complicated by the existence of multiple ecosystems such as the enterprise and consumers. Activities can exist in, transfer between and commoditise within any of these ecosystem.

For example, whilst CRM has mainly commoditised in the enterprise space, email has crossed over to the consumer space in which it has been more aggressively commoditised. Today, these heavily commoditised mail services (such as Google Mail) create adoption pressure for the enterprise. These concepts are summarised in the figure below.

Figure 1 - Consumerization & Lifecycle Across Ecosystems (click on image for higher resolution)

When discussing an activity, a key issue to consider is in which ecosystem the evolution of that activity is governed i.e. is it being driven by competition in the enterprise or the consumer space. Back in 2001, Douglas Neal and John Taylor described the process by which governance transfers from the enterprise to the consumer ecosystem as Consumerization.

The consumerization of technology and its subsequent commoditisation in that consumer ecosystem can have a profound effect on the operation of any enterprise using that technology. It should be noted:

• Not everything transfers. Some activities find meaningful use within only one ecosystem and hence tend to develop only in that environment. For example, CRM.


• Barriers to entry can erode quickly. The impact on the media industry of the commoditisation of the means of mass communication occurred primarily in the consumer space over a single decade. Originally, the principal means of mass communication was provided through printing presses and supply & distribution chains. These acted as a major barrier to entry into those industries. The internet and digitization of content brought new means of mass communication into the consumer ecosystem and subsequently were aggressively commoditised resulting in widespread use of blogs, wiki and social media tools such as twitter. A consequence of this is that the barriers to entry into the media industry have been completely eroded. Anyone can become a digital newspaper.


• Ecosystems are not isolated. People are part of multiple ecosystems, hence activities in the enterprise (which may be provided through products) tend to start with custom-built solutions and hobbyist equivalents in the consumer space.


• The rate of commoditisation is not uniform between ecosystems.

Numerous activities have undergone this change. The pressure for adoption created through consumerisation has created positive responses such as BYOT (bring your own technology) and negative responses such as shadow IT i.e. when an organisation fails to deliver what its internal consumer need (Joe Baguley gave an excellent presentation on this topic).

All organizations should, as a matter of course, examine both their own and the consumer space for the growth of highly commoditised activities and consider the impacts of these and whether barriers to entry into their own industry will reduce.

For point of reference, computer utilities (i.e.Cloud computing) is expected to reduce the barriers of entry into many industries. In such cases an approach of adoption of the service and creation of higher-order systems can be used to re-establish those barriers.

Equally the impact of consumerization on the means of manufacture is going to hit a wide range of industries. This video is a foretaste of what is going to happen.

Building a private cloud

As we move from a world of infrastructure products to a competitive market of infrastructure services, private clouds have a valid role as part of a hybrid strategy in mitigating transitional risks (such as concerns over data governance, trust, transparency and security of supply).

The key word here is transitional i.e. this is not a permanent solution. Furthermore by using a hybrid strategy you're deliberately sacrificing some element of the benefits of public provision (economies of scale, focus etc) in order to mitigate those risks. It's a trade-off.

However, for this post I'll assume you've decided to make that trade-off. Hence you're going to be using a combination of public and private resources in a hybrid form for the immediate future. So dealing specifically with infrastructure, at what price should you be building a private cloud?

First, you need to recognise that a consequence of a shift to standardised components provided through utility services will be an explosion of innovation in higher order systems - i.e. the competitive landscape is going to become even more competitive. Second, cost efficiency won't result in reduced IT budgets because increased competition, the long tail of unmet demand and co-evolution effects will result in far greater consumption. Lastly, one of the key benefits of cloud is speed and when this is combined with cost efficiency, you'll end up doing more, faster with the same budget.

It's worth noting that there is little correlation between total IT spending and business value but there is strong correlation between speed and business value. How efficient your provision is in terms of speed is critical here. Now assuming you're not planning something daft like a long winded requisition process for new virtual machines, then the speed of provisioning will be minutes whether public or private. Assuming this is uniform, the distinguishing feature becomes cost efficiency - i.e. how much MORE you can do with same level of spending over competitors.

With public provision such as Amazon EC2, a small instance (I'll refer to this as a VM) costs currently $740 per year. Now whilst Amazon doesn't release figures, we believe the cost of provisioning such a VM can be much lower, possibly as low as $220 per VM per year though from the comments possibly much lower. Hence if your private cloud is being designed to cost $1500 per VM per year, you might be forgiven for thinking that this is only twice Amazon pricing and represent good value for a hybrid trade-off. What you're probably discounting is that your private environment will have to exist for three to five (in a worst case) years and during that time Amazon's pricing could fall rapidly, possibly to the order of $300 per VM per year, possibly lower.

In such circumstances, a competitor with a pure public play would have upto a 5x cost advantage (depending upon how much of your hybrid strategy was private provision) which is an astronomical difference in a world of increasing and more rapid competition. But that's a key point to note, the impact depends upon the type of industry you exist within and your competitor actions.

Assuming you're working in a high tech reliant industry, then in order to give your private cloud a fighting chance, you have to be designing for a target price of between $220-$540 per VM per year.

These figures (and lower) are achievable if done at reasonable scale and with a relentless focus on commodity provision. For reference purposes only, I've provided a hypothetical example (based upon real data from real examples) of a VM farm. I've used the following assumptions:

1. Base VM: 1.7 Gb RAM, 160 Gb HD, CPU equivalent to a passmark of 420
2. Target Price (VM per year): $510
3. Max Utilisation (to determine level of over-provisioning required): 70%
4. No. of VMs: 100,000
5. Base Unit: Rack
6. Focus: Commodity (No multiple PSU, Hot Swap)
7. Depreciation for hardware: 3 yrs
8. Current High Interest Rates of: 3%
9. Networks: All internal equipment and network upto external routers but not external bandwidth.
10. PUE:1.3

Given this, then figure 1 shows as a rough guide, how your target price per VM per year breaks down into its various components.

Figure 1 - Cost per VM per year.
(click on image for larger size)

Now obviously there are many ways to skin a cat, and pricing is sensitive to various factors such as scale, density of VMs, power consumption, number of units per rack, location etc etc. However, I put this here to start a discussion as I would like to know what's your cost per VM in your private cloud?

-- 12 Feb 2015

It's 2015 and companies are still commissioning / building private clouds as new projects! Ouch. There's going to be some very costly white elephants.

Componentisation

Herbert Simons showed in the Theory of Hierarchy how the evolution of a system is dependent upon the organisation of its subsystems. In short, as an activity becomes commoditised and provided as ever more standardised components, it not only allows for increasing speed of implementation but also rapid change, diversity and agility with systems that consume the activity as a subsystem.

In other words, the creation of standard sizes of bricks, pipes, utility services and other architectural building blocks led to a faster rate of house building and a wider diversity of housing shapes. It's the same with electronics and every other field you care to look at.

Commoditisation to standard components leads to an explosion of innovation for higher order systems.

The cloud is no different. It's the creation of good enough, standard components which will cause a wide diversity of higher order systems. These will in turn undergo their own evolution to more of a commodity. Without this process we would never have got to a CPU, let alone cloud or the hugely complex systems that will develop from it.

This also doesn't mean that innovation stops with the standard components. Whether it's brick making or electricity provision, there is a huge amount of operational innovation hidden behind the "standard" however the "standard" acts as an abstraction layer to this. Just because my electricity supplier has introduced new sources of power generation (wind turbine, geothermal etc) doesn't mean I wake up one morning to find that we're moving from 240V 50Hz to something else. If that constant operational innovation was not abstracted behind the standard then all the consumer electronics built upon this would need to continuously change - the entire system would collapse in a mess.

Whilst cloud computing is all about commodity provision of computing resources through utility services, these components act as the abstraction layer. We're not quite there yet, we're still in transition from a product to a utility service world. However, even in that utility service world, providers will still innovate in terms of operations but to a consumer that will be abstracted and ultimately reflected in better price and quality of service. To the consumer this will be seen as operational efficiency and the activity itself for all sense of purpose will remain a commodity, just like electricity and every other industry that has undergone this change. Of course, I'll do more things with it, create new and ever more complex higher order systems but my innovation is based upon the activity being provided as a commodity.

I can understand how many people don't relish the prospect that some activities within IT are becoming more of a commodity. They may not realise the benefits this will create in terms of progress but at the same time as they cry this isn't happening they happily consume systems which depend upon this process.

If you really don't like the idea of commoditisation and componentisation then try getting by without using anything which has depended upon it. Try building a computer without a CPU, electronics, electrical power supply or even nuts and bolts.