This is the age of disruption ... oh, give it a rest.

What we can demonstrate

1) Companies consists of many value chains comprising of components (activities, practices and data) that evolve due to competition.

2) As those components evolve their properties changes from uncharted to industrialised

3) As those components evolve we develop inertia to change.

4) The interplay of inertia and competition creates three economic states for any component - peace, war and wonder.

5) When those components have a broad effect (i.e. are part of many value chains) then those changes are seen at a macro economic scale. We call these ages or more appropriately Kondratiev waves.

6) Every age begins with commoditisation of a pre-existing act, disruption of past industries stuck behind inertia barriers (the causalities of war), co-evolution of practice (leading to new forms of organisation), an explosion in higher order and novel systems (the wonder),  reduction of barriers to entry and rapid increases in unmodelled data.

7) After a period of re-organisation which occurs during the war and wonder stages then the effected industries settles down and the novel activities created mature. Past practices and companies unable to adapt die off.

This is how, back in 2005, we knew that cloud computing (commoditisation of a range of IT acts) would lead to explosion in higher order systems, co-evolution of practice (devops), rapid increases in unmodelled data (Big Data), new forms of organisation (next generation), reduction of barriers to entry and disruption of past industries unable to adapt.

This is nothing more than a repetition of every cycle that we've been through countless times before ... and guess what, commoditisation of manufacturing through 3D printing will have exactly the same effects. Figure 1 gives a simplified overview of that repeating cycle.

Figure 1 - A repeating cycle (2007-2008 research)

Now, sometimes a means for communication becomes more of a commodity (postage stamp, phone, internet etc) and as a result the entire process of evolution speeds up. It's not that we've become more innovative but instead the speed at which something genuinely novel becomes a commodity has accelerated.

Ok, so what's wrong with the 'Age of Disruption'.

For starters it's not an age. It's nothing more than an phase of an economic cycle. If you really want to call it the 'Age of Disruption' then you should call it the 'Age of Disruption v6.0' because it's about the 6th major one we've been through in the last 300 years. Actually, we've been through several hundred more smaller and localised ones, so in construction you could probably call it the 'Age of Disruption v57.0' - it's a really poor title.

It's also worth remembering that the reason why companies are being disrupted is not because of some unexpected change. This form of disruption is highly predictable and preventable. Alas, those companies being disrupted on the most part share one common characteristics - very poor strategic play.

In figure 2 (from a project a few years back), I examined the level of strategic play versus willingness to use openness as a competitive tool among over 100+ different companies. The thing worth noting are those companies showing high level of strategic play (in particular the Players) had high market cap growth over the last seven years. Those which showed low levels of strategic play (in particular the Chancers) have low levels of market cap growth, often negative and with some going bankrupt.


Figure 2 - Market Cap, Strategic Play (2011-2012 research)

Notes on graph

1) Each quadrant is given a label - thinkers (high strategic play, low levels of openness used as a mean to compete), players (high strategic, high open), believers (low strategic, high open) and chancers (low strategic, low open)

2) I've marked two groups in grey, the top group shows higher market cap growth and the bottom group shows lower levels of market cap growth. The players were strongest and the chancers were weakest.

3) Size of bubbles equals volume of companies.

Whilst culture is important all the time, Strategic gameplay appears more important in specific states of the economic cycle. IT is in one of those states when it matters, the state of war.

However, we're already entering the state of wonder, those past players will be cleared out and a new dominant form of organisation (as per Fordism in Electricity Age, American System in Mechanical Age) will emerge. This new form, which we call the next generation have the following characteristics (see figure 3).

Figure 3 - The Next Generation (2010-2011 research)

The first problem with the 'Age of Disruption' is it implies something new. It isn't. It's just a repetition of what has happened before governed by exactly the same mechanics (competition, evolution and inertia) with the same results. So, please take the effort to make it clear that this is not unusual.

The second problem is that this type of disruption is predictable. When we talk of disruption as in product vs product substitution then that's hard to predict and the interaction of culture and inertia becomes critical for survival for a company. In this case, because the change is predictable then the issue of culture and inertia are solvable well in advance and strategic gameplay is what matters most. 

The fundamental reason why companies WILL be disrupted by this predictable change is not because of culture and inertia (i.e. we need to be more like a Silicon Valley company won't help you) but instead because you've got a bad case of poor strategic play. 

So, if you wan't to call it an age (which it really isn't) then a more accurate title might be the 'Age with quite a number of out of their depth CEOs driving companies needlessly to disruption'. Now, obviously this isn't going to be a popular title and so I'm sure the 'Age of Disruption' will stick and numerous other factors (culture, inertia, unexpected change, customers) will be blamed as the principle cause of failure.

However, just remember that this 'Age of Disruption' is a repeating phenomenon throughout history and don't confuse it with the type of unpredictable product vs product disruption (such as the different format hard drives that Christensen talked about).

Oh, and what follows next? An age of wonder, followed by an age of peace, followed by an age of war (aka age of disruption), followed by an age of wonder, followed by .... and on, and on, and on.

A pet favourite - Inertia

Organisations consist of a mass of evolving activities, practices and data. As those components evolve from uncharted to industrialised then their properties change. It's that change of properties which means that one size fits all methodologies don't work despite organisations endlessly pursuing simple techniques (agile vs six sigma, insource vs outsource, push vs pull, network vs hierarchical, Hayek vs Keynes etc).

Figure 1 provides a simple view of organisation demonstrating a set of components at different states of evolution, for more on mapping read here or watch the detailed video on the LEF site. Figure 2 provides a list of common characteristics.

Figure 1 - An organisational map

Figure 2 - Changing properties

Now maps are not only useful for increasing situational awareness and an understanding of how things should be managed but they can also be used for various forms of scenario planning, strategic gameplay and economic learning.

For example, by using mapping then it can be quickly discovered that as components evolve they also go through different economic states - one of peace, one of war and one of wonder.  Depending upon how widespread the components are in other value chains, then sometimes these states manifest themselves as macro economic waves (known as Kondratiev waves) but in the majority their effect is usually much more localised (e.g. a specific industry). More details on this can be found here.

The different economic states are important because in the state of peace we experience that sustaining change tends to exceed disruptive change whilst in the state of war then disruptive change tends to exceed sustaining. The states also have different levels of predictability, for example the war state is highly predictable in terms of what is going to be happen but just not when (it depends upon actors actions). This means you can be prepare for the war state many years in advance but alas we often find companies being disrupted by highly predictable and ultimately defensible change. Cloud computing is an example of this.

You can also very roughly characterise the different states with wonder being breakthrough, peace being incremental and war being disruptive. Naturally our response to this depends upon the components in our value chain, how evolved they are, competitors actions and the economic state. This is all part of gameplay.

Unfortunately, if you can't see the map then business is like playing a game of chess without seeing the board - every action is haphazard - sometimes outsourcing works, sometimes it doesn't. It doesn't have to be like this.

The economic states are also useful in predicting how organisations will evolve,  It's no coincidence that every major age starts with commoditisation of a pre-existing act, that commoditisation results in a state of war and co-evolution of practice and those companies that survive have a different set of characteristics. The Electricity Age didn't start with the Parthian battery but with Tesla and Westinghouse and it led to Fordism. The Internet Age gave us Web 2.0. The Mechanical Age gave us the American system. Cloud has given us the Next Generation.

With experience of mapping, it turns out that many aspects (not all) of change are surprisingly predictable, defendable against and manageable.  Even those areas of high uncertainty such as the genesis of the novel and new can be managed by exploiting others (see ecosystems).  Of course, for those who don't look at the chessboard then everything seems highly confusing, random and difficult to understand.

In such circumstances, terms like ecosystem are completely misunderstood, strategy is often a mess of operational, tactical, purchasing and implementation details with little or no 'why'. Managers grab onto truisms like 'culture eats strategy for breakfast' or 'be a fast follower' or simply grasp the latest fad - agile vs six sigma,  'Open by default' without any understanding. These Chancers only survive because competitors are equally as blind. 

These Chancers also tend to get caught out by inertia to change. Now inertia is critical to gameplay and can be used to set up an environment where the competitor's biggest threat becomes itself.  The use of inertia is one of my favourite specialities. It's far beyond the more common tactical plays (tower and moats, ecosystems, alliances etc), the dark arts (manipulation of constraints, misdirection etc), altering competition (the use of open as a weapon, changing buyer vs supplier relationships, lowering barriers to entry) and the use of effective management. There's a special type of wicked delight that comes from setting in play a situation where a competitor self implodes.

However inertia is much more than just a tool, it also a key part of governing economic states. The states occur due to an interplay of competition (supply and demand) which drives evolution combined with the build up of inertia due to existing models, relationships, business and practice. Understanding when and how to use inertia is essential to the finer points of gameplay. Of course there are many forms of inertia.

In figure 3, I've characterised many of the main forms of inertia. All of them (with practice) can be exploited and certainly are things which an organisation should watch for.

Figure 3 - Types of Inertia.

Something for the future II

A list of companies in two groups. I'm putting this here in order to return to the lists in 2020.

It's a prediction test (useful for me, hence I'm putting it somewhere public) but probably not useful for anyone else. There is no obvious relevance to the order (it's actually two different scoring systems).

This is simply two lists.

[System 1.0 No Change]

Group 1

1. Amazon
2. Google
3. Samsung
4. BP
5. Baidu
6. China Telecom
7. EMC
8. Lloyds Bank
9. ARM
10. NetFlix
11. Ebay
12. Yahoo
13. Intel
14. Facebook
15. BAE Systems
16. Lenovo
17. Salesforce
18. Time Warner
19. Huawei
20. Canonical
21. Citrix
22. Fastly
23. Bromium
24. Opscode
25. Juniper Networks

[System 2.0 Moderate Change]

Group 2

1. DuPont
2. GSK
3. Microsoft
4. WalMart
5. Berkshire Hathaway
6. Goldman Sachs
7. Barclays Bank
8. Walt Disney
9. Cable and Wireless
10. Canon
11. Dell
12. SAP
13. IBM
14. Twitter
15. Cisco
16. Oracle
17. CouchBase
18. HP
19. Puppet Labs
20. RedHat
21. Apple
22. PistonCloud
23. Rackspace
24. Nokia
25. Zynga

Openstack ... this is the year! Again!

Every year, I hear people tell me that this is the year that OpenStack will make it big and challenge Amazon.

Every year, I repeat the same response.

If OpenStack is ever going to be big, it's going to need to create a competitive public market. It needs a player (or players) with a brutal commodity focus and smart gameplay to make a massive investment (today, probably around the order of $5 - $10 billion+) in order to create public clouds that are AWS clones. It's going to need those players to lead the project and go head to head with Amazon.

It doesn't need a company trying to enforce its own API regardless of the market as part of an on-ramp to its own services.

It doesn't need a primary focus on private cloud, a transitional space which will come under increasing pressure over the years. It must look at the public market i.e. AWS and GAE.

It doesn't need endless discussions and a collective prisoner's dilemma differentiating with each other.

It's doesn't need a free for all and a lack of strong leadership.

As more time passes and without that player(s) emerging then despite all the noise about OpenStack, my view remains the same. 

Context, Situation, Components, PaaS, Dead or Alive … it's all semantics isn't it?

tl;dr Caveat Emptor

Figure 1 – HS2

Figure 1 provides a visual view of an IT system related to the HS2 project.  The map is created by taking a value chain of components required to meet some user need and then plotting those components against how evolved they are. This is not a permanent view but a snapshot in time because the many components that make up the system are evolving due to competition (both supply and demand side).  

The components themselves include activities, practices and data and as each component evolves it moves from one defined set of characteristics (known as uncharted) to another (known as industrialised). Since the characteristics of the component change with evolution then how you treat a component depends upon how evolved it is.

On Situational Awareness

Situations or events occur in a context i.e. there are surrounding facts pertaining to the actual event. Being aware that an event has context is known as contextual awareness. Being aware of what that context was is known as situational awareness.

Knowing that the state of evolution impacts characteristics and hence changes how a component should be treated is known as contextual awareness i.e. we know that the context, the state of evolution, has an influence. Knowing that a component like Infrastructure is in a commodity stage of evolution is situational awareness i.e. we know what the context is and how it should be treated.

Contextual and Situational awareness are not the same.

On Composability

Each component is normally part of one or more value chains.  The value chain described by the map is therefore composed of many components and we describe it as being composable.

However each value chain is normally a component of one or more other value chains i.e. the output of one (e.g. brick manufacture) is normally a part of another (e.g. housebuilding). Hence the entire value chain may in fact be part of a larger composable system.

Furthermore the components of the map may indeed represent their own value chains.  Hence when you look at the map in figure 1, the component ‘Web Site’ is in fact likely to be an entire value chain consisting of many components (from content, data to web farm).

On Componentisation

From figure 1, at the top of the value chain is the user need that we are attempting to provide. At the bottom of the value chain are the myriad of sub components that are consumed to enable this.  There is a link between evolution and value chain.  It is known as componentisation (from Herbert Simon's Theory of Hierarchy). 

As components evolve to provide ever more mature and standard components then they enable the rapid development of higher order systems i.e. standard nuts and bolts enabled machines, standard building materials enabled housebuilding, standard electricity provision enabled consumer goods.

Syntax and Semantics within a composable system.

The components of a system need to interact (i.e. communicate) with each other. There are two important forms of this interaction – semantic and syntactic.

Take a simple tap. The tap has certain physical properties such as size and weight and an interface for use i.e. an angular force. We apply a clockwise force to the tap and it turns (we hope). The syntax refers to the interfaces i.e. the method by which we communicate. In this case the message between one component (such as ourselves or some controlling machine) and the tap is through the application of angular force.

Semantics refers to the understanding of meaning. For example, I apply a clockwise force because I wish the tap to turn off.  My meaning is ‘turn off’, the method of communication is ‘clockwise force’. Whether the tap actually turns off or on depends upon how it’s designed and the screw thread. So it’s quite possible that I might mean ‘turn off’, I apply a clockwise force to convey this ‘message’ but the tap ‘understands’ it to mean turn on.

In the above the syntax might be understood but the meaning is not. Hence when talking about a system we often to refer to the level of syntactic and semantic interoperability between components.

In computing terms, syntactic interoperability refers to such things as parameter passing mechanisms and timing assumptions and relates to the ability of one component to communicate with another. Semantic interoperability refers to the issue of common understanding or meaning of what is communicated between the different components e.g. when you pass data to an API that the receiving system understands the data in the same way.

On Substitution

Whilst composability is the ability to assemble components into various combinations of systems,  substitution is the ability to substitute one of those components for another.

Take any Meccano set. You have a mass of different components that can be assembled through an instruction set (a booklet on how to build) into various forms with components which have interfaces (e.g. application of angular force) and which operate as expected (e.g. clockwise force means tighten nut). 

If you look at the set you also have many duplicate items i.e. many identical nuts and bolts with the same apparent properties. However, the instruction set doesn't tell you which one of the identical pieces to use at a specific point and instead you can use any of the 'identical' pieces. This is a concept known as substitution.

I italicise 'identical' because the components aren't actually the same, they are just different instances (substantiations) of the same thing i.e. you're not replacing a nut or bolt with the same nut and bolt but a different nut or bolt which is hopefully identical. Substitution simply refers to changing one substantiation of a component in a system for another substantiation of the same component. Syntax and semantics are again important here.

For example, I can substitute one substantiation of a component (i.e. tap which is tightened by force) for a syntactically compatible substantiation of the same component (i.e. a tap which is tightened by force) but whose semantics are different and hence it operates in a different way (i.e. use of anti-clockwise force to open rather than clockwise). When you substitute a component for one that is not syntactically and semantically compatible then this often requires a change to the overall system. Such a change means work.

For example, suppose you have a chemical plant and you change one component for a syntactically but not semantically compatible version. Well, at the point your control system might want to cut off flow to a reaction chamber, you might get a nasty surprise and hence you're going to have to spend a bit of time adapting the entire system to this changed component. 

The greater the degree of syntactic and semantic compatibility that exists between different versions of the component the less work is needed to change a system. Since work normally involves time and resources then the sensible answer is not to redesign a plant but to use a component which is compatible (i.e. an identical replacement).

The same effects are important in computing. For example, let us assume I'm using Amazon EC2. Let us suppose I decide to change one m1.large machine instance for another then if both instances are not syntactically and semantically compatible then this will require a change to the overall system including management systems, orchestration etc. 

Fortunately with Amazon EC2 both machine instances are syntactically and semantically compatible from the point of view of the user. This is even true over regions. Hence a machine instance in one region uses the same API and it operates in the same way as another region, as opposed to having different EC2 APIs for different regions.

Of course, if they weren't syntactically and semantically compatible then the work needed could be alleviated by the introduction of a translation system i.e. an abstraction of the actual interfaces and provision of a common interface which translates to the various incompatible forms. This is always inefficient compared to compatibility between the different substantiations of the same component. 

I emphasise 'from the point of view of the user' because it is entirely possible that the systems that Amazon runs in different regions aren't actually syntactically and semantically compatible and the Amazon EC2 API is acting as a translation layer to different underlying interfaces. From the point of a user you won't know unless Amazon tells you. 

Unfortunately, whilst we have high degrees of compatibility within and between different regions of AWS, you have varying degrees between AWS and other cloud providers. Hence you’re always faced with an issue of work in substitution or use of a translation layer unless you stick with the one provider.

So why would you consider using another provider? Well, substitution is also important in business terms for various reasons such as pricing competition, balancing buyer and supplier power and second sourcing options.  However, substitution is equally important in economic terms. 

If we consider componentisation and its ability to enable us to rapidly develop higher order systems then that depends upon three factors :- 

1) higher order system being composed of components

2) interactions between the components

3) substitution of components for different compatible instances

Without the ability to replace a component with a different substantiations of the same component then in Meccano terms there would only be one instance of every type of nut and bolt i.e. every nut and bolt would be different. It would be the equivalent of having only one instance of a brick and hence every brick being different.  Under such circumstances it would be impossible to have common architectural plans. You could not rebuild any model that I built as you'd have to use different components. This would incur severe costs in terms of work in building any higher order system.

We even have a name for highly interoperable components that can be substituted for compatible versions - we call these commodities.  It's the provision of commodity components like bricks, electricity, nuts and bolts that has enabled rapid higher order system development and created the wealth of architectural building, consumer electronics and mechanical devices that we experience today.

It should be noted that large degrees of variation in compatibility of underlying susbsystems can have seriously negative effects on our ability to build. We call this sprawl. However, one nut and bolt doesn’t fit all purposes  (i.e. we require specific properties for medical components) and there are also issues with systemic failure (i.e. if all rice was the same type then the entire species can be eliminated by a single type of pathogen).

Hence for reasons of stability and agility then systems normally tend towards a limited range of types of a component with each type having defined semantic and syntactic interoperability with other components of the system. Each type of component also has syntactic and semantic compatibility between multiple substantiations of the same component ideally through multiple sources. 

Hence with nuts and bolts we have a range of standard types with defined properties.  Each standard type is produced in volume with ‘identical’ nuts and bolts produced by multiple providers.

It's important to understand that these commodities represent a limitation of choice i.e. there is a range of types for bricks or nut and bolts. It's that limitation of choice which enables our agility in building higher order systems.

On Degeneracy

There is another very important term we also need to consider and this (in biology) is known as degeneracy.  It's the ability of one component to take on the role of another component within a system. In engineering terms, it's the ability to redeploy a system for another purpose such as turning your refrigerator into a heating device.  Degeneracy is very important for adaptability to changing circumstances i.e. turning a wagon train into a defensible but makeshift fort.

On Context

If we look at figure 1 again, we can see it contains many components interacting with each other and higher order systems built from lower order subsystems. However, the overall characteristics of the components change as they evolve.

In the genesis state, a component is relatively unique. Our models of understanding are only just developing (i.e. it's a time of exploration). These components are not ideal for building higher order systems but they can consume lower order components. They generally show low levels of syntactic and semantic interoperability with other components and there is little or no substitution.

As the component evolves (due to competition) and we start to see custom built examples then our model of understanding of what this component is matures.  In this stage we normally see early forms of interoperability with other components along with  attempts of building higher order systems with it. For example, the early custom-built generators (such as the hippolyte Pixxi) were used to conduct all sorts of experiments in creating higher order systems like lighting.

As the component continues to evolve then we start to see the first products. Syntactic and semantic interoperability with other components starts to improve. Increasingly the product is used as a component of something else, for example Siemens generators being used to power machinery. Our models of understanding of what the component is become reasonably mature and even common understanding appears with expected norms of behaviour. We see early examples of substitution but syntactic and semantic compatibility between different substantiations of the same component from different providers is rare.  However, the importance of communication with other systems often leads to standards for communication between components. Hence whilst products can often be used and communicated to in the same way, substitution of one for another is often complex. 

As the component continues to evolve it eventually becomes more of a commodity and suitable for utility provision. Whilst our understanding of the component is very mature and expected norms commonplace, there is a period of transition during this change as we move from a product mentality (one of feature differentiation) to a commodity mentality (one of operational efficiency).  In this stage, syntactic and semantic interoperability with other components is well established. Syntactic and semantic compatibility between different substantiations of the same component from different providers develops strongly over time.  This provision of standard forms of the component with standard interfaces enables a rapid acceleration of building higher order systems.

The connection between these is provided in figure 2

Figure 2 - Evolution, Syntax and Semantics.

The Importance of Evolution

It’s important to understand the process of componentisation and how evolution enables this to happen in order to make sense of the change that occurs around us in business.  If you have any doubts about evolution then I suggest you pick a commodity item and go to your local library and look up its history. You’ll find that the types of publications around the item have changed over time from ‘wonder’ through ‘building’ through ‘operation and maintenance’ through to ‘use’. I’ve expanded the certainty axis from a standard evolution graph (figure 3) into figure 4 in order to give you some pointers.

Figure 3 – Evolution 

Figure 4 – Evolution and Type of Publication

There are those who would have you believe that evolution doesn’t exist and that the progress from genesis to commodity doesn’t happen or it's governed by some magic that only they know the secrets of.  Contrary to such mysticism, evolution is not a belief but instead a model of a surprisingly simple, repeatable and discoverable process driven by competition. You can discover it for yourself by taking that trip to the library and simply looking at how things have changed e.g. from early abundant guides on 'How to Build a Radio Set' to later dominance by use such as radio listings like the Radio Times. 

You live in a world where yesterday's rare wonders becomes today's invisible, commonplace and commodity subsystems.

Evolution also has impacts. It creates cycles of change, it causes inertia, it drives things to a more standardised form, it can be manipulated and its course accelerated through open means. If you don’t understand how things evolve then it’s practically impossible to gain strong situational awareness in business. The lack of this has demonstrable negative impacts.

A Case in Point

I wanted to specifically outline the importance of limitation of choice in the above because there is a current vogue for arguments over ‘PaaS is dead’, ‘App Containers are PaaS’ and ‘App Containers are the future of PaaS and all other PaaS is old hat’. 

Most of these these arguments appear to be based upon a flawed understanding of evolution, componentisation and the importance of limitation. So, I want to first look at an example of what a PaaS should be – such as Heroku, google app engine or cloud foundry. 

If you examine a system like Cloud Foundry then its focus is on the developer rapidly creating higher order systems.  This is achieved through a limitation of choice in underlying components i.e. specific buildpacks, defined services for common activities etc. The focus of the developer is pushed towards writing code, building data and consuming services.

Of course, with any new system that is built then the code and data can be packaged into a product and ultimately, if Cloud Foundry observes the ecosystem carefully then new services can be determined from this and provided to all.

Now, containerisation per se (examples being docker, warden, LXC) is a reasonable approach to isolation of compatibility issues in underlying infrastructure systems. PaaS environments like Cloud Foundry use containerisation under the covers.  I’ve summarised this in figure 5 and it’s an example of what I would describe as a strong PaaS play combining componentisation, limitation and consideration of how things evolve.

Figure 5 – PaaS and Limitation

However, an alternative view is the idea of App Containerisation. Unfortunately, containerisation is often conflated with shipping containers and how that industry changed through their use. However, shipping changed not because of the introduction of containers (there were a plethora of different shaped cargo containers in the past) but through the limitation of choice and the introduction of highly standardised containers.

The problem with the idea of App Containers is the application, the framework, the configuration is all contained within it and rather than limiting choice it allows for a wide range of permutations (see figure 6). This is often promoted as flexibility but such flexibility is the antithesis of componentisation and any desired agility in creating higher order systems. 

Figure 6 – App Containers

Admittedly App Containers are better than what exists in some firms today i.e. building everything yourself where everything is flexible but it’s far weaker in comparison to a PaaS that limits choice for what are common activities. Now, there are ways of solving the problems of App Containers by harvesting the ecosystem to identify common App Containers and weeding out all counter examples but that’s a highly skilled and difficult game.  Given that App Containers are often touted in Private PaaS environments then it is also unlikely to occur in many firms.

In all probability, as with the past where every major company seemed to have their own home grown linux distro then we’re likely to see major companies have their own permutations of application, development framework and configuration for every single activity even when it’s common. The sprawl will become horrendous and that sprawl will have negative consequences and cost.

The Pig and the Brick House

The best example of this was a thought model used by Herbert Simon to explain componentisation and it's based upon the story of the three pigs and the big bad wolf. Imagine you're a pig and you've decided to build a brick house.

Now unfortunately you've only time to do twelve things before the big bad wolf appears. Those things could include make a brick or cement two things together. Anything which isn't completed and stable is blown down. The house has four walls, each wall is ten bricks high and ten bricks long.

Now, unfortunately you need to start by building bricks from raw ingredients but fortunately each brick is a complete unit. So in the first turn before the wolf turns up you can build twelve bricks. Now, you'll need 4 (walls) x 10 x 10 (bricks) equals 400 bricks. Hence it'll take 34 turns and visits of the Wolf to get enough bricks. In fact in the last turn, you'll have 8 moves to do other stuff.

Now, you have a problem. If you try to build the whole house in one go then whatever you put together gets blown down by the wolf when it visits. You've got 400 bricks and you can't turn that into a house in one turn! As an alternative you can first cement ten bricks together into a line. Each line is stable component which will resist the power of the wolf. Whatever you do with the other bricks doesn't matter but at least after the next visit you have a stable line of bricks.

After ten more visits of the wolf, you have ten stable lines and so you can use the next visit to create a stable Wall cementing all ten stable lines together (one of top of the other). Repeating this process then after the 78th visit of the wolf you have 4 walls. Before the 79th visit you can cement your four walls together to create your stable component of a house and then be safe from the Wolf forever more.

The point of this thought experiment is to show that building through stable subsystems is essential for development of higher order systems. In practice it has an exponential effect. 

An even better scenario for the above would be a BricksRUs service where you can just buy-in pre-built bricks, lines and walls.  Of course, those pre-built components will limit your choice - a line is 10 bricks, a wall is 10 lines and a brick is a specified shape and size etc. But use of it will accelerate your rate of development. I could even have my house built in one turn, before the Wolf even gets there.

The key thing to understand is there exists a trade-off between flexibility of the lower order and agility in building higher order systems. The limitation of choice is essential for rapid development of more complex systems.

In the case of systems like Cloud Foundry then they are deliberately limiting choice by provision of defined subsystems e.g. services to be consumed, development environment to build in etc. This is a really good thing to do and is analogous to the BricksRUs example.  In the case of App Containers there is no limitation nor enforcement of such and there is only flexibility - you can make any brick you want, any shape, any material etc. This is a really bad idea as the permutations here are vast and this is what causes sprawl and limits agility.

I cannot emphasise enough how important an understanding of how things evolve, how things change with evolution, the benefits of componentisation and the necessity of limitation of choice are to navigating a safe path through the turmoil of today. 

PaaS has a bright future when we’re talking about Heroku, GAE, Azure, Cloud Foundry and equivalent systems. I'm also heavily in favour of industrialising apps and any common component where possible, something which CSC's Dan Hushon has recently talked about. Also, I'm very positive about the future of underlying tools and components like Docker.

Unfortunately there’s a lot of stuff trying out there to pass itself of as PaaS and a lot of misunderstanding on componentisation. Whilst components like Docker are extremely useful (and deserve to spread), there are those trying to portray it as a key defining characteristic of a PaaS.

Forget it, Docker will become a highly useful but also invisible component of PaaS and the success of PaaS will depend upon the limitation of choice and certainly not the exposure of underlying systems like Docker to end users. It’s extremely easy to take a path that will lead you down a route of sprawl. There are some very exceptional edge cases where you will need such flexibility but these are niches. I'm afraid some businesses however will probably get suckered into these dead ends. 

Hence the message of today is ... Caveat Emptor.

A Wardley Map

I was recently asked what do you call the maps (see figure 1) that are produced by the mapping technique I developed?

Figure 1

Are they Digital roadmaps? Product roadmaps? Strategy maps? Chessboards for business? 

First, the maps cover activities, practices and data and aren't limited to a specific field such as technology. They can be used to identify common services, differences, areas of efficiency, potential strategic gameplay, solve communication issues and ... a long list. Hence all the above terms are inappropriate.

Second, I created the technique at Fotango in '05 after determining a basic pattern of evolution at Foo in '04.  I've refined this since then, finally determining a clear and demonstrable pattern of evolution in 2007. 

Third, during all this time I have provided the mapping and evolution work under creative commons and given this freely to the community.

If you really wan't to give the maps a name - well, I'd appreciate some kindness back and recognition of my effort.  Hence, if they need to have a name, I'd be grateful if you could call it a 'Wardley Map'. It has the advantage of not limiting the technique to a specific field as the maps are generally applicable in multiple different spheres.

That said, my absolute preference is for them to be just called Maps.

--- Update 16th April 2014

I'd also be perfectly happy if you called them 'Wardley-Duncan maps', in homage to @jamesaduncan who was an important influence in the creation of them back in 2005-2006. My preference however, is that you just call them 'Maps' and don't stick other terms like digital, strategy etc onto them.

Whilst I'm at it, an alternative to the ILC model is the 'Wardley-Thompson Technique' in homage to the important influence that Mark Thompson had in refining the model back in 2010.

Is war a mother of invention?

Over the last decade, I've done quite a bit of work on refining the impacts of evolution on business. Figure 1 provides the typical pattern for how things evolve from their genesis to more commodity provision with this pattern being driven by demand and supply side competition.

Figure 1 - Evolution.

As things which were once novel and highly uncertain (i.e. uncharted) become more of a commodity (i.e. industrialised) then not only do they become more efficient but due to componentisation effects they enable rapid development of higher order systems which turn out to be new sources of wealth - see figure 2.

Figure 2 - Evolution begets genesis begets evolution.

It's through understanding this process of evolution, the impacts of inertia and the cycles of economic change that this interplay creates that I've found many modern mantras of management wanting. For example:-

1) The answer to the question 'Should I be first mover or fast follower to any change' is not one or the other but both. You should be a first mover to industrialise an act and a fast follower to genesis (i.e. the novel and new).

2) The answer to the question 'Should I use agile or six sigma' is not one or the other but both. Agile development is best suited for the uncharted whilst methods like six sigma are suited for the industrialised.

3) The answer to the truism 'Culture eats strategy for breakfast' turns out to be far more nuanced. Whilst both strategy and culture are important there are parts of the economic cycle where culture is more important and others where strategy is more important.

There turns out to be quite a long list of gross oversimplifications in ideas about innovation, change and management and it's one of these that has just recently piqued my interest whilst I've been going through many of the data points that I used to plot the evolution curve. My issue is with the idea that 'war is a mother of invention' and that 'war can drive technological innovation'. By war, I mean 'real wars' not the type of competition between industries that gets described as war.

To explain why I have a problem with this idea, I need to look at the history of Radio. In figure 3, I've shown the history of Radio in the UK between 1905 and 1962 plotted against Ubiquity vs Certainty.

For reference :

a) Ubiquity is a power function of (adoption / mature market adoption). It's a measure of how widespread something is.

b) Certainty is a function of (volume of phase II and III publications / mature volume of phase II and III). It's a measure of how complete and well understood something is.

Figure 3 - Radio, 1905 to 1962, UK.

 (correction made to the image above)

What's noticeable is that the development of radio shifted off the normal pattern between 1939 to 1944 and was less widespread during 1945 to 1954 than would be expected. This is not a surprise I hear you say because there was a world war and subsequent consequences of this.

However, had radio continued on its path then it is likely to have become a commodity much faster and consequently the benefits to high order systems (i.e. those novel things which are created with radio as a components) would have been achieved much earlier. This aberration of the general pattern of evolution occurs with other activities that were evolving during the same time period.

Which then leads me to my question. Whilst there seems to be this vague notion that 'wars are good for innovation', the above suggests (and admittedly weakly so) the opposite happens and that wars are not only bad for innovation but have longer term impacts after the war.  Had the war not occurred, then radio is likely to have continued on its path (as with other activities) evolving much faster that it did and hence our overall technological progress would have been faster.

I'll also note that this 2013 paper suggests it has 'found statistically significant negative associations between past war participation and residential as well as nonresidential patent grants'. I tend to agree, looking through my data leads me to the view that war has a somewhat negative effect overall.

Of course, I'll need to test this with other countries, participations in wars, impact on acts evolving etc. However, I'm curious does anyone know the rough source of the 'war is good for innovation' idea because it feels like one of those assumptions that is highly questionable.

For comparison to the image above - I've provided figure 4 which covers TV (1934 - 1978), Telephone (1912-2007), Radio (1905-1962, marked in blue), Home Video (1976-2003) measured on the same axis of ubiquity vs certainty. As should be noted there is always some variation and hence the examples I've found of a negative impact may well just be statistical noise.

Figure 4 - TV (1934-1978),  Telephone(1912-2007),  Radio(1905-1962),  Home Video (1976-2003)

Also some links on the background of evolution and the mapping technique I use which developed from it.

Looking for a Sci-Fi novel – recommendation please

I note with interest that Malta is in the process of selling EU passports to the wealthy. The scheme allows those  (and their dependents) with capital to simply buy the right to EU citizenship.  Of course, there are those who will argue that this will benefit inward investment and raise questions on those that oppose it.  I take the view that investment is more than financial capital and we should instead introduce a scheme to provide passports to those who have something positive to contribute rather than wads of cash.

What interests me however is there exists a rather nasty and hypothetical scenario for the future and it certainly seems a good subject for a Sci-Fi novel. 

The scenario would be as follows: -

First, a government might oppose this purchasing of passports but under the auspices of the TPP then a company selling passports would be able to sue for loss of profits if the scheme was denied. 

Second, the act turns citizenship into a commodity that can be bought … and therefore also sold. It’s the later that is the key to this scenario.

Third, bitcoin is growing and becoming more stable. Unfortunately with bitcoin comes long-term issues on taxation. This could mean that in practice income tax is likely to end up voluntary to an extent and the only secure means of raising taxation will be through land and citizenship tax.

Land tax (and as a first example of this then the mansion tax could be used but it’ll have to expand from there) would lead to centralisation of land control in private hands and extensive lobbying efforts to limit it.  Hence, a case could be made that we’re more likely to see a citizenship tax over time.

This itself creates a problem. Suppose I’m fabulously wealthy in bitcoins. There are many ways to obfuscate that wealth and proving the identity of an address holder can be made very difficult. Hence, it’s relatively easy for me to claim I’m poor when I’m not. So, how do you determine who is actually poor (and can’t pay a citizenship tax) against those who are wealthy and hide it? The answer is you can’t.

You can certainly claim I live in a nice house, drive a nice car etc but I could just as easily create charitable vehicles to provide these things. In a world of well managed bitcoin addresses with people specialising in wealth obfuscation then I can make it extremely difficult for any Government to determine any income or wealth owned. However, I’ll pay just for the right of citizenship.

But what about the poor person who can’t play? You can’t know whether they’re wealthy and just hiding it or if they’re genuinely unable to pay? The cost of investigation will be significant and alas you can’t simply ignore this because everyone else would quickly learn that they could dodge the citizenship tax.

Fortunately there’s a solution. If someone can’t pay their citizenship tax then you can simply sell their passport (citizenship being a commodity) to someone who can pay the citizenship tax and buy the person a cheaper passport (i.e. citizenship) in some other cheaper part of the world. 

It’ll become a monetarist’s laissez faire dream enabling the transportation of poor people to poor countries and turning parts of the world into a haven for the wealthy.  Of course, there’d be exceptions for military service - you need some measure of force to keep the divide and naturally we’d have to conveniently forget that poor people don’t make poor financial decisions simply because of incompetence but instead because they’re poor and the stress that being poor creates.

Endless arguments could be created to justify such a dystopian nightmare, the reinforcement of such a social divide and the inevitable repression and dearth of social mobility.  Rather than have 500,000 people in the UK relying on food banks and unable to afford citizenship tax, you could sell their citizenship for 1M Euro (say about 50 bitcoins in the near future), buy them citizenship somewhere else for 100,000 Euros (say 5 bitcoins in the future), give them 30 bitcoins (about 600,000 Euros in the future) as a ‘thank you and on yer bike’ and pocket the remaining 15 bitcoins (say 300,000 Euros). 

It’ll turn citizens into a commodity to be traded and a nice little earner at that.  Which is why someone, somewhere must have written a Sci – Fi novel on this? 

Now, I don’t think this is likely to happen – my view is the idea is ludicrous. I’d respond with about the same level of derision as if you had asked me twenty years ago that the NHS could be privatised. 

Still, I’d love to read a novel on this. Any recommendations?

The evolution of evolution

Back at EuroFoo 2004, I gave two talks - one on 3D printing and the other on Commoditisation of IT. In the latter talk, I discussed a generalised pattern for how things in IT evolved from their 'innovation' to more 'commodity'

By 2006, I had be using this pattern in various forms at around 40 conferences worldwide. The problem with the pattern was though I had numerous examples of it, I had none of the mechanics underlying it. The terms I used for the pattern (and also in mapping) are show in figure 1.

Figure 1 - The general pattern of evolution, 2004-2006

By mid 2007, by collecting and aggregating over 4,000 data points, I had managed to determine some of the mechanics of the pattern which covers activities, practices and data. This wasn't something that could be correlated over time but by comparison of ubiquity vs certainty (see figure 2)

Figure 2 - Evolution, the link between ubiquity and certainty, 2007

By late 2007, I had refined the pattern and determined the driving forces of supply and demand competition (see figure 3) though I did use both figure 2 and figure 3 throughout 2008.

Figure 3 - Forces behind evolution, 2007

Whilst the actual model itself hasn't changed since 2007, the modern version I use today is a cleaner visual representation with the term 'innovation' dropped in favour of genesis in 2011, see figure 4.

Evolution is a cornerstone behind the concept of mapping which is actually where the real value can be found. Naturally mapping had to change (in the sense of the terms used) as evolution did.

Figure 4 - The evolution graph - today

However, for me, it's interesting to note how my concept of evolution has itself evolved over time to a more stable form.

The use of mapping has enabled me to discover many different patterns of strategic play, answer common management issues and explore economic cycles. It has had real world effects since I first introduced the technique (in its earlier form) in 2005. A modern map, using the axis of evolution is provided in figure 5.

Figure 5 - A Map.

I allow myself a small bit of pride in both mapping and evolution because they are both genuine pieces of original work which did not exist beforehand and they both have use. Naturally, I know full well that at some point they will be superseded.

One thing to note is the link between evolution and diffusion. Evolution is derived from a series of diffusing and ever maturing instances of an act caused by competition ... that of course should be the subject of another post.

Should I be ‘outside-in’ or ‘inside-out’?

One of the areas of focus for LEF at this moment is the issue of outside-in business models where an organisation places emphasis on co-creation with others and the use of ecosystems. This often leads to the question of should a company be ‘outside-in’ or ‘inside-out’? 

The answer, as with other binary questions from 'should I be agile or ITIL' to  'should I be push or pull marketing' is always both. It's never an absolute but one of relative balance.

Our tendency with companies in the recent past has been more ‘inside-out’ and the current vogue represents a shift to a more balanced form and is part of the normal process of organizational evolution. 

In our 2011 study on 'Learning from web 2.0', we tested a model of how new organisational forms emerge via the interplay of evolution with existing activities, co-evolution of practice and inertia to change.  As with past examples - the rise of the American system, Fordism and web 2.0 - the current industrialisation of a range of IT activities from product to utility (nee cloud) has not only caused co-evolution of practice (and the emergence of devops) but also a new organisational form with different practices, activities and focus. A list of these characteristics are provided in figure 1.

Figure 1 – Traditional versus Next Generation

This 'next generation' of companies use ecosystems extensively, are driven by big data (as opposed to simply using it), have cell based structures (e.g. like Amazon's two pizza model), use open approaches as a weapon and demonstrate high level of situational awareness and strategic gameplay. They also show a shift in project management practices from the extremes (e.g. agile or six sigma) to a more balance approach of using mixed methodologies.

The different characteristics of 'next generation' are not independent but highly connected. For example, the use of mixed methodologies requires an ability to break down (deconstruct) large complex environments into components and this in turn requires and enables high levels of situational awareness (see mapping). An example of this is provided in figure 2 and the deconstruction of various aspects of HS2 IT.

Figure 2 – Mapping of HS2 IT

This high level of situational awareness is essential for new forms of gameplay such as the use of an open approach as competitive weapon in order to change a market. It was through mapping the competitive landscape, understanding the basics of economic change and a deep understanding of gameplay that Ubuntu was able to successful steal a march on the future against Red Hat (see figure 3)

Figure 3 – Gameplay of Ubuntu

It's the highly strategic gameplay of Canonical which is likely to be behind the recent acqui-hire of CentOS by RedHat. Don't get me wrong, this is a sensible (but somewhat desperate) move by RedHat to gain mindshare in the cloud given Ubuntu's near total dominance (usually estimated as 65%+ of the market) and RedHat's almost complete absence (usually estimated at between 3% to 5%). This is a world apart from the past dominance of RedHat on the server.

Deconstruction is also a necessary step in the creation of cell-based structures. The successful use of ecosystems requires not only deconstruction but also extensive use of big data. Furthermore the use of cloud not only requires but also has enabled the co-evolution of practice (devops) and the rise of big data. All of these things are tightly interconnected. Before you think there is a sudden appearance of these changes, it's worth noting that many of these changes have been diffusing over the last decade. In several cases we have over stretched and retreated to a more balanced view. 

For example, agile development was all the rage in 2002 but by 2005 many software companies had started to learn the lesson that it wasn't suitable for all classes of problems and a more balanced / mixed approach started. Today, using mixed approaches in a single organisation is becoming essential for competition as the one size fits all (either agile vs six sigma or in-house vs outsource) incurs unacceptable costs. Throughout the industry, more balanced approaches of breaking down complex systems into components and using appropriate methods are growing whether it's mapping (see figure 4) to USAF's implementation of FIST. The binary approach is becoming mixed. 

Figure 4 – Mapping and use of methods.

Hence when I talk of 'outside-in' it's not that all will become 'outside-in' but instead there is rebalancing of the 'inside-out' approach that has became commonplace.

Of all the 'outside-in' changes probably the most interesting for me is the use of ecosystems. Alas, the term itself is as much misused with as many different types as there is with 'innovation'. In this post I'll focus on one specific model known as ILC (innovate-leverage-commoditise) because it simultaneously embodies what I consider to be the best of both 'outside-in' and 'inside-out' approaches.

The starting point of the model is a supplier creates a highly industrialised component for other companies to use e.g. utility electricity provision or a compute utility (such as Amazon's EC2). The ecosystem is comprised of those companies that consume the component hence we often call it a component ecosystem.

The purpose of the model is not just volume operations from use of the component but to encourage other companies to ‘innovate’ in creating the novel and new e.g. building big data systems on Amazon EC2.  Those novel activities created are highly uncertain but potentially of huge future reward (they are uncharted) and provision of the subsystem as a utility simply reduces the cost of production and hence encourages development. As these novel activities start to evolve through multiple waves of diffusion of ever improving systems, then the supplier can spot successful growth through consumption of the underlying component. This allows the supplier to leverage the entire ecosystem to identify future change - an exercise that at scale requires big data systems. Once identified, the supplier can commoditise an act to a new component and in effect harvest part of the ecosystem to provide a benefit to all. 

An example of this is provided in figure 5. Of course, we don't know if Amazon is deliberately running such a model and harvesting its ecosystem, we can only observe that the results are similar.

Figure 5 – ILC model

Each time a component is harvested the supplier has to balance accusations of 'eating the ecosystem' against the overall benefit that the component will provide to the entire ecosystem. If you 'harvest' too much then members of your ecosystem might flee elsewhere.  Once harvested, the new component can then be used to repeat the ILC cycle with members of the ecosystem building novel and new components on top of it (whether activities, practices or data).

The success of the ILC model depends upon numerous factors including the scope of the components built (i.e. specific to an industry or widespread), ability to identify and harvest successful change (leveraging the ecosystem, a big data problem), speed of harvesting and ability to manage the ecosystem (e.g. to ensure more benefit is provided than harvested). 

If implemented successfully then the supplier benefits are huge as their apparent rate of successful innovation (the innovation in fact being done by others), customer focus (leveraging the ecosystem to identify successful and useful change) and efficiency (economies of scale) now all depend upon the size of the ecosystem rather than the physical size of the company.  Furthermore all these effects can increase simultaneously. 

Now whilst we don't know if Amazon is running such a model, the signs of ecosystem harvesting and continual increases in innovation, customer focus and efficiency are all there. This is what potentially makes Amazon such a dangerous beast. The power of Amazon is in its ecosystem and its exploitation of such.

So which bits are 'outside-in' and which bits are 'inside-out'?

Encouraging the ecosystem to innovate (and hence build potential futures) and leveraging the ecosystem to spot future success are all examples of 'outside-in' approach. The decision to harvest the ecosystem by commoditising a component is however 'inside-out'.  The latter is an act of active management and enforcement of the will of the supplier on the ecosystem. The former are acts of nurturing and sensing where the ecosystem tells the supplier what is important.

In some respects, the supplier is acting as the gardener of the ecosystem. Nurturing a crop, sensing what works, harvesting when suitable and ensuring the garden doesn't get out of control. The key difference here is the crop can move and hence the gardener has to act as the benevolent dictator of the garden ensuring the benefits of staying in the garden outweigh the threat of being harvested. A similar dilemma has faced many open source projects and the various commercial interests involved. The existence of a benevolent dictator (whether a person, organisation or company) is often a pre-requisite for success.

Learning these techniques is all part of playing the game of chess that is business and competition. Mapping itself is all about situational awareness and is inherently 'outside-in' starting with a focus on user needs to the components that a supplier has to use to meet those needs. However, the act of altering a map through strategic gameplay such as a decision to use an open approach for one component, to outsource, to insource, to erect barriers to entry or to demolish others barriers to entry is inherently 'inside-out'.  So, should I be 'outside-in' or 'inside-out' … alas, the answer is never simple and binary. Over time you'll need to become both. 

However, it's worth noting that the route to greater situational awareness is always with an 'outside-in' focus.  There is more data, more information and more awareness outside the organisation than inside and if you fail to exploit this then don't assume your competitors will return the favour.  For the time being, since many companies are dominated by 'inside-out' thinking then a strong dose of rebalancing is needed.  Hence if you ask me whether a company should have more of an 'outside-in' focus today then in almost all cases I'd strongly agree.

It is essential to understand that at this moment in time the 'centre of gravity' for the firm is shifting from 'inside-out' to 'outside-in'. It's not an absolute but a movement away from the 'inside-out' dominated approach of the past. This is why I take a great interest in our work at the LEF on ‘outside-in’ and the work of @dmoschella.