Tuesday, July 31, 2012

Everything evolves ...

Everything, by which I mean :-
  • every activity (what we do)
  • every practice (how we do something)
  • every mental model (how we make sense of it
... evolves from chaotic (poorly understood, rare) to more linear (well understood, commonplace).

This manifests itself in several ways. Hence :-
  • For activities we have the evolution from genesis to custom built examples to product (with rental services) to commodity (with utility services).
  • For practices we have the evolution from novel to emerging to good to best practice (aka Cynefin framework)
  • For data we have the evolution from unmodeled (e.g. we don't know what the structure is and hence we call it unstructured) to modelled.
  • For every scientific pursuit we have the evolution from concept to hypothesis to theory to universally accepted.

This process of evolution results in a cycle of change with :-
  • Commoditisation of one set of activities enabling the genesis of new higher order activities (componentisation)
  • Flows of capital from past to the future higher order systems (creative destruction)
  • Increasing pressure (through extraction of higher order value, efficiency and agility) for competitors to keep up with an evolving system (Red Queen Hypothesis)
  • Increased consumption of underlying sub systems as they become more commoditised (Jevons Paradox)
  • Coevolution of practice with activity. Hence best practice in the product stage <> best practice in the commodity stage. This creates transition cost to cover the technical and architectural debt to the past and is one of the forms of inertia.
  • Past success creating inertia to change and acting as the gatekeeper between the economic eras of peace, war and build
  • Varying rates of sustaining to disruptive change with the era of war seeing the highest rates of disruptive change due to past industries stuck behind inertia barriers
  • New forms of organisation appearing in the war era initiated by evolution of activity that take advantage of the co-evolved practices (e.g. American System, Fordism, Web 2.0)
  • The necessity of mapping value chain vs evolution and learning to apply the right strategies dependent upon the state of evolution of a component.
  • One size never fits all and the inevitability of arguments such as Agile vs Six Sigman, Push vs Pull, Networked vs Hiearchical due to the consequences of Ashby's Law.
  • Exploitation of ecosystems and structure to more effectively cope with the flow from chaotic to linear e.g. ILC or Pioneer-Settler-Town Planner
  • NB. When analysing you have to break down value chains into component activities - i.e. you cannot aggregate the mass of activities in a sector to describe an industry as a single activity such as "IT", "Mining" or "Healthcare". Hence IT is not one thing but consists of many component activities which are not the same e.g. Financial ERP, Infrastructure and Social Media. Even these can consist of multiple sub components which need to be broken down. 
Oh, we could go on and on for hours but I won't because I've done this to death many times before over many, many years.

However, I've been asked a couple of times recently about chaotic to linear, so I thought I'd add a number of  graphs which outline the changes and then summarise in the last two. This should be self explanatory to those who know my work. For everyone else, come find me at OSCON or London Cloud Camp at some point in the future where I take time to talk about this within those communities.

First, things diffuse but diffusion is only part of the story. There's a difference between diffusion of a particular phone and the evolution of phones from the first phone to a modern day phone.

Second, the process of evolution is common. To see it you have to first overcome the tendency of mapping with time (the axis you need are ubiquity vs certainty) and second you need to overcome the desire to call everything an innovation. The innovation of the first phone (Genesis), an innovative phone which has some new feature and the innovation of rental services for provision of an existing activity are not the same.

If you don't overcome this you'll just see innovation -> innovation -> innovation -> innovation over a random and unmeasurable time pattern rather than genesis -> custom built -> product -> commodity over defined and measurable axis.

This process is unavoidable because it's driven by user and supply competition.

Third, commoditisation and not genesis of activities cause a rapid change in society. It's the commoditisation of an activity that enable higher order systems to appear (nut + bolt -> machine, electricity -> computing etc). The new higher order systems are the new sources of wealth (hence we get creative destruction and capital flows) and the old "new" thing now becomes cost of doing business, standardised and more automated.

Hence as we evolve, we move up (and create) higher orders of our value chain.

It's worth noting that the net effect of commoditisation is not only efficiency but increased rates of higher order system creation and agility (componentisation) and the ability of companies to extract wealth from this. This combination of effects result in what is known as the Red Queen i.e. you have no choice but to evolve in order to retain a relative position compared to a surrounding ecosystem that is evolving.

Fourth, it's not just activities that evolve but practices (from novel, emerging, good to best). Practices are how we do something, activities are what we do. Now practice can co-evolve with activities i.e. best practice in the product world is not the same as best practice in a commodity or utility world. This change of practice is one of the many sources of inertia for company due to the cost of transition (which can also be thought of as technical or architectural debt).

Fifth, this entire process can be represented as a cycle of change with the commoditisation of pre-existing activities enabling the genesis of higher order system which in turn commoditise. Inertia (both user and provider) acts as an important gatekeeper and separates three economic eras of peace (relative competition), war (fight for survival) and growth (time of wonder).

Each of these eras have different modes of "innovation" i.e. the peace era, the time of highest margin, is all about sustaining change (which tends to exceed disruptive).

In the war era (caused by commoditisation of an activity due to its ubiquity and certainty achieving a state that this is possible i.e. volume operations of standard components) the disruptive change exceeds sustaining.

The growth era which is subsequently propelled by this, is a time of great experiments, of wondrous change due to the formation of these higher order systems which are by nature uncertain (see evolution graph near the top).

Sixth, this entire pattern repeats ad nausea - from the industrial age to the age of machines to the age of electricity to the internet age. What we see is commoditisation of a pre-existing activity initiates a state of war where new organisations form taking advantage of new co-evolved practices and these new organisations disrupt past industries stuck in peace mentality behind inertia barriers.

During this period where we focus on efficiency, automation and standardisation we also see an explosion of growth of higher order systems which we can't predict (electricity led to radio, hollywood etc), a shift of capital and the formation of new giants. And so it continues ...  Oh, and yes ... each time we see explosions of unmodelled data and endless arguments over classification etc.

Seventh, this occurs but at a macro scale (i.e. ages or k-waves if you prefer) and at a local scale (specific industry). Cloud is an example of this and has resulted in new forms of organisation (the new Fordism) as did every other example of this cycle (American System, Fordism, Web 2.0 etc etc)

The new Fordism practice will diffuse just like the old Fordism practice did. So, I've added a table of what you probably look like today and what you will look like tomorrow.

Seventh, when it comes to strategy, in order to effectively play the game you need to understand the landscape (situational awareness). 

The landscape is your value chain (i.e. what you do and the component which make what you do possible) and evolution stage of those components. The reason you need both is that the properties of activities, practices etc change as they evolve (more on that next).

If you don't see the landscape then any strategy you play whether an ecosystem game or open source or anything else ... is hit and miss, a shot in the dark. You could easily commoditise a barrier to entry into your industry or fail to effectively manage a threat to your value chain.

Lastly, as the activities (or practices or data etc) evolve they moves through three different stages of chaotic, transitional and linear. The characteristics of each stage is different and polar opposite at the extremes. This is why one size never fits all - whether it's project management method, structure, customer focus ... the whole lot. 

This is also why we end up with tiresome arguments over extremes such as agile vs six sigma, push vs pull marketing, networked vs hierarchical, NoSQL vs SQL, listen to vs don't listen to customers ... when the actual answer is always both.

Ok, this brings you upto about 2008 in terms of core models with some extra and more modern goodness in terms of how organisations change and the relationship between economic ages.

Don't ask me for papers on this, I don't write public papers, nor do I publish the thousands of data points I've collected. Why? Because I have too much fun using this stuff against others.

Do feel free to help yourself to anything you find useful. Be skeptical, be very skeptical. It's nothing more than weak hypothesis (i.e. concept, data, causation, correlation and prediction testing).