Componentisation

Herbert Simons showed in the Theory of Hierarchy how the evolution of a system is dependent upon the organisation of its subsystems. In short, as an activity becomes commoditised and provided as ever more standardised components, it not only allows for increasing speed of implementation but also rapid change, diversity and agility with systems that consume the activity as a subsystem.

In other words, the creation of standard sizes of bricks, pipes, utility services and other architectural building blocks led to a faster rate of house building and a wider diversity of housing shapes. It's the same with electronics and every other field you care to look at.

Commoditisation to standard components leads to an explosion of innovation for higher order systems.

The cloud is no different. It's the creation of good enough, standard components which will cause a wide diversity of higher order systems. These will in turn undergo their own evolution to more of a commodity. Without this process we would never have got to a CPU, let alone cloud or the hugely complex systems that will develop from it.

This also doesn't mean that innovation stops with the standard components. Whether it's brick making or electricity provision, there is a huge amount of operational innovation hidden behind the "standard" however the "standard" acts as an abstraction layer to this. Just because my electricity supplier has introduced new sources of power generation (wind turbine, geothermal etc) doesn't mean I wake up one morning to find that we're moving from 240V 50Hz to something else. If that constant operational innovation was not abstracted behind the standard then all the consumer electronics built upon this would need to continuously change - the entire system would collapse in a mess.

Whilst cloud computing is all about commodity provision of computing resources through utility services, these components act as the abstraction layer. We're not quite there yet, we're still in transition from a product to a utility service world. However, even in that utility service world, providers will still innovate in terms of operations but to a consumer that will be abstracted and ultimately reflected in better price and quality of service. To the consumer this will be seen as operational efficiency and the activity itself for all sense of purpose will remain a commodity, just like electricity and every other industry that has undergone this change. Of course, I'll do more things with it, create new and ever more complex higher order systems but my innovation is based upon the activity being provided as a commodity.

I can understand how many people don't relish the prospect that some activities within IT are becoming more of a commodity. They may not realise the benefits this will create in terms of progress but at the same time as they cry this isn't happening they happily consume systems which depend upon this process.

If you really don't like the idea of commoditisation and componentisation then try getting by without using anything which has depended upon it. Try building a computer without a CPU, electronics, electrical power supply or even nuts and bolts.

The Red Queen Hypothesis ... Part II

Organisations contain a mass of different activities and a network of people performing those activities.

If you take away both the activities and the people, you are left with what an organisation really is, which is nothing (bar reserves of capital). Organisations only exist in the interaction between people and activities. However, people come and go and, as previously mentioned, activities are in a constant state of flux. Hence all organisations are continuously exposed to change.

No organisation can ignore such changes for long as they are not islands but instead live in a competitive environment. If an activity becomes more of a commodity and the organisation fails to respond, the result is a competitive disadvantage. Organisations must therefore continuously respond and adapt to these changes, in people and activities, in order to retain their competitive position against others.

This is the business equivalent of the Red Queen Hypothesis from Genetics. It should be remembered that there are two very different and powerful forces of change in any competitive environment:-

1. Adaption: the need to constantly respond to changes in people and existing activities.


2. Creative destruction: the constant destruction of the old ways of doing things by the creation of the new.

The general rule of thumb is:-

"You need to adapt in order to survive today but you also need to innovate in order to survive tomorrow."

The Red Queen Hypothesis ... Part I ... Activities

The Red Queen Hypothesis is used in Genetics to describe why systems need to constantly adapt in order to remain competitive. Formally, it is stated thus:-

"For an evolutionary system, continuing development is needed just in order to maintain its fitness relative to the systems it is co-evolving with."

(Leigh Van Valen, 1973, from wikipedia)

I want to describe this effect in terms of business, however to do so we need to first look at how business activities change. Let us start by examining the use of CRM.

The concept of CRM (customer relationship management) systems was an innovation back in the 1980s. However as everyone sought to exploit this new concept, CRM became far more ubiquitous and well defined. The activity has undergone a metamorphosis from innovation to leading edge to product to even utility services. This is not an unusual event, as there is always a constant pressure towards commoditisation of any activity as everyone tries to take advantage of any innovation (see figure 1).

Figure 1 - The metamorphosis of CRM.
(click on image for larger size)

In figure 2, I've mapped this transition on a graph of ubiquity (how common something is) vs certainty (how well known or defined something is).

Figure 2 - A graphical representation of the transition of CRM.
(click on image for larger size)

The transition of an activity from an innovation to something ubiquitous and well defined (or more commodity-like) is fairly standard. Most activities (whether processes, sub process or the results thereof) are in a continuous state of transition.

Organisations consist of a mass of activities, and those activities exist somewhere on that graph. The activities are all connected and you can even map this out. However, for the time being I've provided a representation of an organisation in figure 3 in graph form.

Figure 3 - Activities in a organisation
(click on image for larger size)

Whilst these activities are at different stages of their lifecycle, they are all undergoing a metamorphosis from innovation to commodity. This transition is independent of the organisation itself, as an activity becomes common when others adopt it.

All organisational activities are therefore in a constant state of flux.

Now, I'll use this concept in the next section to explain the Red Queen Hypothesis and its application to business.

XTech talk ...

By extracting the audio from Ian Forrester's recording and mixing it with the original slides, I've put together a video of my talk from XTech.

Overall the talk was so, so.

It needs a bit more sparkle and some of the concepts didn't come across as clearly as I had hoped. There are possibly too many interwoven ideas and a few of the graphics are looking a bit tired as well.

The main topics are:-

• An introduction to commoditisation, creative destruction and competitive advantage.
• An introduction to innovation.
• An introduction to the underlying processes.
• Why nothing in management is simple.
• How this impacts IT.
• Why open is essential for a service world.

Why open matters
from innovation to commoditisation
(approx. 43 mins)

Move over Bellatrix ....

In the software business, there are several unforgivable curses which deserve a one way ticket to Azkaban. These include "What backup?", "What version control?" and "What testing environment?". Fortunately, the days when you might have heard such words are long gone as we all have learnt the folly of the dim distant past.

However, we are always living with our future curses, e.g. the naive and foolhardy things we do today which we will laugh at tomorrow. So, I've decided to pick out a couple of candidates for future curses.

What model?
The purpose of business process modelling (BPM) is not just to provide a view of what we do as an organisation but also to enable an architecture to be built to support our current and future activities. Obviously we cannot model innovations with any certainty, they are constantly changing. However commonly repeated activities can be modelled and subsequently used to create a supporting infrastructure. I was under the misguided impression that companies embarked on a Service Orientated Architecture (SOA) after first having examined what they do by Business Process Modelling (BPM). Apparently this is not the case, some companies start building without actually knowing what it is they do. There is always a balance to be struck between the two morals of "an imperfect plan executed today is better than a perfect plan executed tomorrow" and "proper planning prevents poor performance". I'm not sure that this novel approach achieves this.

What lifecycle?
Whilst BPM will give you an overview of what you actually do and help in the design of an architecture, it doesn't actually tell you how you should manage an activity or process. As I've mentioned before, I tend to "colour-in" my models to identify activities at different stages of their life-cycle. This provides me with information on how to deal with an activity, for example :-

• whether an activity is ripe for outsourcing or SaaS (assuming a suitable external ecosystem of providers exists).
• how I should manage a particular activity (for example more agile or more defined processes e.g. SCRUM or Prince 2).
• how I should measure it.

I'm fully aware that this runs contrary to our desire for simple measures; however even a simple measure such as ROI (return on investment) is only valid for particularly stages of the life cycle. For innovations, you need to work on a worth based mechanism whilst cost is your only ally with CODB (cost of doing business) activities.

Arthur C. Clarke said that "any sufficiently advanced technology is indistinguishable from magic". We shouldn't forget that we are all just underage magicians and tomorrow will look back at much that we thought was "magic" and just cringe.

Additions

I couldn't resist but add a few more unforgivable curses.

We only release once per month
Commodity like activities should be released or updated as little as possible, once per year is probably once too often. Innovations need a completely different timescale, once a week is possibly too slow. The one size fits all idea is instead one size doesn't fit anyone.

Anyone feeling cold?

I was recently asked, "where is IT heading?"

Apparently, there seems to be a view that commoditisation will lead to the end of IT. Though it will certainly lead to changes and a shake-out in some of the practices and personnel, the idea that the end is nigh for IT is greatly exaggerated. Outside the support and training function, I would argue that IT is more likely to fragment into three different types of role.

One role, which I'll call the Pioneer, will focus on helping the business create new mashups, widgets and services to exploit the interactions between a common framework of services and information in the outside world. The expertise of such a role will be in experimental modelling and agile processes & novel business development. In my diagram (see figure 1), the pioneer's domain is in the innovation (think novel, first time) and the custom built sections of the activity graph.

Figure 1 - Stages of an Activity life-cycle.

(click on image for larger size)

To gain an understanding of what this Enterprise Mash-up world will look like, I'd suggest following the writings of Dion Hinchcliffe

A second role, which I'll call the Coloniser, is probably the most demanding. Their focus will be on bridging the gap between the new innovations of the pioneers and the common framework of services upon which the organisation depends. The coloniser's job is to find, investigate and make the call on whether a new activity (an internal mash-up or process or an external one) should make the journey to be included in the framework, to start to turn those custom made things into products. Their world is in the minutiae of tactical decisions from open source to standards plays. Their focus is to gain as much advantage as possible from an activity that is becoming more common, whilst avoiding the dreaded cost of migrating to an alternative standard. They need to constantly decide which horse to back and to understand the future potential consequences in a world of tactical play and misdirection. In my diagram, their domain is the product (transitional) section of the activity graph.

The final role, which I'll call the Town Planner, will be focused on managing componentisation, outsourcing and providing the framework for all common processes and services that a business uses. The expertise of such role will include service orientated architecture (SOA), enterprise architecture (EA), business process modelling (BPM), six sigma, volume operations, software as a service (SaaS), contract negotiations, risk & security management, compliance and all the activities we associate with a well ordered world. In my diagram, the town planner's domain is in the commodity section of the activity graph.

So, will commoditisation lead to the end of IT? No, IT is only starting to get interesting. However it will cause a big shake-up and the army of half-competents who hide behind the obscurity of today's "complex" systems will find themselves butt bare naked in a world where a cold wind of change blows.

If you make your living around telling business that ERP / CRM or any of such ilk are a source of competitive advantage, you're wearing the Emperor's new clothes and you'd better find some real ones.

--- 4th August 2014

It's over six years later and ... we're slowly starting to see the signs of change in organisations. I estimate it'll take another 10-15 years after the first major move which probably puts this around 2030. The most recent set of organisation changes mainly involved cell based structures. This more adaptive structure is still a long way off.

The original image link was broken, thanks for pointing that out. I used the same image from another blog post to replace. If you find other broken links, do tell me. I keep all my old presentations and can go back to source.

--- 30th June 2015

Slowly we're starting to see more public discussion on the spread of cell based structures and also structures that combine not only aptitude but attitude e.g. the three party system of the hybrid dynamic model. We've probably got a good decade to go before this start becoming more noticeable. Alas, we've also got a resurgence of a dual operating system model which to be honest I thought was dead and buried long ago. A dangerous space to be, re-invoking those images of Elois and Morlocks. Been there, done that, bought the flares, wish I hadn't, not doing it again.

I'll huff and I'll puff and ... ohh I like how you've painted it.

Whenever I'm mapping out the activities (for example business processes) of an organisation, I try to use colour codes for the different lifecycle stages of an activity. I find this helps me when visualising what the organisation needs to do and how it needs to change.

It's just something I do. I've provided four images to show the colour codes I use.

Stage 1 - Innovation, First Instance
(click on image for larger size)

Stage 2 - First Movers, Bespoke examples
(click on image for larger size)

Stage 3 - Transition, Products
(click on image for larger size)

Stage 4 - Commodity.
(click on image for larger size)

It's worth remembering that any activity has an effect on the organisation. In the case of a created product the effect is overall on cashflow, however you could equally have a process whose effect is on the efficiency of operations and cost reductions.

Generally, any innovation should be built upon many commodity or transitional like activities. If it's not, you need to ask yourself why not?

Componentisation of systems into activities and use of commodity components where possible is a massive accelerator for innovation. It is the reason why I advocate using Service Orientated Architectures (SOA), Software as a Service (SaaS) and Enterprise Architecture (EA) frameworks like Zachman.

The speed at which a complex system evolves is much faster if it is broken into smaller stable components and hence organised into one or more layers of stable subsystems (for more on this read up on Howard Pattee).

If you:-

1. take a system of k elements
2. group every s number of k elements into a new component, l
3. group every s number of l components into new component, m
4. keep on repeating this grouping until you can't group any more

Then, with each component being considered stable, the rate of evolution of the system will be proportional to the log to base s of k.

To show this in action, consider the three little piggies building a house. Let's say each house requires 100,000 bricks and whilst the big bad wolf can blow down an unfinished item, any stable component is too strong to be blown apart. Our three little piggies will follow different strategies:-

• Piggy 1 : Build the house in one go with each brick being a single component.
• Piggy 2: Build stable components, each component containing 10 sub-components. i.e. 10 bricks = a line. 10 lines = a section of wall etc.
• Piggy 3: Build stable components, each component contain 100 sub-components.

OK, let's say on average you can put together 1,000 components before the big bad wolf returns. Then :-

• Piggy 1 : will never be completed.
• Piggy 2 : will be completed by the 12th visit of the wolf.
• Piggy 3 : will be completed by the 2nd visit of the wolf.

In general: build in blocks, use small stable components.

[NB: For simplicity of explaining the analogy, I've taken the initial act of combining 100 or 10 or 1 brick(s) into one component as creating one component. If you instead treat each brick as a component, then the times are Pig 1: Never, Pig 2: 112 visits, Pig 3: 102 visits.]

It sounds obvious, but knowing the lifecycle stage of an activity along with componentising systems which can be componentised is a necessary step to increasing innovation. In other words: if someone has already built a hammer, use it and don't rebuild it.

Equally essential is to use different methodologies at different stages of an activities lifecycle (it's not agile vs six sigma or networked vs hierarchical or innovation vs efficiency - it's a mix of each, all the time). In other words: get used to living with change.

Using such an approach you can balance the innovation paradox between the order needed to survive and the disorder needed to create a future.

In summary: build in blocks, use a hammer, expect the plan to change and don't forget to add a splash of colour.