Sunday, March 08, 2015

Evolution, diffusion, hype cycle and early failures.

With mapping, one of the axes is evolution (the other being value chain - I've written an introduction on Wardley Mapping which covers the basics.).

One axis describes what something is (the value chain, from the user needs to the components required to meet it through a chain of needs). The other axis (evolution) describes change.

I've posted before on how the evolution axis was determined but when exploring this subject, I didn't start with the evolution axis as I had to discover it. In the very early days (2000 - 2004), I tried all sorts of different forms of measuring change. All failed.

The evolution axis (see figure 1) has a number of specific properties. It is :-

Figure 1 - Evolution

1) Universal. It appears to apply to all things that undergo competition whether activities, practice, data or knowledge. 

2) Causation. It is caused by the interaction of demand and supply competition. Without competition then it doesn't occur. It also doesn't require the need for a crystal ball as time is abolished from the axis. Evolution is measured over ubiquity vs certainty.

3) Measurable. The evolution of any component can be measured using a combination of publication types and how widespread the component is in a field. Unfortunately, only the past can accurately be measured which means the component has to become defined on the certainty axis (i.e. a commodity) before its past can be fully described. Until that point, there is an increasing element of uncertainty with position as the component becomes more uncertain (i.e. unknown).

We have no crystal ball and cannot predict precisely when things will evolve (I had to abolish time to create the evolution curve) however we can test evolution through the measurement of the past and secondary effects (i.e. other consequences caused by evolution). We can also use weak signals to anticipate evolutionary change.

4) Usefulness. There is a long list of reasons why mapping across the evolution axis is useful.

5) Consistency. There is only one evolution path. The evolution of one activity can be directly overlaid on the evolution of another. There is no need for a vague 'time' axis where 'time' is not a measurement but a direction of travel. 

6) Direction. A thing evolves in a single direction, i.e. the past is behind it, the future is ahead. It doesn't start evolving, become a product and then suddenly jump back into genesis as an unknown thing. When examining complex products, what we find is objects like a 'smart phone' are actually a grouping of many underlying components.

7) Recursive. Evolution can be equally applied at a high level (e.g. a specific business activity) or at a low level (e.g. a component of an IT system).

The early days

Before discovering the process of evolution, I did try to map with all sorts of things. These were inevitably failures. I thought I'd go through a couple and explain why.

Diffusion Curves.

Diffusion Curves (as per Everett Rogers) are extremely useful in many contexts. They are measurable (adoption vs time) but unfortunately the time span is inconsistent between different diffusion curves. If you overlay multiple diffusion curves on exactly the same time axis then you get multiple different curves. They also lack direction i.e. the diffusion of a particular phone A1 (from early adopters to laggards) is followed by the diffusion of a better phone A2 (from early adopters to laggards) which is followed by the diffusion of an even better phone A3 (from early adopters to laggard).  ... and so on. Hence the evolution of an activity is seen as a constant set of jumps back and forth rather than a direction of travel and the future of an act can be both ahead and behind it on the map. I've compared a map based on evolution (see figure 2)  with a map based upon diffusion (see figure 3).

Figure 2 - Map based upon evolution, direction.

Figure 3 - Map based upon diffusion, direction.

Hence from a mapping perspective, this makes diffusion curves useless. However, don't confuse that with diffusion curves being useless, in other contexts they are extremely useful.

Hype Cycles

Hype cycles are extremely useful in many contexts. They are not however based upon any physical measurement but instead are aggregated opinion. This means they are not testable, so we just have to accept them at face value (even when they do change the axis from visibility to expectations). The other axis is time (though it used to be maturity).  Time, in this case, is more of a vague direction of travel rather than any measurement of time. It also suffers from a lack of direction in the same way diffusion curves do. 

In figure 4, I've added four different hype cycles from 2002, 2005, 2010 and 2013. First thing is the axes have changed from visibility vs maturity to expectation vs time whilst the curve has remained the same. I even have a couple with expectation vs maturity.  This doesn't matter. The hype cycle isn't based upon measurement of some physical property, it's based upon opinion, so this is fine.

However, we have software as a service / ASP being in the slope of enlightenment in 2005 and cloud computing being in the peak of inflated expectations in 2010. Now, this all depends upon opinion and whatever definition they wish to choose. But, as with diffusion there's no direction i.e. one thing doesn't evolve into another but rather one thing in the slope of enlightenment becomes another related thing in the peak of inflated expectations. So, when it comes to mapping rather than compute (as a product) evolving to include compute (as a rental service) then evolving to compute (as a commodity) including compute (as a utility), with the hype cycle you have the same back and forth that you have with diffusion curves.

Figure 4 - Four Hype Cycles

This variation of direction (i.e. movement) is shown in figure 5.

Figure 5 - Direction

As I found out long ago, the hype cycle in the context of mapping is not useful (as with diffusion curves). Don't confuse that with the hype cycle being useless itself. In many contexts as a source of aggregated opinion, then it is very useful.  

I looked at many techniques to measure change and found all of them wanting. I spent years finding out that lots of things weren't useful for describing evolution. This is why I spent so long in the British Library cataloguing many thousands of publications. There was no effective means of describing the process of evolution until I'd done this work and found a process that seemed to work.

But then that's the point, diffusion curves measure diffusion, hype cycles provide an aggregated opinion on hype, whilst the evolution curve measures evolution (it doesn't measure either diffusion or hype). All have their purpose.

When it comes to mapping and improving situational awareness then you need to focus on what an organisation is (the value chain) and how the components of the value chain are changing (evolution). Does this mean the evolution curve is right? No, of course not. It's just the best model at this moment in time for examining evolution and as such it's an essential part of mapping.

--- 13th March 2015

Based upon Henry's question (below), I've added a graph to show the link between diffusion curves (Adoption vs Time) and Ubiquity.

The first graph shows adoption curves for different instances of an evolving activity A [through various instances A1 to A6], but rather than simply add adoption as the Y axis, I've added applicable market. The reason for this is adoption is to an applicable market and the market sizes for each evolving act are different.

Figure 6 - Diffusion Curves (adoption vs time). 

I've then added where each evolved instance of the act A is in the evolution curve.

Figure 7 - Evolution

[The two graphs above are purely illustrative. I've taken a few liberties to make the transition less complex.] 

Now, a couple of things are worth noting.

Ubiquity is a measure of how ubiquitous something is. In order to determine it, you need to know the end point i.e. when the activity has become a well established commodity. This point of ubiquity can only be accurately estimated when the act has become certain. At this point the past history can be graphed.

I have to emphasise, that the accuracy at which you can determine where something is or where something was on the graph increases as the act becomes more certain i.e. a commodity. If the act is not a commodity then where it currently is on the ubiquity axis becomes increasingly uncertain the newer the act is.

For this reason, you cannot when something novel appears (e.g. the genesis of electricity with the Parthian Battery in 400 AD) determine how ubiquitous electricity will become some 1600 years later. No-one can. This requires a crystal ball.  However by 1960s, you've a pretty good idea how ubiquitous electricity is and can graph the past.

The certainty axis is determined from publication types (see figure 8). In particular, it is a relative measure of type II and type III publication types.

Figure 8 - Publication types.

To graph the past, you need to first use the publication types (used to create the certainty axis) to determine if something has become an established commodity. You can then determine the point of ubiquity (what I call the reference point) and use this to determine how ubiquitous something was. You can then plot ubiquity against certainty in the past. An example of this covering a range of entirely different activities is provided in figure 9.

Figure 9 - Ubiquity vs Certainty.

You can then overlay the different areas where different states (custom built, product etc) dominate which gives figure 1 - the evolution curve at the beginning of the post.

To graph the future, you can't. The best you can do is to make a best guess at to where something is on the curve. Which is why with mapping I use broad categories - genesis, custom built, product etc. I get groups of people (with experience of the field) to estimate how far along it is. The accuracy of this will increase as the act becomes more certain.

To see evolution then you need to abolish time from graph (i.e. there is NO way to measure evolution over time in any form of repeatable pattern). However, whilst things will evolve over some (unspecified length of) time, you cannot accurately demonstrate where it is the evolution curve until it has become a commodity. Then you can accurately demonstrate where it was.

I cannot emphasise this more. The future is an UNCERTAINTY barrier which we cannot peek through. The more UNCERTAIN something is (i.e. genesis, custom built) the less able we are to see the end state. ONLY when something is close to being CERTAIN can we see the point of ubiquity (the reference point).

I do see people draw diffusion curves and start claiming things about them like here is a commodity, this end is an innovation. Alas, there is no repeatable graph of evolution over time.

We do NOT have a crystal ball. No-one does.

Diffusion <> Evolution. Evolution consists of many diffusion curves and there are many chasms to cross. The two are not the same, mix this up and you'll get lost.

Despite lacking any crystal ball, we can however say that something will evolve from genesis, custom built to product (+rental services) to commodity (+utility services). It will become more common and more certain. This change is driven by supply and demand competition. This is what figure 1 shows.

--- Just for Fun (13th March 2015)

Oh, and just because I like a good debate. I'll leave you with this graph of progress. Whilst being efficient with energy is good, anyone who thinks we can somehow reduce energy consumption permanently whilst maintaining progress needs to think again. Whatever we do, eventually if progress continues our energy consumption will exceed today. If you want to solve the energy related issues then you need to look for less damaging forms of production. Competition (whether life or business) is a constant exercise of reducing entropy within the local system through the consumption of energy.

Ubiquity vs Adoption (16th March 2015)

I thought this was fairly obvious but it seems there is still some confusion. I need to clarify.

100% Adoption of the iPhone 1 <> 100% Adoption of the iPhone 6. The markets are different.

Q. But can't we look at the smartphone market? 

Well, what do you measure against?

Q. When it's ubiquitous?

Define ubiquity

Q. When everyone has one?

So, gold bars aren't ubiquitous because not everyone has one? Gilts aren't ubiquitous because not everyone has one? CRM systems aren't ubiquitous because not everyone has one? Coffee beans aren't ubiquitous because not everyone has one? Computer servers are not ubiquitous because not everyone has one?

Except ... Gold bars, Gilts, CRM, Coffee beans and computer servers are all ubiquitous in their markets. It's just their markets are very different.

Q. But how do you convert adoption to ubiquity?

You need to find the point of ubiquity in a market. To do this you need to look at publication types and work out when something is a commodity. Then you can find the point of ubiquity and plot back how ubiquitous something was vs how certain (i.e. well defined, well understood) something was.

You cannot simply take adoption of something in its market and call that ubiquity. You cannot jump from an Everett Roger's diffusion curve to an Evolution curve. Just because they both happen to have an S-Curve shape doesn't make them the same thing.

Q. But that's what everyone does!

Well, they are all wrong.