Wednesday, August 26, 2015

On Diffusion and Evolution

I recently saw this tweet and unfortunately, despite good intentions, there's a lot wrong with it. I've taken the main image (unknown copyright) for figure 1 and I'll go through what is wrong.

Figure 1 - Evolution mixed with diffusion

The fundamental problem with this image is it conflates diffusion with evolution. Whenever we examine an activity, practice or form of data then yes, it tends to diffuse. But, it also evolves through the diffusion of ever more mature, more complete and more certain forms of the act. Hence in the evolution of an act there maybe hundreds if not thousands of diffusion curves involved. The problem you have with trying to tie diffusion curves to evolution is in trying to determine which diffusion curve you're on. So lets go through this. 

Diffusion curves were first noted by Everett Rogers (made famous by Geoffrey Moore in Crossing the Chasm) and they show adoption over time. It's normally presented in the cumulative form (an S-curve) as shown in figure 2 rather than the above figure 1.

Figure 2 - Diffusion Curve

Now, as Roger noted, there is no single diffusion curve - they operate over different timescales and its adoption to the relevant market. Examples of diffusion curves can be seen in figure 3 from a wide range of technology related activities.

Figure 3 - Diffusion of technology

From the above you can clearly see how these S-curves have different time spans and the problem is often with when do you determine the origin of the technology (not an easy task in itself).

Now technology doesn't just appear and diffuse, it also evolves. Electricity sources for example started with the Parthian Battery (as best as we can tell, in 400AD) but had evolved to utility provision of electricity by 1886 (e.g. Tesla, Westinghouse and Edison). But is this a result of diffusion?

A simple test for this is to first ask yourself - "Are smartphones a commodity?" - now whilst some might say yes, others would argue not which means there is variability. However, in a market like the US there are more smartphones than people and so you can say it is widely diffused.  Now ask yourself - "Are gold bars a commodity?" - to which almost everyone would reply yes. However, it is certainly not the case that everyone in the US owns gold bars, they are far less diffused. You cannot therefore argue that some level of diffusion (i.e. adoption in a market) relates to some level of evolution. In some cases, a thing may become a commodity when 5% of the population have it. Whilst in other cases, a thing is still evolving when 95% of the population have it.

Evolution instead often involves multiple improving examples of the act, all of which diffuse through some form of applicable market. But the relation to overall diffusion is anything but simple. For example, let us take an activity A[x]. Let us suppose it evolves through multiple diffusing instances of the act (e.g. if A[x] was telephony then A[1], A[2], A[3] and so forth would represent ever better phones). I've added these diffusion curves into figure 4 below.

Figure 4 - Diffusion of an activity A[x]

Now each of these diffusion curves can cover different time periods and different applicable markets but each one goes to 100% adoption of its applicable market. Each will have a chasm i.e. in the evolution of A[x] there will be many chasms to cross and not just one. So, when examining the question of early adopters to laggards then we have to ask, which diffusion curve are we on? The laggards of A[1] are not the same as the laggards of A[5].

The natural tendency is to respond with - "well, we will measure the overall one i.e. when it becomes ubiquitous" but this leads to issue highlighted above because gold is a commodity (i.e. well defined, understood, standardised) but diffusion amongst the population is low. The problem is that ubiquity is to a market and so you can't just say "measure its ubiquity" because you need to understand the applicable market first. Hence in some cases, a ubiquitous market is 5% of the population owning an example of this thing (i.e. that's all it will ever get to) but in other cases a ubiquitous market is everyone in the market owning fifty examples of this thing. 

So how do you determine the appropriate market and how ubiquitous something is? Actually, this was the trick I discovered and refined back between 2005 to 2007. As things evolve, they become more defined and certain and the type of publications associated with the act change. It's through understanding uncertainty related to an act that you can determine the ubiquitous market and how evolved something is. But how?

To begin with, there are four basic types of publications show in figure 5.

Figure 5 - Publication Types.

So when something appears e.g. Radio then we first write about the wonder of radio, then how to build and construct a radio crystal set, then we move onto to differences between radios until finally being dominated by guides for use.  I used just over 9,000 articles to determine these four types and used this to develop a certainty axis (shown in the figure above and developed from type II and type III publications) and a bit more detail on this is provided here and here.

Now, the transition from Type III to Type IV in the graph above is critical because this defines the point of stability (i.e. something stops changing in characteristics) and this can then be used to identify the point of ubiquity in an overall market (see figures 6 & 7 below). In other words, you find when something is stable and then determine the size of the actual market. You then use this to trace back history.

Figure 6 - Point of Stability

Figure 7 - Point of Ubiquity

If I take this as the point of ubiquity and now plot back through history over both ubiquity and certainty for a wide diversity of activities then you get the following (figure 8)

Figure 8 - Ubiquity vs Certainty

The figure above represents a large range of different activities from telephones to TV to fridges etc. It's important to note that there are multiple activities in the above but one pattern that emerges. This is because it's not a diffusion curve (which is dependent upon an applicable market and varies in timescale) but instead it's a pattern for evolution. It's unfortunate that the pattern happens to be s-curve shape because I'm sure that adds to the confusion. However, the above is not adoption vs time but ubiquity vs certainty.

Now, if you overlay the the different publication types (i.e. type I, II, III and IV) as shown in figure 9 and extend for both the future continuing after it becomes ubiquitous & stable and the actual invention was before someone wrote something down about it then you create the evolution curve (see figure 10). This is what I published almost a decade ago.

Figure 9 - Adding publication types

Figure 10 - Evolution curve.

What drives this evolution is competition (supply and demand) and that's marked on as well. The point of the evolution curve is it shows you the path of how things evolve - hence its name. Ubiquity is to the applicable market and certainty is a measure of how well understood, defined and stable it is. The evolution curve itself is the x-axis for Wardley mapping (a mechanism of visualising a competitive environment). Again this has been used for a decade across many companies and even Governments but that's another post.

When can now go back to our diffusion curves in figure 4 and plot them on the evolution curve. I've illustrated this onto figure 11 (nb. this particular graph is just an illustration, not based upon data). For example, what you'll find is that a thing becomes diffused in its market (100% adoption) but still has a long way to go before it's evolved e.g. 100% adoption of A[4] is on the border of product / commodity. 

Figure 11 - Diffusion on Evolution

So when we look at A[1] from a diffusion point of view we might have crossed the chasm and the laggards maybe joining but it's very much in the early stages of evolution. We know from the publication types that despite the act reaching close to 100% adoption of its market that the market is nowhere near evolved. But at A[5] the act is very evolved and we already know that we've reached the point of ubiquity in the market from the publication types. It might not be the case that everyone has this item but this is what the ubiquitous market for this item looks like and it is now a commodity.

Now with evolution I can add all sort of changing characteristics i.e. genesis is very different from commodity (see figure 12). So for example, I know those activities or components in the genesis phase are uncertain, rare, risky, a point of differentiation, poorly understood, chaotic, deviates from the past, a source of worth, rapidly changing etc etc. This is a subject I talked on at various conferences (wrote articles including in the Butler Group Review 2008) during 2007-2009 and it's based upon the two extremes identified by Salaman & Storey in 2002 in their Innovation Paradox.

Figure 12 is my "cheat" sheet. I call it that because by looking at the characteristics I can roughly determine how evolved something is. So when someone says to me "Gold Bars" or "Smart Phones" then I can look at the cheat sheet and roughly determine how evolved it is.

Figure 12 - Changing Characteristics

So, lets go back to the original image at the beginning of the post. You can't just mix diffusion and evolution together in that manner. Everyone might own something (i.e. a smartphone) and it can still be constantly changing or barely anyone can own something and it might be stable and more commodity like. You certainly can't add concepts of time into this because you have no idea what diffusion curve you're on and diffusion curves have different timescales.  I understand what the author and tweet was trying to convey but alas - as simple and as seductive as it sounds, it's just plain wrong.

For those wanting to use the diagrams above, they all date from 2007 onwards and are creative commons licensed. The original work (i.e. data collection and testing) was done in 2006 & 2007 and the concepts actually date back much earlier in case you're interested (e.g. I was using the "pattern" of evolution back in '04/'05 though at that time it was just a noticed pattern rather than something with substance).

--- 4th Feb 2016

Added graphs 6,7 and 9 to make it clearer and tidied up a few typos. Added a few lines to make it clear what the evolution curve is, a description of the path of how things evolve.

Also, worth noting that evolution is only a model and like all models it will also be wrong and will be superseded by something better. However, that said, it's not an excuse for just going around taking diffusion curves and randomly adding characteristics to them. I'm sure the author means well but alas I keep on coming up against endless "graphics" in management. It would be nice if people actually put some effort in, collected some data and critically questioned. It shouldn't have taken much to realise that some technology can be in the hands of everyone (i.e. widely adopted) and still not stable, defined, complete and commodity like.

It's really important to understand this distinction otherwise people start making huge leaps. They start plotting diffusion curves in the general population and going - "well, it'll become a commodity at this point in time". You can't do that as the act of shifting from product to commodity depends upon individual actors' action (See Hayek and the pretence of knowledge) and a number of factors (concept, suitability, technology and attitude) which means you can't predict the change over time in this manner. However, what you can do is look at publication types and estimate when combined market forces are likely to make the change occur i.e. it's a probability function.

For example the probability function may indicate that it is increasingly likely that a commodity version will appear (all the factors are in place, the publication types indicate this) when 20% of people have one or it may indicate this when everyone has several of whatever it is (say 150% of population). Hence knowing that 10% of people have one today doesn't help you as you don't know if the change will occur at 20% or 150%.