Saturday, March 21, 2015

The curious case of #RPSFail

The Register screamed 'Another GDS cockup: Rural Payments Agency cans £154m IT system' which it then cried 'escalated to £177m' and so the media told of us a lamentable Government IT failure. 

These things happen, the private sector has a terrible record of IT project failures but fortunately a big carpet to sweep it under. The NAO is bound to investigate.

The Register told us how GDS championed "the new agile, digital approach by the Rural Payments Agency" whilst apparently "The TFA has always been opposed to the 'digital-by-default' dogma"

The Register made it plain, it "understands that the Government Digital Service was responsible for throwing out a small number of suppliers working on RPA instead and went for a 40-plus suppliers approach - focusing too much attention on the front end, and little attention to integration between front and back." 

The finger of blame pointed firmly at GDS.  So, what actually happened? How did GDS get this so wrong? Well, we won't know until the NAO investigates or GDS posts a post mortem. Everything else is speculation and El Reg loves to try and cause a bit of outrage. But since the media is in the speculation game, I thought I'd read up more and do a little bit of digging.

The first thing I noticed in the mix was Mark Ballard's article. It started with the line "Mark Grimshaw, chief executive of the Rural Payments Agency is either an imbecile or a charlatan, if Farmers Weekly is anything to go by."

What? I though this was GDS' fault?

"He's been telling the agricultural press that his agency's prototype mapping tool is a failure. That's like saying a recipe is duff because your soufflé collapsed on your first trial run."

Prototype? A £177 million prototype?

"Farmers were apparently unhappy the prototype was not working as well as a production-quality system. So Grimshaw called a press conference yesterday and announced that the sky was falling down."

Hmmm. Something doesn't seem right here.

"The odd thing was that the mapping tool had only just been released as a public beta prototype. A date hadn't even been scheduled for a live roll-out."

Hence, I decided to check on the Rural Payment System (RPS) that was being built for the Rural Payment Agency (RPA). Sure enough, the RPS had only recently been released as a beta. Also the cost is only (cough) £73.4 million. Still, an awful lot but how did it get upto £177m? I checked El Reg, it was apparently a "source".

One eye opener when checking on the Government site was that RPA would "help prevent fines (‘disallowance’) for making payments that don’t comply with CAP rules (~£600m since 2005)"

Really? How comes we've been paying through the nose for compliance failures? A quick search and I stumble upon the fact that the RPS is the second incarnation of a system. The first incarnation (SPS) was so shockingly poor that in 2009 NAO urged DEFRA agency to replace the £350m system even though it was only 4 years old

£350m? … Oh but it gets worse.

In 2009, according to the NAO then along with a £350m system, we had incurred an additional £304m administration cost and £280 million for disallowance and penalties and £43m irrecoverable overpayments. The cost per transaction was £1,743 and rising (22% over 4 years). This compared to £285 per claim under the simpler Scottish system. This was 2009? Heaven's knows the cost to date.

What marvels of genius had created this 'complex software that is expensive and reliant on contractors to maintain' - why expensive consultants. In fact, 100 contractors from the system's main suppliers at an average cost to taxpayers of £200,000 in 2008/2009. The SPS is so complex and "cumbersome" because of "customisation which includes changes to Oracle's source code"

Now, £73m is a bad loss but nowhere near as bad as £650m (system + administration) for the previous system with two main contractors. El Reg's wisdom of keeping it down to a few suppliers has just gone up in smoke. However, just because the past was a debacle doesn't mean the future has to be. This was one of UK Gov's exemplars and GDS had been pretty upbeat about it.

A bit more digging and I come across Bryan Glicks article. Of note - 

'The Government Digital Service (GDS) introduced new controls over IT projects, designed to avoid big, costly developments depending on contracts with large suppliers. When Defra/RPA went to GDS with its initial proposal – a 300-page business case, according to one source – it was quickly knocked back.'

'Instead of a few big suppliers – , for example – RPA would be agile and user-led, with multiple small- and medium-sized suppliers.'

This sounds all very sensible. But still it went wrong … even if far less money was lost. Now the project sounds complex, according to the article there are - "multiple products involved that need integrating – more than 100, according to one insider".

Two suppliers seemed to be called out - one in the article Kainos and one in the comments Abaco along with the line 'this Italian company have not delivered most of the work on time and is a factor in the whole project being delayed'.

Interesting ... who are they? are they really involved? what is the role of any delays? So, more digging.

Kainos is the sole delivery partner for the original mapping prototype which was comprehensively praised by Defra. It's an agile development house, regarded by the Sunday Times as one of the top 100 places to work and seems to have a pretty strong pedigree.

Abaco provide SITI AGRI -  first thing I noticed, which caused my heart to sink was 'Oracle'. Don't tell me we're customising again?  It mentions "open" but I can find no record of their involvement in the open source world. It talks about SOA and even provides a high level diagram (see picture) and attractive web shots.

But are they involved or is this a red herring? I dig and discover a DEFRA document from 29/09/2014 which does in fact talk about SITI AGRI's spatial rules engine and use in the RPA.

So why do I mention this? Well Bryan had also noted :-

'even with very few users, back-end servers would quickly reach 100% utilisation and “fall over”'

'core of the problem was identified as the interface between the mapping portal and the back-end rules engine software'

OK, so we can now take a guess that part of the RPA solution involved the graphical Kainos mapping solution providing the front end with a connection to the services layer of the Abaco 'Oracle' based spatial rules engine system.  This sent alarm bells ringing. 


Well, Abaco had a mapping tool and it claims to be web based - it even provides good looking screen shots. If this is the case, why not use it?

I'm clutching at straws here and this is into wild speculation based upon past experience but being able to provide a system through a browser doesn't mean it's designed to scale for web use, especially not 100,000+ farmers. Systems can easily be designed based upon internal consumption. There is a specific scenario called 'Lipstick on a Pig' where someone tries to add a digital front end to a back end not designed to cope with scale. It's usually a horror story.

Could this be what happened? A digital front end designed for scale attached to a back end that wasn't? That might explain the 'interface' and 'utilisation' rates and given an Oracle back end then I could easily believe the license fees and costs would be high.  However, it doesn't ring true and that's the problem with such speculation. Context.

GDS is full of highly experienced engineers. I can't see them commissioning a back end system that doesn't scale and trying to simply bolt on a digital front end. They would know that internal web based systems are rarely capable of scaling to public usage. This would have have had red flags all over it. 

Something else must have happened. As to what, we'll have to wait to find out.

Bryan also noted "Contrary to some reports, the £155m RPA system has not been scrapped entirely. According to the RPA 80% of farmers have already registered using existing online". This raises the question of what can be saved for future use? What has actually been lost? What is the cost?

At best we know that there was an original monumental IT disaster caused by customising an Oracle system with expensive consultants. It cost over £650m and had mounting fees. Certainly some lessons seem to have been learned by breaking the project into small components and using a broad supply base.

An approach of prototype and testing early with feedback seems to have been used but there is an issue about how this has been handled by RPA. If Ballard is correct then it might be worth explaining to Chief Execs of departments that if a prototype is still undergoing development then a) don't invite everyone b) don't run around claiming the sky has fallen on a prototype c) do communicate clearly.

However, there is no denying that something has obviously gone wrong with the current approach, though it's unclear how much of the £73m has been lost. However there are some questions I'd like to see asked.

1) What is the the actual cost spent and where did this go - license fees, consultants, other?
2) How much of what has been developed can be recovered for future or other use including other projects?
3) Was this system broken down into small components?
4) Was the interface between mapping system and the rules engine the cause of failure?
5) Is SITI AGRI the rules engine? 
** a) Were there delays as claimed?
** b) Was the rules engine designed for web scale?  
** d) If it wasn't designed for web scale then why did GDS commission the system for use on the web?
6) Did they adopt an open source route? Was there an open source alternative?
7) Who pulled the plug and why? 
** a) Should the plug have been pulled earlier?
8) Was this a case of 'Lipstick on a Pig?'

Of course, this is all speculation based upon a little digging and reading my own past experience of disasters into the current affair. What the actual story is we will have to wait to find out.

Since I'm in the mood for wild speculation, I'll also guess that despite the headline grabbing bylines - the Register will pretty much get everything wrong in terms of cost, the cause (number of suppliers) and its tirade.

Tuesday, March 17, 2015

In terms of strategy, WHY is irrelevant without WHERE

Think of a map in combat or a chessboard. There are two things that both instruments tell you.

First, is the position of things or WHERE things are in relation to other things. This Knight is on this part of the board next to the King etc. The troops are on this hill and the enemy is in the pass below.

The second thing they tell you is WHERE things can move to. We can move these troops positioned on the hill, down the hill to the south but we can't move the same troops north because there is a cliff which falls into the sea etc. We can move this Knight here or that Rook there.

A Wardley Map enables you to draw a line of the present (an existing business or organisation) on a landscape of position (value chain) vs movement (evolution). See figure 1.

Figure 1 - A MAP

On the map we have an hypothetical business represented by points A to B to C in a chain of needs. We can see the relationship between components. We know all these components will evolve due to supply and demand competition. We know we can manipulate this evolution (open efforts, patents, FUD, constraints etc). We know that as components evolve they can enable higher order systems to appear.

This is what we call Situational Awareness.

Hence, from your map (even a simple example like above) you can see multiple points WHERE you might attack. Do we want to commoditise a component? Do we want to drive up the value chain into creating a higher order system?

We're going to deep dive soon on some strategic gameplay and weak signal techniques but before we do, I need to make this bit very clear. The WHY of strategy is always a relative statement as in WHY here over there? Why move the rook here over there? Why move the troops down the hill over providing suppressing fire to the pass? Without the WHEREs then WHY becomes ... hand wavy ... and usually deteriorates into how, what and when (i.e. action statements).

You can usually tell competitors who have little to no situational awareness because they talk about the need to focus on why but when you ask them why are they attacking this space over another, they often get flummoxed, a bit hand wavy and quickly move into inspirational, vision and action statements -

"We're going to make the lives of cats better everywhere. It's part of our vision to become the best supplier of cat food. That's why we're building the internet of things for kittens!"

This is always useful to know. These people are potential fodder to your cannons and easy prey. They are your soft targets, the ones to take out first. With practice, you can often use inertia to get them to self implode and fight themselves.

BUT be warned. When asked the same question you need to reply in the same vague, hand wavy way. Don't give away information. Misinform. So do a bit of digging on your competitors before you attack. Just in case they know what they're doing. Chances are, you'll be fine.

Which is the other point of mapping. Don't just map yourself, map your competitors. If a competitor doesn't understand the game, don't hesitate to help yourself to their market. It's always easier to build by taking out the easy prey before you tackle the hard targets.

Monday, March 16, 2015

Continuous and Sustainable Competitive Advantage comes from Managing the Middle not the Ends

The mechanics of biological and economic systems are near identical because they're driven by the same force - competition. The terms we use to describe economic systems from evolution, punctuated equilibriums, ecosystems, red queen, cell based structures, co-evolution, componentisation, adaptive renewal cycle and so forth, all have their origins in biology. 

I'm going to use a few of these to argue why the future of continuous and sustainable competitive advantage comes from the middle not the ends of evolution.

First, we need to understand evolution. Evolution is not the same as diffusion. It describes how something evolves as opposed to how a particular instance of something diffuses (see Everett Rogers) in an ecosystem. 

Evolution is not predictable over time but it occurs over time. The best model I know of describing evolution is given in figure 1. It's based upon a pattern that was described in 2004 and primary research to determine why that pattern occurred between 2006 and 2007.

Figure 1 - Specific form of Evolution

It's the combination of supply and demand competition that causes things to evolve along a standard pathway. As things evolve their characteristics change. However, evolution doesn't just impact activities (the things we do). It also impacts practice (how we do things), data (how we record things) and even the mental models that we create (how we understand things).

The general form of evolution is given in figure 2 and in table 1, the characteristics of the different classes along with several general economic patterns are provided.

Figure 2 - General form of Evolution

Table 1 - Characteristics of General Form.

There are all sorts of subtle interaction (e.g. co-evolution of practice with activities, how to define ubiquity) but that's not the purpose of this post. What I want to concentrate on is the issue of evolutionary flow i.e. the competition that causes all things to evolve.

As a thing evolves then it enables higher order systems to appear through an effect known as componentisation (see Herbert Simon). For example, utility electricity provision enabled television, radio, electric lighting and digital computing. These news things exist in the unexplored territory, the uncharted space. They are about experimentation and exploration. They are only economically feasible because the underlying subsystems (i.e. the things they need, such as power) became more industrialised.

When this happens, visible user needs are no longer about the underlying subsystem but instead about the new higher order systems that have appeared. People only want electricity in order to power their computer, their TV, their radio and so forth. 

These higher order things are the visible user need, the underlying subsystem becomes increasingly buried, obscured and invisible. But any new higher order thing also evolves e.g. digital computing has evolved from its genesis to utility forms today. This in turn allows an even higher order of systems to be built.

When I use a cloud service, I don't care about the underlying components that the supplier might need (e.g. computing infrastructure). I care even less about the components below that (e.g. power).  I did care many years ago about things like servers and even power but today I care about what I can build with cloud services. Those underlying components are far removed from my visible user need today. Why? Because I'm competing against others who also use these services.

It is supply and demand competition that creates the evolutionary pressure which drives the novel to become the commodity, the uncharted to become the industrialised and the cycle to repeat. For activities that means genesis begets evolution begets genesis (see figure 3). For knowledge it means concept begets universally accepted theory begets concept.

Figure 3 - Genesis begets evolution begets Genesis

As an aside, this cycle has specific effects. It has three stages of peace, war and wonder due to its interaction with inertia to change. There are numerous forms of inertia and they act as what Allan Kelly described as a homomorphic force.  This cycle has a corollary in Hollings adaptive renewal cycle which is unsurprising given both are driven by competition. The war element of the cycle creates a punctuated equilibrium with the past due to network effects of competition i.e. the more people adapt the more pressure to adapt increases. The latter is known as the Red Queen Effect.

Of course, whenever you examine something it has a past, a present and a future. If I use a cloud based analytic service, I know it needs an underlying computing infrastructure, I know that computing infrastructure needs power and I know that the generators that make power need mechanical components. I also know that all these components evolved and were once novel and new. They each had their genesis. 

I've shown this constant evolution in figure 4. What's important to remember is the chain of needs (shown in a solid black line) is a line of the present. Those components evolved from the past and are evolving into the future (the dotted lines). The further you go down the chain, the more invisible the component becomes to the end user.

Figure 4 - Chain of Needs, The line of the present.

Being able to show the present on a landscape of what something is (the value chain) against how it is evolving (change i.e. past, present and future) is what mapping is all about. It's an essential activity for improving situational awareness, gameplay, operations and organisational learning.

It's key to understand that a map is a picture of the present in a dynamic landscape. All the components are evolving due to competition. That evolutionary flow affects everything. But if you know this, if you understand how evolution works, the change of characteristics, the requirement for different methodologies and common economic patterns, then you can exploit this. Figure 5 has an example map and a particular play known as Fool's mate.

Figure 5 - Map

With a map you can mitigate risks, obliterate alignment issues, improve communication and even examines flows of business revenue along with a host of other techniques. With enough maps you can remove duplication and bias in an organisation. 

You can also use maps to organise teams into cell structures and to solve the issues of aptitude and attitude i.e. we might have lots of engineering (an aptitude) but the type (an attitude) of engineering we need to create novel activities is different from the type of engineering we need to run an industrial service. This is a structure known as Pioneers, Settlers and Town Planners - a derivative of commandos, infantry and police from Robert X. Cringely's Accidental Empires, 1993.

You can use maps for scenario planning, for determining points of attack (the WHERE of strategy,  WHY is always a relative statement such as why here over there) and you can even create a profile of a company, an industry or a market i.e a frequency count of where components are along the evolutionary scale. 

Why bother with a profile? 

The components in the profile are all evolving from left to right (i.e. competition drives them to more industrialised). If nothing novel was ever created then it would all (assuming effective competition) become industrialised. You can use a profile to determine the balance of your organisation and whether you need more pioneers, more settlers or more town planners (see figure 6).

Figure 6 - Profile

To give you an idea of how to apply the three party structure. I've covered this before (many, many times) but this diagram will help.

Figure 7 - Applying PST (Pioneer, Settler and Town Planner) to a Map

As with the need for using different methods whether purchasing or project management (agile for genesis, lean for transition, six sigma for industrialised) then the roles of pioneers, settlers and town planners are different. So, is their culture. So is the way they organise. So are the type of people. See figure 8.

Figure 8 - Characteristics of Pioneer, Settler and Town Planner

Ok, but what has this to do with the middle?

The settlers and the transitional part of the profile, that is your middle.

This form of structure is the only way I know of sustainably solving the Salaman and Storey innovation paradox of 2002. This isn't "nice ideas", this came from competitive practice over a decade ago and subsequent re-use. Why it worked so well could only be explained afterwards in 2007 when the evolution curve went from hand waving pattern to something more concrete.

Of course, the techniques of mapping and evolution have evolved themselves over the last decade. Mostly this is by refining the terms used to make the language more transparent to others or through more adoption. However, every now and then something new appears and the latest addition is from a post by Dan Hushon on 'Digital Disruptions and Continous Transformation'. If you're familiar with mapping, I recommend you go and read now!

There's an awful lot to like in Dan's post. However, to explain why it's important, you need to first consider that a map can also be used to provide information on business revenue flows i.e. you can investigate a map to identify different business opportunities (see figure 9)

Figure 9 - Business Revenue Flow.

Some of those revenue streams will be profitable, others not. But even with a profitable revenue stream then this won't remain static. Take for example the pipeline of content in figure 5 above from commissioned shows to acquired formats i.e. the evolution of a new show like X-factor to a broadly repeated format show. The nature of the show has evolved.

With anything new (i.e. genesis) then you'd expect to add a high margin because of the risk taken. As that thing evolves and becomes more widespread and defined,  then you'd expect the margin per unit should be reduced and compensated with a much greater volume in a functioning marketplace.

In his post, Dan looks at this from the point of view of value vs evolution and overlays a PST (pioneer - settler - town planner) structure. I've slightly modified this but kept very close to the original in figure 10. This is not a graph of how things are, it's a concept that hasn't been tested yet. But evolution would suggest that this is how things should operate.

Figure 10 - Modified version of Dan Hushon's Graph

Why do I like this graph so much? Well, think of your organisation. You have core transactions you provide to others (in order to meet their needs). Those core transactions consist of underlying components (which can be mapped) that are evolving, These components effect the profitability of the transaction. The transaction itself is also evolving. As it evolves, the margin (or value) created by each unit of transaction should also change and reduce. Novel transactions should show high value per unit (and high margins). Commodity transactions should show low value per unit (and low margins) over time. 

You can use this graph to create a financial profile of your organisation and where your transactions are. They should all be evolving along the arrow. Of course, as transaction evolve to more commodity then you should be looking for those higher order systems to replace them. This can be used to create a financial pipeline for change and whilst the ends are important, it's the middle we really need to manage, the flow from one to another.

Ok, I get this but why is the middle so important?

If it wan't clear, situational awareness is key to a vast number of aspects of managing an organisation and the ONLY way I've found to significantly improve situational awareness is to map and compare the present against evolution. 

Evolution shows us that we're living in a dynamic environment with the constant flow of change from the novel to the industrialised. This impacts everything from project management to purchasing to team structure to finance.

Whilst there are two extremes (genesis vs commodity) which require polar opposites in methods, culture and structure (Salaman and Storey, Innovation paradox, 2002) and everything (activities, practice, data and knowledge) in a competitive market (with demand and supply competition) evolves from one extreme to another ... there is a constant. That constant is change. The middle represents and governs this evolutionary flow from one extreme to another.

Yes, it's important to manage the extremes but both extremes can be outsourced either to suppliers or through the use of an ecosystem model, such as ILC.  Some firms will be able to hide in the relative safety of the industrialised space using ecosystem models to sense the future. Some will try and survive in the high risk space of constant genesis. For most of us then the one thing you need to manage above anything else is the flow between the extremes. This is where all the tactical plays matter. This is where situational awareness is critical. For most us, this is where continuous and sustainable advantage can be gained.

If you're going to build a two mode structure (bimodal, two speed IT, dual operating system etc) then focus on creating public APIs for utility services and get every other company to build on top of it. Let everyone else be your R&D labs and your pioneers. Hide in the industrialised spaces and use ecosystems as your future sensing engines. Remove pioneering internally through the use of a press release process. Use the skills of the settler to mine the ecosystem for new successful changes which you then industrialise with town planners to commodity components or utility services. Alas, there's only a limited number of firms who can play that game.

This is why I'm not a fan of bimodal, dual operating system, two IT speed concepts in general. Most of us have to cope with the entire spectrum of evolution. These two mode methods appear to focus the mind too much on the extremes. It doesn't matter if they've included the all important middle (the transitional part, the settlers) in one group or another. You've just buried that which truly matters.

The Settlers (like the infantry) are key. They make success happen. They control your destiny. They make the entire process sustainable. They play the most important games. They are where open source becomes a weapon. They are mostly ignored in favour of the extremes. Let us "take the pig and lipstick it" seems to be the mantra. Bolt on a digital side to our legacy and ignore that the digital will itself become legacy. What do we call the legacy then? Legacy Legacy with Legacy Digital and ... let us add some more lipstick?

Organising by extremes creates two opposing camps, conflict and bottlenecks are inevitable and something I've seen over the last decade. Yes, these two mode methods may give you a short term boost in terms of efficiency and innovation but it's the sustainability issue that's the problem. That's why I strongly suspect in a decades time, the same people telling you to become a two mode type organisation will be flogging you a three mode solution to your new two mode problems. Save yourself the trouble and another round of re-organisation.

On the upside, all the mapping stuff (2005 onwards), all the profile work, flow, the PST structure and ILC models (2007 onwards) are creative commons share alike. It's all scattered throughout this blog and has been presented at hundreds of conferences between 2007 to 2014.

That's an invitation to help yourself.

Improve your situational awareness. Learn to play the game.

You'll soon discover why the "middle" is the all important battleground.

Oh, and on the question of bias ... am I biased towards mapping? Yes! I've been using mapping to outplay other companies in many commercial markets for over a decade and more recently in the last four years within Governments. I'm as biased as you can get. In my world, situational awareness helps. Yes, it's also true that maps are complex. In some cases, you can spend a whole day to create a first map that you can use.

--- 18th March 2015

Added figure 7 just so people can make the jump between PST and a map.

Sunday, March 15, 2015

Two Speed Business? Feels more like inertia.

These days I use maps of organisations to apply a cell based structure ideally using a Pioneer, Settler, Town Planner model.  The map is based upon a value chain (describes the organisation), how things evolves (describes change) and the changing characteristics of activities, practices and data from two extremes (the uncharted to the industrialised) - see figure 1.

Figure 1 - PST.

I use these maps across many business and governments to reduce duplication, bias & risk whilst improving situational awareness, gameplay, correct use of multiple methods & communication - see figure 2.

Figure 2 - Map of a Media Company.

Key to mapping is the evolution axis i.e. you can't actually anticipate change over time in any effective manner (we don't have a crystal ball). See figure 3.

Figure 3 - Evolution

A bit of HISTORY.

My early attempts to map an environment were a disaster because I attempted to use time and time based sequences to measure change. I eventually postulated a path for how things evolved and presented this at Euro Foo 2004, EuroOSCON 2006 and many conferences in between. - see figure 4.

Figure 4 - Early Evolution [from EuroOSCON 2006].

This path is what I used to create my early maps in 2005 (though James A. Duncan tells me it was 2004). The problem was, I couldn't demonstrate the path was based upon anything other than a concept. It took until 2007, to collect the data necessary to demonstrate that the evolution path (see figure 3 above) had some merit.

In those early years, I also used to refer to how different methodologies were required e.g. figure 5.

Figure 5 - Different Methodologies [from EuroOSCON 2006]

This was based upon the innovation paradox of Storey & Salaman, Theories about the process of innovation, 2002

”paradox is at the heart of innovation. The pressing need for survival in the short term requires efficient exploration of current competencies and requires ‘coherence, coordination and stability’; whereas exploration / innovation requires the discovery and development of new competencies and this requires the loosening and replacement of these erstwhile virtues”

But it wasn't until 2007 that I could clearly demonstrate the changing characteristics of activities, practices, data and knowledge as it evolved. This I have spoken about at literally hundreds of conferences and video talks to hundreds of thousands of people around the world using various examples (e.g. figure 6 and figure 7)

Figure 6 - Characteristics of Innovation (as in a novel thing) [from Web Strategies 2008].

Figure 7 - Characteristics of the same activity as it evolves to commodity [from Web Strategies 2008].

This also helped explain why the Pioneer, Settler (or what I used to call Colonist) and Town Planner structure worked. I had introduced this structure many years earlier (2005) to replace a failing two bit mode model before it - see figure 8.

Figure 8 - PST [from Web Strategies 2008].

The Pioneer, Settler and Town Planner model is a derivative from Robert X. Cringely's model of Commandos, Infantry and Police in the 1993 book, Accidental Empires.

Of course, some of the terms have evolved over the years to make them more meaningful whilst the basics remain as is.
  • I don't use Colonist anymore. I use Pioneer, Settler and Town Planner.
  • I don't use innovation anymore. I use the term genesis (of a novel and new thing) because innovation is so abused to mean everything.
  • I don't use chaotic anymore. I use uncharted to describe the unknown space.
  • I don't use linear anymore. I use industrialised to describe the defined space.
E.g. take a look at older presentations and you'll see many of the same concepts just with different terms.

OSCON 2010

Today, what amazes me is that companies are starting to realise that one size methods don't work. I hear talk of bimodal, two speed IT and dual operating systems for a company.

Why does this amaze me? 

There are a few exceptional leading lights from very early on - Robert Strassmann and Robert Cringely come to mind - but the leading edge of companies were exploring these topics around 2002-2008.  I know, I was there helping the discussion to happen. After this, the early adopters started to appear (Silicon Valley Startups etc). Increasingly I see this work in various Governments.

However, all of a sudden in 2014 - an old model, not cognisant of how things evolve, long thought dead has raised its head. The bimodal, two speed IT, dual operating system is becoming a very popular concept within what I describe as legacy companies. We're talking ten years late to the party and getting the wrong address. 

This feels like inertia. A hope that adding lipstick to the pig will somehow keep the pig alive.

A warning

Mapping, use of a three party system, the extremes of uncharted to industrialised are not new. Despite refinement to the terms, this stuff is all nearing or over a decade old. The field has moved on since then into understanding common economic patterns, weak signal analysis and other forms of organisational learning.

By the time something of use gets written in a book or a publication, the value related to it has often been extracted long before. You can't get to the bleeding edge of competition by reading books or publications. You have to live it. Even models like pioneers, settler and town planner are starting to feel dated and being examined through lenses of a continuous spectrum and other structures.

The laggards are nowhere near getting to the spaces where the leading edge have long since moved on from. 

The gap appears to be widening.

I used to think the gap between the leading edge and laggards was five to seven years, it now seems like fifteen or possibly much more (i.e. how long it will take the laggards to catch up with where the leading edge is today). This may be a normal consequence of inertia, an acceleration in the underlying cycle of change combined with poor situational awareness - of this, I'm not sure. It needs testing.

However, within the laggards it does feel as though the copying of easy to digest memes appears to be more rampant, circulated by those who benefit from the memes to those that need an 'easy solution' in a marriage of convenience. As Rosenzweig once said

“Managers are busy people, under enormous pressure to deliver higher revenues, greater profits and ever larger returns for shareholders.They naturally search for ready-made answers, for tidy plug-and-play solutions that might give them a leg up on their rivals. And the people who write business books – consultants and business school professors and strategy gurus – are happy to oblige.”

I see an awful lot of companies plunging down paths they shouldn't with little or no situational awareness. This isn't getting better for some. There's going to be an awful lot of casualties. 

Saturday, March 14, 2015

A long journey ... but a happy one ... Karma is kind.

Seven years ago, I wrote the presentation below (one of many). I presented it at several large companies and then at a public Strategy conference in April 2008 to around 600 people. I got into several arguments.

So, how have things progressed in the last 7 years? Well, during the talk, I covered a subset of my research including :-
  • The cycle of commoditisation
  • The evolution curve (ubiquity vs certainty)
  • Evolution vs Diffusion
  • Three layers of IT shifting from product to utility
  • Why we had no choice (pressure for adoption, Red Queen)
  • Accelerators to evolution (network effects, open source, standards)
  • Polar extremes of an organisation (innovation to commodity)
  • Properties of those polar extremes
  • Why one size methods don't work and the need for different methods  (Agile and Six Sigma)
  • How to organise for these two extremes (Pioneers, Settlers and Town Planners)
  • Different investment / competition effects as activities evolve
  • The importance of a maps (though I didn't cover how to map).
  • The issue of bias
  • Creative destruction
  • The innovation paradox
  • Componentisation
  • How reducing non strategic cost is both friend (operational efficiency) and foe (reducing barriers to entry)

I also provided Four Predictions
  • An increase in the rate at which innovative services and products are released to the web.
  • More disruption in the information markets.
  • Increasing organisational pressure within IT.
  • The architecture will never be completed.

7 years later ... the predictions all hold and every item on the presentation is still relevant. 

In 2007 to early 2008, I was funding my own full time research. I burned most of my own life's savings in order to understand why the experiments which I had undertaken in Fotango during 2002-2006 worked. Of course, I made the concepts all creative commons - I work on a principle of Karma. 

Naturally the work has evolved since then - I've tidied up terms, made a few adjustments - but in principle it's all the same. There has however been some important moments of change. Whilst I used the mapping technique in Fotango (2005) and Canonical (2008), it wasn't until a good friend of mine - Liam Maxwell, who I had worked with (giving my time freely) on the 'Better for Less' paper which included many of my concepts - persuaded me to talk more openly about the topic did I gain the confidence to speak publicly on How to Map.

Despite my public speaking, confidence was an issue.

Back in 2007, in most business events then the ideas I expressed usually created a hostile reaction from  fruitcake, idiot, ridiculous, nonsense, delusional, sad waste of time, rubbish to gibberish. Hence the arguments. I'm not shy of a fight especially when it gets personal.

Back then it was all one size fits all, our method will solve everything, few understood evolution and even basic ideas of competition were rare. This does wonders for your confidence if you've just spent your life savings investigating something and the majority tell you that you've got it all wrong. I hoped that this was just an educational issue. I had to push slowly at the door.

But I'm made of stern stuff and so I continued. As more understood then I exposed more of the underlying concepts. Eventually, I had to find a job. Karma was kind and Canonical found me because of a discussion I had with Mark Shuttleworth on evolution. Even though the first five employees I met at Canonical in 2008 told me the 'cloud was just a fad', it changed. Today Canonical dominates the space.

These days all the topics are either considered conventional wisdom or well on the path to such. Even if companies do go off on the wrong track (e.g. two speed IT, bimodal) then I know that eventually they'll work it out. 

Today, I often listen to people speak at conferences and talk about the ideas that I once presented. The hostility is gone. The nodding of heads is commonplace. Knowledge also evolves but then I knew that. Everything evolves.

These days, any arguments are about who created the ideas first. Usually some consultancy firm is claiming them. This makes me smile. Most of the ideas come from long dead economists. I like long dead economists, they work cheap and don't grumble.

It's marvellous to see how things have changed.

Today, I work at the LEF where many governments and companies now fund me to research into even more marvellous areas. Had I never taken the risk, spent my life savings and given the concepts all away then that would have never happened. I'd never had worked for Canonical. I'd never have written the 'Better for Less' paper. I wouldn't have met the good friends I have today.

Karma is kind. 


The slides are just an aid to the talk. At some point I'll record the video again (I have the original text) and it'll make more sense but the slides are enough for now.

Friday, March 13, 2015

On Pioneers, Settlers, Town Planners and Theft.

I often talk about the use of cell based structures (e.g. think Amazon Two Pizza, Starfish model) which are populated not only with aptitude (the skill to do something) but the right attitude (type of people). A map is an essential part of building such a structure (see figure 1).

Figure 1 - Pioneers, Settlers and Town Planners.

The concept of Pioneers, Settlers and Town Planners is a derivative of Robert X. Cringely's idea of Commanders, Infantry and Police as expressed in the delightful 1993 book - Accidental Empires. The first time I used this structure was around 2005-2006.

Pioneers are brilliant people. They are able to explore never before discovered concepts, the uncharted land. They show you wonder but they fail a lot. Half the time the thing doesn't work properly. You wouldn't trust what they build. They create 'crazy' ideas. Their type of innovation is what we call core research. They make future success possible. Most of the time we look at them and go "what?", "I don't understand?" and "is that magic?". In the past, we often burnt them at the stake. They built the first ever electric source (the Parthian Battery, 400AD) and the first ever digital computer (Z3, 1943).

Settlers are brilliant people. They can turn the half baked thing into something useful for a larger audience. They build trust. They build understanding. They make the possible future actually happen. They turn the prototype into a product, make it manufacturable, listen to customers and turn it profitable. Their innovation is what we tend to think of as applied research and differentiation. They built the first ever computer products (e.g. IBM 650 and onwards), the first generators (Hippolyte Pixii, Siemens Generators). 

Town Planners are brilliant people. They are able to take something and industrialise it taking advantage of economies of scale. This requires immense skill. You trust what they build. They find ways to make things faster, better, smaller, more efficient, more economic and good enough. They build the services that pioneers build upon. Their type of innovation is industrial research. They take something that exists and turn it into a commodity or a utility (e.g. with Electricity, then Edison, Tesla and Westinghouse). They are the industrial giants we depend upon.

What you want is brilliant people in each of these roles.

How do you get things going within a company?

To Start
Determine (by mapping) an activity that is currently expressed as a product but is ready for industrialisation to a commodity and even better a utility service.  It needs to be suitable, widespread and well defined enough, the underlying technology should exist and there should be a frustration in the market with the current provision.

Now, find some town planners to build the more industrialised form for you (remember they are brilliant, smart and won't be cheap). 

You'll now need some Pioneers (genesis to custom)

The pioneers create the entirely novel and new (genesis) by consuming (building on top of) those industrialised components (it reduces cost of failure, increases speed etc). Ideally the pioneers are not just within your company, but more favourably contain outside companies building on top of your component. Hence, it's a really good idea to expose your newly industrialised form via a public API.

The more pioneers there are (both inside and outside) the bigger the ecosystem around your component is. This will not only give you greater economies of scale but you can mine the ecosystem to identify future success (it operates as a future sensing engine). If you don't know how, read up on the ILC model here.

Often what the pioneers produce is something just beyond a prototype (more towards a MVP). Often it has bugs or some other failings but it is useful. Within a company, you can incentivise internal pioneers on the creation of new things that are eventually turned into new products & services by use of an internal royalty.

Now, you need some Settlers (custom to product and rental services).

The job of the settlers is to identify common patterns in the ecosystem (whether just internal pioneers or internal & external).  This can be done by leveraging consumption data of the underlying components or simply inspecting a range of new activities for common elements or simply taking something an internal pioneer developed.  Once a pattern or activity is identified, the job of the settlers is to turn it into some sort of product i.e. they steal from pioneers and productise it (make it manufacturable, documented, profitable, stable etc). You can incentivise settlers by product profitability and by which products make it to utility services.

A key role of the settlers is to steal from the pioneers who are in effect acting as the settlers R&D centre. Sometimes an internal project is going nowhere and the settlers will steal it, replace with something from the outside market. There is a problem here in that settlers can sometimes want to keep something going for too long. Also if you're using an ecosystem to identify future success then remember, you're are the gardener of that ecosystem. If you harvest too much, it'll die off. So think carefully - you need to harvest and nurture. 

Now you need some ... wait ... you've already got these ... Town Planners (product / commodity and utility services)

The job of the town planner is to build the core, volume operations based, good enough, ultimately (long term) low margin but highly industrialised services & commodity components. They use the portfolio of the settlers, any consumption data, comparison to outside to determine whether something is suitable. Once identified they build the core service and inform the other groups (e.g. you tell pioneers so they have something to build upon, settlers so they know that this part of the portfolio is being industrialised). You incentivise town planners on volume and breadth of service offerings. You actively encourage them to cannibalise your own business as fast as possible.

By creating this virtuous cycle which is incentivised so that each group steals the work of the former - town planners stealing from settlers who steal from pioneers who build on the work of town planners - you can accelerate innovation, customer focus and efficiency all at the same time and remove the threat of inertia to change.

Some things to note.

Each group innovates but innovation is not the same for each group i.e. the innovation of an entirely new activity is different to the feature differentiation of a product which is different from converting a product to a utility service. Unfortunately, despite being different forms of innovation that won't stop people pretending there's only one and it's all the same. Try not to do this.

Each group has different attitudes though aptitudes (e.g. broad skill description such as finance, engineering or marketing) might be the same. Engineering in the pioneering group is not the same as engineering in the town planners.

Each group has a different culture. I hear endless stuff about company culture. Go ask special forces, infantry and police whether they have the same culture or not.

On Disruption
  • Pioneers don't disrupt. There is nothing to disrupt.
  • Settlers sometimes disrupt (as in product to product substitution in an industry). This is unpredictable.
  • Town Planners often disrupt past industries. This can be anticipated quite a time in advance.
  • There are two forms of disruption (predictable and non predictable) but that doesn't stop people pretending there is one.

Some Exceptional Models

Whilst building two party systems e.g. pioneers and town planners usually cause huge headaches, there is one exception.

IF (and only IF) ...
  • You are using an outside ecosystem to act as your pioneers (e.g. providing utility services through public APIs and mining consumption data)
  • You introduce some form of forcing function to prevent novel activities being created internally e.g. introducing a requirement to create a press release before writing a business plan or prototyping anything. [Hint : people can't write press releases for novel, uncertain activities which haven't been explored yet]

Then in these exceptional circumstances you can build a settler and town planner structure by outsourcing your pioneering capability to the outside world. You're really still three party but you've outsourced one part.

Whilst one party systems e.g. one size fits all usually cause huge headaches, there is one exception.

IF (and only IF) ...
  • You are using an outside ecosystem (consumption) to act as your pioneers
  • You are using an outside ecosystem (supply) to act as your town planners
Then in these exceptional circumstances you can build a settler structure by outsourcing your pioneering capability to the outside world along with the component supply chain. In this type of structure you purely manage and exploit the flow from the uncharted to industrialised. You're still really a three party structure just you've outsourced two of the parts.

Thursday, March 12, 2015

Wardley Map - a set of useful posts.

Ok, for those who want to learn how to map, I've provided a few links to what I consider useful posts.

A quick video (speaking is my preferred medium)

A longer and more detailed version.

Various Posts.

NB Some use the older terms Chaotic & Linear which I changed to Uncharted & Industrialised (more apt). I've put this list here (there's about 950+ posts on this blog). I'll try and add more, tidy things up and you never know ... persuade someone else to turn it into something readable.

An introduction to Wardley Maps
A step guide to mapping
On creating a value chain
Getting stuff done.
A guide to mapping.
From strategy to mapping to pioneers (slides)
The good bits about mapping
The amazing bits of mapping
The wow of mapping

Properties of evolution
Componentisation II
There are many chasms to cross
The Two Extremes
Oh, no Six Sigma vs Agile
Early failures
When to use a curve
On services

Of Perils and Alignment.
Company Age
On Ecosystems and Porter.
Punctuated Equilibriums
On the two forms of disruption
What's right and wrong with Christensen.
On evolution, disruption and the pace of change.
Does maturity matter
Ten graphs on organisational warfare.

Basics of operation
Let the Towers Burn
Government, Purchasing
How we used to organise ourselves
Pioneers, Settlers and Town Planners
(another post on PST)
Two speed IT / Bimodal.
Other Tools I use with Mapping
Business Model Canvas ... the end of a long journey
Maps and the Target Operating Model.
Maps are imperfect but that's ok.
Rough guide - use cloud, build cloud and microservices.
On maps, components and markets
This is not the data you are looking for.
Why no consultants.

Open Plays
Open source, gameplay and cloud
Scenario planning
Strategy vs Action
On disruption and executive failure.
Epic Fails of Sensible CEOs
Tower and Moat.
On Chess and Business
Preparing for War
Dungeons and Dragons vs The art of Business.
On D&D and Ant Battles.
Quick route to building a strategy.
Four basic smackdowns on competition
Does commoditisation lead to centralisation.
Fast Follower Conundrum.
Attack, defend and the Dark Arts.
Self disruption and super linear.
On the death of great companies.
The interesting thing about cutting costs.
Jevons in a nutshell

General Stuff
A useful summary post
Project, Products, Open source and Proprietary.
Context, situation and components.
Is war the mother of invention?
The abuse of innovation
Half completed book

Final Last Few
Continuous and Sustainable Competitive Advantage comes from Managing the Middle not the Ends
Two Speed Business? Feels more like inertia.
On Pioneers, Settlers, Town Planners and Theft.

There's also the team at WardleyMaps (I'm not affiliated with) who are trying to turn my long and sometimes rambling concepts into something readable.

For reference, all my writings are creative commons 3.0 share alike licensed, as is the entire mapping concept, maps and evolution diagrams.

Sunday, March 08, 2015

Evolution, diffusion, hype cycle and early failures.

With mapping, one of the axes is evolution (the other being value chain - I've written an introduction on Wardley Mapping which covers the basics.).

One axis describes what something is (the value chain, from the user needs to the components required to meet it through a chain of needs). The other axis (evolution) describes change.

I've posted before on how the evolution axis was determined but when exploring this subject, I didn't start with the evolution axis as I had to discover it. In the very early days (2000 - 2004), I tried all sorts of different forms of measuring change. All failed.

The evolution axis (see figure 1) has a number of specific properties. It is :-

Figure 1 - Evolution

1) Universal. It appears to apply to all things that undergo competition whether activities, practice, data or knowledge. 

2) Causation. It is caused by the interaction of demand and supply competition. Without competition then it doesn't occur. It also doesn't require the need for a crystal ball as time is abolished from the axis. Evolution is measured over ubiquity vs certainty.

3) Measurable. The evolution of any component can be measured using a combination of publication types and how widespread the component is in a field. Unfortunately, only the past can accurately be measured which means the component has to become defined on the certainty axis (i.e. a commodity) before its past can be fully described. Until that point, there is an increasing element of uncertainty with position as the component becomes more uncertain (i.e. unknown).

We have no crystal ball and cannot predict precisely when things will evolve (I had to abolish time to create the evolution curve) however we can test evolution through the measurement of the past and secondary effects (i.e. other consequences caused by evolution). We can also use weak signals to anticipate evolutionary change.

4) Usefulness. There is a long list of reasons why mapping across the evolution axis is useful.

5) Consistency. There is only one evolution path. The evolution of one activity can be directly overlaid on the evolution of another. There is no need for a vague 'time' axis where 'time' is not a measurement but a direction of travel. 

6) Direction. A thing evolves in a single direction, i.e. the past is behind it, the future is ahead. It doesn't start evolving, become a product and then suddenly jump back into genesis as an unknown thing. When examining complex products, what we find is objects like a 'smart phone' are actually a grouping of many underlying components.

7) Recursive. Evolution can be equally applied at a high level (e.g. a specific business activity) or at a low level (e.g. a component of an IT system).

The early days

Before discovering the process of evolution, I did try to map with all sorts of things. These were inevitably failures. I thought I'd go through a couple and explain why.

Diffusion Curves.

Diffusion Curves (as per Everett Rogers) are extremely useful in many contexts. They are measurable (adoption vs time) but unfortunately the time span is inconsistent between different diffusion curves. If you overlay multiple diffusion curves on exactly the same time axis then you get multiple different curves. They also lack direction i.e. the diffusion of a particular phone A1 (from early adopters to laggards) is followed by the diffusion of a better phone A2 (from early adopters to laggards) which is followed by the diffusion of an even better phone A3 (from early adopters to laggard).  ... and so on. Hence the evolution of an activity is seen as a constant set of jumps back and forth rather than a direction of travel and the future of an act can be both ahead and behind it on the map. I've compared a map based on evolution (see figure 2)  with a map based upon diffusion (see figure 3).

Figure 2 - Map based upon evolution, direction.

Figure 3 - Map based upon diffusion, direction.

Hence from a mapping perspective, this makes diffusion curves useless. However, don't confuse that with diffusion curves being useless, in other contexts they are extremely useful.

Hype Cycles

Hype cycles are extremely useful in many contexts. They are not however based upon any physical measurement but instead are aggregated opinion. This means they are not testable, so we just have to accept them at face value (even when they do change the axis from visibility to expectations). The other axis is time (though it used to be maturity).  Time, in this case, is more of a vague direction of travel rather than any measurement of time. It also suffers from a lack of direction in the same way diffusion curves do. 

In figure 4, I've added four different hype cycles from 2002, 2005, 2010 and 2013. First thing is the axes have changed from visibility vs maturity to expectation vs time whilst the curve has remained the same. I even have a couple with expectation vs maturity.  This doesn't matter. The hype cycle isn't based upon measurement of some physical property, it's based upon opinion, so this is fine.

However, we have software as a service / ASP being in the slope of enlightenment in 2005 and cloud computing being in the peak of inflated expectations in 2010. Now, this all depends upon opinion and whatever definition they wish to choose. But, as with diffusion there's no direction i.e. one thing doesn't evolve into another but rather one thing in the slope of enlightenment becomes another related thing in the peak of inflated expectations. So when it comes to mapping rather than compute (as a product) evolving to include compute (as a rental service) then evolving to compute (as a commodity) including compute (as a utility), with the hype cycle you have the same back and forth that you have with diffusion curves.

Figure 4 - Four Hype Cycles

As I found out long ago, the hype cycle in the context of mapping is useless (as with diffusion curves). Don't confuse that with the hype cycle being useless itself. In many contexts as a source of aggregated opinion, then it is very useful. 

I looked at many techniques to measure change and found all of them wanting. I spent years finding out that lots of things weren't useful for describing evolution. This is why I spent so long in the British Library cataloguing many thousands of publications. There was no effective means of describing the process of evolution until I'd done this work and found a process that seemed to work.

But then that's the point, diffusion curves measure diffusion, hype cycles provide an aggregated opinion on hype, whilst the evolution curve measures evolution (it doesn't measure either diffusion or hype). All have their purpose.

When it comes to mapping and improving situational awareness then you need to focus on what an organisation is (the value chain) and how the components of the value chain are changing (evolution). Does this mean the evolution curve is right? No, of course not. It's just the best model at this moment in time for examining evolution and as such it's an essential part of mapping.

--- 13th March 2015

Based upon Henry's question (below), I've added a graph to show the link between diffusion curves (Adoption vs Time) and Ubiquity.

The first graph shows adoption curves for different instances of an evolving activity A [through various instances A1 to A6], but rather than simply add adoption as the Y axis, I've added applicable market. The reason for this is adoption is to an applicable market and the market sizes for each evolving act are different.

Figure 5 - Diffusion Curves (adoption vs time). 

I've then added where each evolved instance of the act A is in the evolution curve.

Figure 6 - Evolution

[The two graphs above are purely illustrative. I've taken a few liberties to make the transition less complex.] 

Now, a couple of things are worth noting.

Ubiquity is a measure of how ubiquitous something is. In order to determine it, you need to know the end point i.e. when the activity has become a well established commodity. This point of ubiquity can only be accurately estimated when the act has become certain. At this point the past history can be graphed.

I have to emphasise, that the accuracy at which you can determine where something is or where something was on the graph increases as the act becomes more certain i.e. a commodity. If the act is not a commodity then where it currently is on the ubiquity axis becomes increasingly uncertain the newer the act is.

For this reason, you cannot when something novel appears (e.g. the genesis of electricity with the Parthian Battery in 400 AD) determine how ubiquitous electricity will become some 1600 years later. No-one can. This requires a crystal ball.  However by 1960s, you've a pretty good idea how ubiquitous electricity is and can graph the past.

The certainty axis is determined from publication types (see figure 7). In particular, it is a relative measure of type II and type III publication types.

Figure 7 Publication types.

To graph the past, you need to first use the publication types (used to create the certainty axis) to determine if something has become an established commodity. You can then determine the point of ubiquity (what I call the reference point) and use this to determine how ubiquitous something was. You can then plot ubiquity against certainty in the past. An example of this covering a range of entirely different activities is provided in figure 8.

Figure 8 - Ubiquity vs Certainty.

You can then overlay the different areas where different states (custom built, product etc) dominate which gives figure 1 - the evolution curve at the beginning of the post.

To graph the future, you can't. The best you can do is to make a best guess at to where something is on the curve. Which is why with mapping I use broad categories - genesis, custom built, product etc. I get groups of people (with experience of the field) to estimate how far along it is. The accuracy of this will increase as the act becomes more certain.

To see evolution then you need to abolish time from graph (i.e. there is NO way to measure evolution over time in any form of repeatable pattern). However, whilst things will evolve over some (unspecified length of) time, you cannot accurately demonstrate where it is the evolution curve until it has become a commodity. Then you can accurately demonstrate where it was.

I cannot emphasise this more. The future is an UNCERTAINTY barrier which we cannot peek through. The more UNCERTAIN something is (i.e. genesis, custom built) the less able we are to see the end state. ONLY when something is close to being CERTAIN can we see the point of ubiquity (the reference point).

I do see people draw diffusion curves and start claiming things about them like here is a commodity, this end is an innovation. Alas, there is no repeatable graph of evolution over time.

We do NOT have a crystal ball. No-one does.

Diffusion <> Evolution. Evolution consists of many diffusion curves and there are many chasms to cross. The two are not the same, mix this up and you'll get lost.

Despite lacking any crystal ball, we can however say that something will evolve from genesis, custom built to product (+rental services) to commodity (+utility services). It will become more common and more certain. This change is driven by supply and demand competition. This is what figure 1 shows.

--- Just for Fun (13th March 2015)

Oh, and just because I like a good debate. I'll leave you with this graph of progress. Whilst being efficient with energy is good, anyone who thinks we can somehow reduce energy consumption permanently whilst maintaining progress needs to think again. Whatever we do, eventually if progress continues our energy consumption will exceed today. If you want to solve the energy related issues then you need to look for less damaging forms of production. Competition (whether life or business) is a constant exercise of reducing entropy within the local system through the consumption of energy.

Ubiquity vs Adoption (16th March 2015)

I thought this was fairly obvious but it seems there is still some confusion. I need to clarify.

100% Adoption of the iPhone 1 <> 100% Adoption of the iPhone 6. The markets are different.

Q. But can't we look at the smartphone market? 

Well, what do you measure against?

Q. When it's ubiquitous?

Define ubiquity

Q. When everyone has one?

So, gold bars aren't ubiquitous because not everyone has one? Gilts aren't ubiquitous because not everyone has one? CRM systems aren't ubiquitous because not everyone has one? Coffee beans aren't ubiquitous because not everyone has one? Computer servers are not ubiquitous because not everyone has one?

Except ... Gold bars, Gilts, CRM, Coffee beans and computer servers are all ubiquitous in their markets. It's just their markets are very different.

Q. But how do you convert adoption to ubiquity?

You need to find the point of ubiquity in a market. To do this you need to look at publication types and work out when something is a commodity. Then you can find the point of ubiquity and plot back how ubiquitous something was vs how certain (i.e. well defined, well understood) something was.

You cannot simply take adoption of something in its market and call that ubiquity. You cannot jump from an Everett Roger's diffusion curve to an Evolution curve. Just because they both happen to have an S-Curve shape doesn't make them the same thing.

Q. But that's what everyone does!

Well, they are all wrong.