Friday, February 18, 2011

Deconstructing Gartner's Hype Cycle

This is a piece of work that I did many years ago but given my recent post on Lifecycle (nee evolution). I thought I'd revisit it. I will assume the reader is entirely familiar with the concepts of commoditisation.

In figure 1, I've taken part of the evolution curve and modeled onto it a differential benefit curve (differential value - cost of implementation). This latter curve shows how the benefit of an activity changes as it evolves from its early innovation (where it is a strain on company resources) to a late product stage where the activity is ubiquitous and of insignificant differential value between competitors.

Figure 1 - Lifecycle & Differential Benefit (click on image for higher resolution)

When a new activity appears, you often get whitepapers written about it. These are generally done at a time when the activity is showing a highly positive differential benefit. Obviously there is a delay between the collection of data, the publication of the whitepaper, a user reading the whitepaper, the decision to do something and then implementation of the activity.

By the time an activity is implemented, the actual differential benefit may be vastly different. This creates a delta for expectation i.e. a difference from what we thought we would get and what we got. Figure 2 provides a graphical notation of this.

Figure 2 - Delta in Expectation (click on image for higher resolution)

Modelling this delta, with an awful lot of assumptions, provides the expectation curve shown in figure 3. There are a complex set of assumptions and conditions around this, however for the purpose of this post then I'm happy to say this curve is somewhat valid ... caveat, caveat ... except where it's not :-)

For the sake of this post, let's just pretend it is. What the curve shows is the early stages start with low expectations (but possibly high hopes), expectations are then quickly exceeded and continuously rise to reach a plateau after which expectations rapidly become unfulfilled leading to a trough of disillusionment before eventually levelling.

Figure 3 - Expectation Curve (click on image for higher resolution)

This all sounds strangely familiar and so it should. The expectation curve quite neatly maps to Gartner's hype cycle - see figure 4. So, we have a potential basis for explaining the underlying forces behind the hype cycle except of course, we have all the assumptions and I'm talking about expectation and not visibility. I'm not even sure what visibility actually means but given it's a hype cycle, I'll assume high hopes (expectations) means high visibility. Clocking up another assumption here.

Figure 4 - Hype Cycle & Expectation Curve (click on image for higher resolution)

Does our new underlying "basis" of the hype cycle shed any new light on the subject? Well, yes. However before I show this, I'd like to examine the final stages of lifecycle.

The evolution of an activity from products to utility services invokes its own expectation curve not through differential value (the creation of a new activity) but operational efficiency (a more efficient means of providing an existing activity).

In figure 5, I've provided the later stages of lifecycle including the transition from products to utility services and modeled an operational benefit curve (operational efficiencies over competitors - cost) of a transition to utility services. Again, lots of assumptions.

Figure 5 - Lifecycle and Operational Benefit (click on image for higher resolution)

NB. this benefit curve is the same shape as the earlier differential curve being derived from the benefit created over competitors, number of competitors exploiting the change and a changing cost of implementation due to maturity. Hence the transition to utility services starts with a period of investment, a rapid benefit over competitors gained by those creating or exploiting such services and then a decline in benefit over competitors as more companies switch to a utility model.

The reason why I mention this, is that whilst Cloud Computing is all about volume operations for ubiquitous and well defined activities (i.e. use of computer resources in business) and is hence all about commodities, this transition will create a similar expectation curve around operational efficiency in much the same way that a genuine innovation creates an expectation curve around differential value. This is shown in figure 6, and the result is the same delta in expectation curve shown beforehand.

Figure 6 - Delta in Expectation (click on image for higher resolution)

Our first "insight" is Gartner's hype cycle tends to show both innovative activities and transitional effects on the same graph. We should be careful to distinguish between operational efficiency (doing the same thing better) and differential value (doing a new thing) lest we start to confuse Cloud Computing with Innovation.

Hence, in the following hype cycle I've highlighted several activities, including :-
  • cloud computing: more efficient provision of the existing activity of "using computer resources in business"
  • social network analysis: a relatively new activity and a potential differential

Figure 7 - Expectation and Hype Cycle (click on image for higher resolution)

Since we started with differential value or operational efficiency, we can now map "value" zones onto the hype cycle. I've done this below.

Figure 8 - Hype Cycle & Value Zones (click on image for higher resolution)

What this suggests is the early stages of the hype cycle has an increasing benefit. In the trough of disillusionment, there is still benefit to be gained but it's diminishing. However, in the slope of enlightenment and the plateau of productivity there is little or no differential or operational benefit over competitors since everyone else is doing it. That's our second "insight".

The lesson of this story has been known in military circles for a long time. An imperfect plan executed today is better than a perfect plan executed tomorrow i.e. if you wait until the activity can be easily and effectively implemented (the plateau of productivity), it'll provide little competitive benefit to you.

Fortune favours the brave.

[A final few comments]

To generate the expectation curve I had to create a model over time. This required lots of assumptions because the evolution (lifecycle) curve does not have a time axis (i.e. you can't predict when something will evolve). There are hence a couple of points I'd like to make clear.
  1. You can't simply overlay the expectation curves of different activities on top of each other - i.e. the axis of time is different (some are stretched, some are shortened). Gartner's curve doesn't define its time axis and we can therefore assume they're referring to a general shape which appears over an undetermined length of time.
  2. The Gartner curve specifically refers to the technology trigger. We can assume this is when the technology starts to spread and ignores any early stage effects (invention etc).
  3. If the Gartner curve was based upon the measurement of some physical property, it would be possible to reverse the process i.e. from Gartner curve to expectation curve to evolution lifecyle and accurately state where an activity was along the uncertainty axis. By very definition this is impossible. I can currently only state where something was in the past once it has become a commodity. Hence I have to conclude that Gartner's curve is not based upon some external measurement of physical property but instead it is more likely by a process of expert review (i.e. averaging where forecasters think something is on the curve) or even more simply, analysts placing dots on the curve.
  4. The Hype Cycle in its current form can have both novel activities and the evolution of the same activity to more commodity forms represented on the same position of the curve at different times i.e. a single activity may well go through the the peak of inflated expectations multiple times. In the first case, this will refer to the differential value of the novel activity but later on, the same activity will appear in the same position due to operational efficiency of a more commodity form. This occurs because the same activity can be given multiple different memes i.e. x86 architecture, client/server, hosting or cloud and whilst those memes are different, the underlying activity (i.e. provision of computing infrastructure) is the same. You cannot therefore use the Hype Cycle to determine evolution, notwithstanding the issue that it is time based and evolution cannot be measured over time (we have no crystal ball).
  5. The expectation curve matches the Gartner curve in certain circumstances. I cannot conclude much about the Gartner curve other than to say that its shape appears to have some validity in specific circumstance and I can approximate the value zones where the curve does match. This doesn't say anything about where the dots are placed on the curve just that the rise of increasing then diminishing then plateau of negligible benefits seems broadly right. That up, down, almost flat shape seems to have merit.
-- Update 10th Feb 2015

This uses an old form of the evolution graph. I've subsequently refined the terms innovation, custom built, product and commodity to genesis, custom built, product (+rental), commodity (+utility). The problem was that whilst I used "innovation" to mean the first ever attempt to put an activity in practice, the common use of the word "innovation" (used liberally to mean genesis of something, feature differentiation of a product, introducing a utility model for an existing activity) made that meaningless. Hence, I changed to use "Genesis" to re-assert the point.

-- Update 7th Sept 2015

Gartner Hype cycles no longer use Visibility vs Time but instead Expectations vs Time. The "time" axis seems to be only a generic sense of travel as the items on the hype cycle are given times to reach the plateau e.g.

I have no idea why they've made this change. Obviously, I was using an expectation curve and so again I broadly agree with the shape. I'm somewhat concerned that the axis changed but the curve didn't but given that the dots are just aggregated opinion, I suspect the curve is just an opinion as well. This doesn't mean it's not useful, as long you keep in mind it's just opinion all the way down.


Tobias op den Brouw said...

Thank you for that. It will require some pondering.

Dean Bubley said...

I agree up to a point, but I also see another effect around expectations in the early phases of new technologies.

Predictable "second order" problems in terms of implementation or deployment are ignored until too late in the cycle. There is a linear process of waiting until the problems actually manifest "for real" before they are dealt with.

This adds in huge amounts of delay and the "disillusionment", which could have been reduced if there were *parallel* efforts made on driving initial launch & growth, as well dealing with the forseeable "gotchas".

You'll never predict everything, but as an analyst I've repeatedly seen markets stall, because of issues that I'd identified a year or two before they become major obstacles.

It ought to be possible to "smooth out" at least some of the hype-cycle peak & trough.

Anonymous said...

Could Copyleft and Open source methods have a bearing? Copyleft and open source methods ensure that innovation can carry on during the exploitation of product. If a product fails or falls short in expectations it can forked and the best of the work can be extended. Will an infinite number of devs/forks create perfection or will the law of diminishing returns always apply?

swardley said...

Hi Dean,

Interesting points, personally I've never looked into how to smooth the curve simply because underneath this exists a lifecycle for which certainty is an axis i.e. in the early stages of an activity we cannot know whether it's going to be successful or what actual form it'll ultimately evolve to. We can only hint at a general progression.

To smooth it would imply some predictive capabilities. This might be possible with activities that are in the later stages of lifecycle and hence expectation curves are built around operational efficiencies.

However, the Hype Cycle has both types (differentials and efficiencies) and it's difficult to see how one could predict new activities.

Thanks for the comment, food for thought.

swardley said...

Interesting follow up read -

Andrzej Łączny said...

If we see both operational efficiency and differential value as innovations (process innovation and value innovation), we would have common term to rely to - "innovation". In that case, seeing similar curve for both the cases, is not that surprising.

Of course, you need "value" to have the same "unit" as "cost", to buid curves in the first place, as you calculate the benefit. Finding such matching "unit" for innovation would be harder.


Oliver Maurice said...

You can read here some tips on how to write a graduation speech. It could be a good idea to check it out beforehand