Sunday, June 11, 2017

To Infinity and Beyond

Chapter 17
(very rough draft)

Meeting a spaceman

I was working on the use of printed electronics within a paper book (think of an interactive book which looks like a normal book) when I got that phone call from a friend about "this guy who really wants to meet you". I was curious, so I went along to meet someone called Mark at Canonical. I didn't know what to expect. The first few minutes were certainly interesting.

Shuttleworth : "I'm Mark. I've been told you're a good UX designer."
Me : "I don't know anything about design."
... silence.

It was an awkward pause. Then Mark realising the next hour was probably a waste of his time asked me to tell him what I did know about. I talked about evolution, the changes in the industry and before long we were into graphs, maps and cloud computing. The time flew by. We kept talking. I was introduced to others and in what seemed like lightning speed, I was working at Canonical. I had one job, to bring Ubuntu into the cloud. I called my friend, asked him what had happened. Steve just responded "I knew you'd get along".  Life is full of pleasant trouble makers like Steve.

The first day I arrived for work, I was all excited and had the usual confused look of a rabbit staring at headlamps. My boss, who also happened to be another Steve, did the usual rounds of introductions. That was an interesting moment. Whilst I delighted in the warmth of the people I met, the first five responses to my role of bringing Ubuntu into the cloud were negative - it's a fad, why are we doing that etc. I knew I was going to have to build a cabal pretty quickly and create some momentum. However my first official task was to look at the virtualisation strategy that had been written. It was one of those "oh, what have I done" moments. Fortunately it didn't take long to find others with common interests - Rick Clark, Soren Hansen, Nick Barcet and many others. Steve George was one of the most supportive people I've worked for, a good friend and then there was Mark. Without Mark none of this would have happened. 

The problem to begin with was Canonical was focused on the server and desktop market. It was up against huge giants such as RedHat and Microsoft. It was making valiant, almost heroic efforts but Canonical was small. Many wanted to focus on the server OS, to generate revenue from support licenses and to a few then the Cloud was a distraction. The problem was one of focus and what I needed to do was change the mindset. To explain this issue and why it mattered I'm going to cover a number of concepts from the Three Horizons to Porter.

The Three Horizons

The three horizons was a model put forward in the Alchemy of Growth, 1999. It discussed three views that any corporation had to take.

Horizon 1 : the core business which provides the greatest profits and cash flows that need to be extended and defended.

Horizon 2 : are the emerging opportunities and businesses that will drive medium term growth. These may include new ventures that you are investing in which are expected to generate substantial future profits.

Horizon 3 : These are ventures that should ensure the company long term future.  They can include research projects or pilot programs or even investment in startups.

For Canonical, horizon one was the core support licensing revenue.  Horizon two included new concepts such as online storage, the app store and extending onto more devices. Horizon three ... well, I'm quite convinced a few would have thought that included myself. Whilst this model of three horizons is a reasonable way of examining a company, I personally find it inadequate. I often find that some confuse it with the pioneer - settler - town planner model of organisation by associating town planners with horizon one and pioneers with horizon three.  To explain the weakness with the model, I'm going to use the map of mapping that I introduced earlier. To save you scrambling back through past chapters, I've provided that map here in figure 213.

Figure 213 - The Map of Mapping.

Let us now assume that we decide to use the map of mapping to build a new business. I'm going to take a part of the above map and concentrate around the provision of forecasting (i.e. anticipation of know changes) to others. I could have quite easily built a comfortable life around the weak signals I developed for forecasting change and built myself a small boutique consultancy providing market and technological forecasts. The premise behind such a business is provided in figure 214. My purpose with such a business is to simply survive (i.e. make money), the user wants an advantage over competitors, they measure this by the return on capital invested in a space, I enable this through anticipation services based upon known climatic (economic) patterns that use maps of the industry. It would be a relatively trivial business to create had I the desire. 

Figure 214 - Forecasting Service

Horizon one would be that boutique consultancy business. I'd have been protecting (i.e. not making creative commons) the twenty odd common economic patterns that I know about which impact the environment. I'd probably use a worth based mechanism (or outcome based as it is called today) for charging. I could also extend this map to cover in more detail the social capital components of trust, the activities needed to perform the analysis and run the company. Remember you can make all forms of capital whether data, practice, activity, knowledge or social. Let us hypothesise that I had done this and by hook or by crook turned it into a small success. What would my horizon two be?

In this case, the diffusion of knowledge and evolution caused by supply and demand competition would drive many of those components to a more industrialised space. At some point, I'd have to prepare for my boutique consultancy entering a world where products did the same thing. I would know in advance that we'd have inertia to that, any shift from one stage of evolution to another (e.g. custom to product) has inertia caused by past success. It's one of the those climatic patterns. I've mapped this in figure 215.

Figure 215 - Horizon two

But, with foresight - and I'd hope that I'd be using mapping on myself - then it would be relatively trivial to anticipate and overcome the inertia. How about horizon three? In this case, we get a divergence. I could for example focus on further industrialisation to a more utility service exposed through some form of API - Anticipation as a Service or AaaS for short. Of course, such as change along with mirth over the name would come with significant inertia created by any existing product based business model. Alternatively, I could expand into something new such as the use of doctrine for competitor analysis or the arms sale of context specific gameplay or even some novel, uncharted, higher order system that I haven't even considered. I've shown these divergent horizon threes in figure 216.

Figure 216 - Horizon three

Now let us add the pioneer - settler - town planner model onto the horizon three map (see figure 217). Remember each team has different attitudes (which is what pioneer, settlers and town planners represent) and each not only build but operate their own work. The important thing to note is that horizon three consists of town planners or settlers or pioneers or all of the depending upon what I choose to do.

Figure 217 - PST added to horizon three.

The first thing to note are the horizons are context specific. You cannot simply overlay them onto a PST model or even the concept of evolution (e.g. by saying that genesis is horizon three) as it depends upon where you are and the landscape surrounding you. The second thing to note is that the horizons can often be broadly anticipatable. This is the thing I find inadequate with the horizon model because without a map and the learning of common economic (aka climatic) patterns then it becomes all too easy to miss the obvious. It is why I find the three horizons useful as a high level concept but overall weak in practice on its own. It also fails to help me adequately deal with inertia or legacy.

The issue of legacy

In chapter 9, we examined the climatic patterns of co-evolution i.e. practices can co-evolve with the evolution of an activity. There is usually some form of inertia to a changing activity and this can be compounded by a co-evolution of practice. In figure 218, I've taken the original diagram from chapter and added some inertia points for the shift from product to utility for both compute and also platform.

Figure 218 - Change of Compute and Platform

As previously discussed, there are many forms that inertia can take. However, the question I want us to consider is what represents legacy in this map.  The two obvious areas are those trapped behind inertia barriers e.g. compute as a product and platform as a product (i.e. platform stacks). The next obvious include the related practices i.e. best architectural practice associated with compute as a product. What is not so obvious to begin with is the issue that as components evolve enabling higher order systems to appear then the lower order systems become less visible and for most of us legacy. The departments that ran switchboards in most companies were once a highly important and often visible aspect of communication. For many companies, that activity has been consumed into either reception or call centres in much the same way that email has consumed the postal room. We still send letter to each other (more than ever before) but they are digital. In this case, the role of the components underneath the platform layer are going to become less visible. Dealing with and managing infrastructure will become as legacy to most companies as the switchboard is today.

Hence another area of legacy would be the practices and activities below the platform layer which includes concepts such as DevOps. In 2017, such a statement tends to receive a strong negative reaction. Most react with the same forms of inertia as those who reacted against cloud in 2006. Many will claim DevOps is more than infrastructure as it's about development and culture. Depending upon how far in the future you're reading this from, you'll probably be quite surprised by this and even more likely have never heard of DevOps. As with all such things, DevOps was a child and reaction against the prevailing methods of management. It co-opted concepts from earlier school of thought (e.g. ITIL) including iterative approaches, use of components, configuration management, services approach, a focus on users and measurement whilst simultaneously distancing itself from them. It added its own dogma and sort to create a separate tribe. The same will happen in platform, a new school of thought will emerge that will copy and build upon DevOps but deny it has any relationship to it. DevOps will become "what my mum and dad does" as the rebellious child declares its independence and denies any association to the former.  If you think of concepts as genes, then many of the genes of DevOps will be found in the new generation (though they will rarely admit it, painting DevOps as some strawman version of itself), some of the genes will become redundant and others will emerge. 

I've marked on the main area of legacy onto our map in figure 219.  To do this, I've used the concepts of inertia and how industrialised components enable not only higher order systems but become less visible themselves. I've also added on a typical PST structure. As we can see, many of the legacy areas exist within the settlers and the town planning teams. 

Figure 219 - adding legacy (a consumer perspective)

Obviously there is a perspective to be considered here. I'm looking from the point of view of someone who consumes compute. If I'm a major provider, whether platform in the future or utility compute today then much of this is definitely not legacy any more than power generation systems are to electricity providers. From the perspective of a major provider then legacy would look more like figure 220 i.e. it will consist of activities (and related practices) that are stuck behind inertia barriers but not the impact of lower order systems becoming less visible. What becomes increasingly invisible to others (i.e. consumers) is still very visible to providers.

Figure 220 - legacy from a provider perspective.

There is an unfortunate tendency of people to associate the town planning groups with legacy. As should be clear from above, then that's not the case. The recent future of computing has been industrialisation by town planners to utility services. The legacy has been past product models, a realm of settlers.  If we take the consumer perspective from figure 219, then the future is a mix settlers building application, pioneers discovering emerging practices that combine finance and development (whilst denying any inheritance from DevOps) and town planners busily create the empires of scale around platform utility services. I've shown this future in figure 221 and it's where companies should be investing in 2017.

Figure 221 - the future, from a consumer perspective

It's important to note that legacy can be anywhere. It can be caused by a custom built activity which has failed to evolve or a product based business in a utility world. Legacy is simply a consequence of a failure to evolve and it is not associated with one group such as pioneers, settlers or town planners but instead all.  When it comes to managing legacy then it's really important to understand those points of change and the impact of co-evolution. This should become second nature to you and it's worth practicing. There's another perspective beyond the three horizons, beyond inertia and legacy that we also need to discuss. It's the perspective of Porter's forces.

On Porter

For those unfamiliar with Porter's five forces, these are rivalry within the industry, threats of new entrants, threats of substitution and the bargaining power of suppliers vs consumers. In this section we're going to examine these five forces through the lens of the peace, war and wonder cycle (see chapter 9).

In the time of wonder, it is a battle to become established. The field is not yet developed and there are no "new entrants" as there are no established figures. Everything is new, uncertain and uncharted. The consumers hold the power and it is they who decide whether this industry will succeed or not.

In the time of peace, there is a constant tug of war between supplier and consumer power over the products produced. The developing giants are normally well protected from new entrants in a game of relative competition, except against the occasional threat of substitution. It is this substitution by a different product which is the dominant factor.

In the time of war, new entrants providing a more industrialised form of the act threaten the existing giants that are stuck behind inertia barriers. It becomes a fight for survival for these giants and they are often poorly equipped. It is not a case of a product becoming substituted by another product but instead an entire industry is being changed to more industrialised forms.  It is often assumed that the shift towards utility provision means centralisation but this is not the case. 

Whilst the interaction of all consumers (demand competition) and all suppliers (supply competition) drives the process of evolution, the question of whether a specific activity or data set centralises or decentralises depends upon the actions of individual actors (suppliers and consumers) in this market.  For example, it would have been relatively trivial for the hardware manufacturers to create Amazon clones and a price war in the IaaS space around 2008-2010 in order to fragment the market by increasing demand beyond the capability of one vendor to supply due to the constraint of building data centres. I had these exact conversations with Dell, IBM and HP throughout 2008 and 2009. I even told them their own inertia would fight against this necessary change and they would deny the existence of the punctuated equilibrium until it was too late. The fact they didn't act and lost their own industry is entirely the fault of their own executives and also one of the major factors why have seen  centralisation in the IaaS space. Centralisation depends upon the actions of specific actors (in this case the inaction of hardware suppliers and hosting companies). In the future, this may in fact yo-yo from centralised to decentralised or find a balance between the two (as with electricity provision and self generation). Of course this is a change in the means of production and the interfaces themselves are unlikely to change i.e. a shift from central to self-generation does not mean a change in voltage / frequency for domestic power provision.

The point to remember is the balance between these forces tends to change as anything evolves. It also isn’t static within a stage of evolution. For example, when an activity becomes more of a commodity or provided as a utility we will often experience a yo-yo between centralisation and decentralisation (with a corresponding yo-yo between Supplier and Consumer bargaining power). However as a general guide, I provided in figure 222 the most dominant forces you're likely to encounter.

Figure 222 - Porter's forces and evolution

Examining Canonical

With a basic understanding of horizons, Porter's forces and legacy, we can now examine the business of Canonical. The horizon one (core business) was related to selling support on the server OS (operating system). However, compute was evolving to more utility provision, Hence, with the exception of large cloud providers then the server OS support was likely to become a legacy business. Instead, we'd needed to focus on horizon two and the commercial use of guest OS on top of these large virtualised computing environments. We understood that companies would have inertia to these changes and being a shift from product to more industrialised forms it was likely to be a punctuated equilibrium (period of rapid change). We also understood that the biggest threats into this space would be new entrants and given the state of strategic play in many companies then we were likely to see centralisation. I've drawn these concepts onto the map in figure 222.

Figure 223 - the changing market

We also understood that co-evolved practices would emerge, that Jevon's paradox meant we were unlikely to see significant savings in IT but instead increased development activity and that a further horizon, the shift of platform from product to utility was possible. I've marked up these horizons onto figure 224.

Figure 224 - the horizons.

In terms of play, we understood that moving fast and land grabbing the guest OS was essential. To help in this, we also needed to support those developing applications or building tooling around those co-evolved spaces. If we found examples of platforms plays in this space we needed to be invested in this. We also understood that many potential customers would have inertia hence we'd have to provide some forms of transitional / private cloud offer. We also knew our competitors had inertia. As soon as I discovered Red Hat salespeople were rewarded bonuses based upon satellite subscriptions (used for security updates) then I quickly set about promoting a message that security should be "free" in the cloud.  There's nothing like threatening someone's bonus to get them to turn against and spread fear, uncertainty and doubt around a change. Our focus was clear within my cabal. Mark did an amazing job of making the focus on cloud the focus of the entire company. Rick and others set about putting in engineering effort to make it happen. Steve gave me all the firepower and cover I needed. For my part, I mainly focused on promoting Ubuntu's cloud message, being involved in the community, highlighting targets to bring on board and trying to stop people rebuilding or getting in the way of things that the community was doing. An outline of the play is provided in figure 225 and the result in figure 226. Within eighteen months, Ubuntu went from a small part of the operating system to dominating the cloud guest OS. My part was a minor but instrumental role and I have to applaud the marvellous teams at Canonical and within the community for making it happen. A small company of three hundred took on the might of two giant hordes but unlike the Spartans, this time we won. My proudest moment came from hearing a CIO talk about how "the future was all RedHat and then suddenly it was all Ubuntu". I played a small part in that.

Figure 225 - our focus

Figure 226 - the results

I often hear people talk about Canonical was lucky, well there's always some element of luck but the moves were deliberate. Obviously, people can just say the timing was lucky but they'd be wrong on that as well. I had a helping hand with timing thanks to Gartner. They probably don't even realise but I think it's worth explaining.

On the question of timing

I'm not a big fan of Gartner but figure 227 is one of the most useful graphs they've ever produced. It's a hype cycle of emerging technologies created in 2008.  It uses the earlier y-axis of visibility which later on became expectations. How does the axis change whilst the graph remain the same? Ah, that's the beauty of it but first, a bit more background.

Figure 227 - Gartner emerging technologies, 2008

During my time in the wilderness prior to Canonical, I had been looking at various ways of measuring impacts from evolution. One of the issues with this is when we look at opportunity then the evolution of any single act creates different waves of opportunity. One of these waves is focused on differential value (i.e. it's something you have but I don't) and the second wave is around operational value (i.e. we both provide this but you do so more efficiently). Both the waves appear to have a learning element and then a sharp decline as the change diffuses and evolves further. I've provided examples of these waves in figure 228.

Figure 228 - An example of different waves of value.

Of course, opportunity is only part of the equation. There's also the cost involved, particularly in development of something novel. There's also risk as the uncharted space is by its very nature uncertain. However, I developed a generalised benefit curve which for differential value is shown in figure 229. An almost identical benefit curve exists for operational value but that occurs much later in evolution and is related to co-evolved practices that emerge.

Figure 229 - A benefit curve for differential value

From the benefit curve, the early stages of genesis are all about investment. As it evolves, the cost of production reduces and we start to realise some of the benefit. We're still in the custom build stage, others are starting to copy but in general the cost of production is reducing fast enough to overcome any differential loss due to copying. Alas, at some point the cost of production is low enough and the activity defined enough that someone produces a product. On the upside the cost to implement is plummeting but alas, the differential value is declining faster as more companies do actually implement. The models I developed all had variations of this shape, so think of it more as a mental model.

What I then became fascinated by - I like to explore - was timing issues. Let us say we've recently read a whitepaper on a marvellous new activity. That activity is described as having some benefit but it also involves cost. By the time I get around to implementing the activity it may well have evolved. It might provide a different benefit to what I was expecting i.e. it costs less because it's a product but there's little differential value as everyone else is doing this. I've superimposed the evolution of an act onto the benefit curve in figure 230 to highlight this point.

Figure 230 - Changing benefit with evolution and implementation

I then modelled this delta between what I was expecting to get and what I got over time. The model I used made lots of horrible assumptions and it's about as solid as a tower of jelly.  At some point in the future, I might go and revisit this but I don't normally mention this little side journey in mapping. However, there was one remarkable thing about the delta expectation curve over time - it somewhat resembles a Gartner hype cycle - see figure 231.

Figure 231 - delta expectation over time (the expectation curve).

We have the same peak of inflated expectation, the same trough of delusion. My first reaction was horror. The evolution curve on which mapping is built uses ubiquity versus certainty. If I can model from Gartner's hype cycle to evolution then I can take the points on a hype cycle and measure precisely where something is on the certainty axis of evolution. For things that are uncertain then this should be impossible. My first reaction was Gartner's hype cycle proved evolution was wrong. I was a bit glum at that point especially since I had found mapping so useful. Fortunately, I met with a friend who pointed to a hole in my argument. I was assuming that Gartner's hype cycle was based upon a measurement of some physical property. If it wasn't, if it was just aggregated opinion (of consultants, analysts or industry) then there's no measurement of the uncertain as it's just opinion. This turns out to be the case, the hype cycle is just opinion. For interest, Gartner now uses expectation on that y-axis.

Along with being quietly relieved that I hadn't yet disproved what I was finding useful, it also opened up a new opportunity. I have two benefit curves - one for differential value and one for operational value. They both shared a common expectation versus time pattern. For example, if I look at an evolving component then where it appears in the early stages on the expectation curve for differential value can be the same place it appears on the expectation curve for operational value when it's more evolved. See figure 232

Figure 232 - Evolution of an act on differential and operational expectation curves.

Now, I already have a weak signal using publication types that could identify when things are likely to start to industrialise and enter a war (see chapter 9). I've reprinted the last analysis on this that I undertook in 2014 in figure 233. What I'd like you to notice is that the shift from product to utility for infrastructure was well into a war in 2014. Whereas the war for 3d printing and the use of commoditised 3d printers is some way off.

Figure 233 - When is the war likely

Now, in 2008, I already knew (from my weak signals) that we were entering the war phase for computing infrastructure whereas 3d printing had a long time to go before it started to industrialise. I also suspected that both a relatively novel activity (e.g. 3d printing) and an industrialising activity (cloud) could appear at the same place on two different expectation curves - one for differential value and one for operational value (figure 232 above). So, let us look at that Gartner hype cycle again and highlight two components - cloud computing and 3d printing.

Figure 233 - Cloud computing and 3D printing.

They both appeared at roughly the same place. This told me something which I've subsequently found quite useful. The Gartner hype cycle doesn't distinguish between differential and operational value as both are on the same curve. So, why does that matter? Well, in the case of cloud computing, which was the industrialisation of computing and all about operational value then you'd want to be going all in during 2008. Being in the early stage of this expectation curve just reinforces the point that people are learning about a change which you absolutely want to be a first mover to. The last thing you'd want to do is wait until it reach the plateau of productivity by which time the war would be well and truly over. If you're a vendor, this would be curtains. Gartner even calls out that this is moving fast with its time to mainstream adoption for cloud (light blue circle).

However, in the case of 3D printing then you do want to wait or be a fast follower. It has a long long way to go before it industrialises and you've got an entire product stage it has to evolve through. In fact 3D printing will reach the plateau of productivity and see relatively widespread adoption as a product long before it industrialises. At some future time, as it starts to industrialise then it'll probably reappear in the technology trigger (usually under a slightly different meme). When it comes to 3D printing then you could wait a bit and get involved in the product space or wait much longer until the "war" is upon that industry at which point you'd need to go all in.  Two points - cloud computing and 3D printing - on almost exactly the same point of the hype cycle required radically different approaches to investment and strategy. One was "all in", the other was "wait and see".

Being aggregated opinion, I do find the hype cycle quite useful as long as I separate out what stage of evolution something is in first.  I often talk to CIOs who tell me they invest when something is in the stage of enlightenment. That's a guaranteed way of losing every major technological war in business.  For me in 2008, this hype cycle helped reinforce the message that we had to go all in, it was a land grab for this territory. I also took comfort that many of my competitors probably read exactly the same hype cycle and thought they had time. Thank you Gartner, you have no idea how much you've helped me take out companies over the years. Better luck next time IBM, HP, Dell, RedHat ... assuming they survive what is to come. Anyway, the gameplay above was 2008 to early 2010. It's also worth looking at another part of my journey at this time into Government but that I'll leave for the next chapter.