This is currently part of a piece of work that I'm writing up on prediction tests but due to a conversation with Florian Otel on the role of actors and the aggregated effects of actors then I thought I'd put up a rough diagram. There appears to be four different domains of prediction (see figure 1)
Figure 1 - The different domains of Prediction
There are the :-
Known (what) Known (when) : these are modeled and predictable trends for both what and when such as the trajectory of a canon ball. The what (i.e. its path) and the when (i.e. where it will be on its path) are predictable to a high degree of certainty. This is not absolute, it never is just that there exists a high degree of certainty. In economics, price elasticity of a mature system or diffusion of an already established product in the market is in this category - you can predict both the effect (the path) and the time.
Known (what) Unknown (when) : these are modelled over what will happen but the when is imprecise and uncertain. You at best have to look for weak signals. The evolution of activities from genesis of the novel and new to commodity is in this category. We can say what is going to happen (hence Parkhill was able to predict the cloud computing industry in 1966) but not when with any precision until we get very close to the time of it happening (e.g. with Cloud computing the last decade).
In the case of the evolution of business activities, the path is dictated by competition between all actors (consumers and suppliers) and hence the overall trend is predictable, however the shift from one economic domain (such as products) to another (such as utility services) depends upon the actions of a specific actor (i.e. someone setting up a utility service) and this is the unpredictable part. The best you can know is it is more likely to happen now through secondary signals. Much the same way that a military of the past could look for weak signals, such as sailors of the opposing force hanging out their clothes on drying lines to anticipate the opposing forces fleet setting sail.
Unknown (what) Known (when) : these are modelled over when but the precise what is uncertain. For example, the commoditisation of a set of pre-existing activities initiates a "war" in that industry and a punctuated equilibrium when the past is destroyed. If those activities can be components of higher order systems (e.g. computing infrastructure as a component of other systems) then the "war" will result in a time of "wonder" with the rapid creation of new higher order systems and un-modelled data. You can predict when the time of wonder will occur but not what new things are created. Hence with cloud computing the rapid growth of new higher order systems, explosions of un-modelled data (Big Data), the co-evolution of practice (DevOps) and the rise of a Next Generation of company was all predictable in terms of time just not what those things would be in detail.
Unknown (what) Unknown (when) : the genesis of the novel and new is in this category because of its uncertain nature, deviation from the past and its dependency on actors actions. We cannot say what will be created or when it will be created with any precision.
Now these divisions appear fairly permanent because of the underlying mechanics involved. In some case we might mis-treat a change such as treating something as an Unknown Unknown because we do not understand the underlying mechanics e.g. the orbit of the planets before we had any concept of gravity, planetary movement etc. Today, it's more of a Known Known but that is because of the existence of a model of understanding.
In other cases, such as how business activities evolve we have a Known (what) Unknown (when). This is because the Known (what) is driven by competition of all actors but the Unknown (when) depends upon the actions of specific actors. We cannot improve beyond this without somehow predicting the actions of individual actors which as Hayek would have pointed out is not currently possible. Hence at best you have to resort to weak secondary signals i.e. act is suitable for provision in this state, the concept exists, the technology exists, there is a willingness for consumers to adapt due to "grumbles over the cost of the existing model" etc.
The Model of Understanding and Why
That "model of understanding" is our best answer to Why something occurs and the pursuit of science is fundamentally concerned with the shift of this Why from unknown to known, from uncertain to more certain. Hence in the flight of a cannonball we have a pretty good understanding of Why and What and When i.e. you can describe it as Known (why) Known (what) Known (when).
In other cases, such as the evolution of business activities we have pretty reasonable understanding of Why and this gives us a model of What not When. Even if the evidence behind the model develops to a point that we are fairly certain of Why the model works, it will still only give us What not When. Hence we're heading in a direction of Known (why) Known (what) Unknown (when).
You should never assume that scientific pursuits (whether natural sciences or economics) will ultimately give us a Known (why) Known (what) Known (when) for everything because in many cases the mechanics prevent this. For example if they depend upon actions of individual actors then the underlying probabilistic nature simply becomes more pronounced. Our best model of understanding might only ever be Known (what) Unknown (when).
It's also worth emphasising that we can only ever get to best models of understanding. Even our most cherished scientific theories, which we're usually fairly certain about, are nothing more than best models. When people say they know how this works, they're actually saying we've got a pretty good model of understanding. It's never absolute, there are no absolutes, there is no absolute truth. Even the beloved field of mathematics only has trivial truths or ones that we define as being true (See Bertrand Russell's paradox and Goedels' Incompleteness theorem). Known is just a short hand way of saying "a very high degree of confidence that this is correct and we will be really surprised if it isn't."
I also emphasise this point of Why because of a pet dislike of mine within management studies - the use of backward causality and the curse of the case study. The fundamental premise is that if successful company X does Y then by doing Y it would mean that I too will be successful. This is normally presented as a case study of Y from a big and successful company which encourages you to follow suit. In my view the bible of this approach was Tom Peter's "In Search of Excellence".
The problem with this popular approach is that it does nothing to improve our model of understanding, it does not give us the Why and it tell us nothing about where we maybe heading. Many examples of those "Excellent" companies such as Kodak and Polaroid are hardly companies most would aspire to today. Unfortunately, the pursuit of Why is often dismissed as too theoretical, too academic and not practical enough. Well, I'll happily agree that the pursuit is difficult but as for being too academic a pursuit ... no.
It's that lack of understanding Why which will almost certainly be behind the highly probable and unnecessary disruption of once great companies such as HP, Dell, IBM, Oracle and SAP by a predictable market change such as cloud computing. These companies by right of their position should never face disruption by a predictable market change. They should only be disrupted by an unpredictable market change as in product vs product disruption such as cable vs hydraulic excavators due to an unexpected change in the value network.
However people talk about cloud computing as though it was unexpected, it wasn't. The What was known, just the When was imprecise but this could have been planned for a long time in advance and weak signals monitored. The issues of inertia, cultural change and the impacts of changing practice could have been managed well in advance. Disruption in this case was entirely preventable. The most interesting thing about these economic states of "war", the punctuated equilibrium where past companies are destroyed is that these are predictable changes and these companies should not have failed. But fail they do, and in general it appears to be because they do not understand the why. They actually think this an unexpected market change. It isn't. It's simply one they failed to prepare for.
Cloud computing wasn't an Unknown (why) Unknown (what) Unknown (when) i.e. an entirely unpredictable change but instead it was a Known (why) Known (what) Unknown (when).
Other changes are however unpredictable, we cannot foresee genesis of the novel and new as it is uncertain by nature. Hence we cannot know that some unpredictable characteristic such as physical size in storage disks will become important. These are cases of the "Innovator's dilemma" where a focus and pre-occupation with existing customers (i.e. our inertia to change) and the unpredictable nature of the change leads to our downfall. But cloud computing is not this.
The probable failure of many of these great companies is due to a predictable change, a preventable disruption combined with the failure of past and in some cases present executives to predict the predictable and plan for it. To put it mildly, many great companies will fail and continue to fail because their executives are out of their depth. They either do not or did not understand the why.
The storm that is cloud computing was foreseeable and was foreseen a long time ago. Companies could have been moved out of its path despite their desire to stay put. This is the role of the CEO and the captain of the ship - to see the storm, to move out of its way and to take advantage of the change. This is the art of strategy. Destruction in this case is lamentable. It's not the employees, it's not the culture, it's not inertia, it's not some random unpredictable change that's to blame but an utter and miserable failure of strategy.
The understanding of Why things change is far from just an academic pursuit.
The Model of Understanding and Why
That "model of understanding" is our best answer to Why something occurs and the pursuit of science is fundamentally concerned with the shift of this Why from unknown to known, from uncertain to more certain. Hence in the flight of a cannonball we have a pretty good understanding of Why and What and When i.e. you can describe it as Known (why) Known (what) Known (when).
In other cases, such as the evolution of business activities we have pretty reasonable understanding of Why and this gives us a model of What not When. Even if the evidence behind the model develops to a point that we are fairly certain of Why the model works, it will still only give us What not When. Hence we're heading in a direction of Known (why) Known (what) Unknown (when).
You should never assume that scientific pursuits (whether natural sciences or economics) will ultimately give us a Known (why) Known (what) Known (when) for everything because in many cases the mechanics prevent this. For example if they depend upon actions of individual actors then the underlying probabilistic nature simply becomes more pronounced. Our best model of understanding might only ever be Known (what) Unknown (when).
It's also worth emphasising that we can only ever get to best models of understanding. Even our most cherished scientific theories, which we're usually fairly certain about, are nothing more than best models. When people say they know how this works, they're actually saying we've got a pretty good model of understanding. It's never absolute, there are no absolutes, there is no absolute truth. Even the beloved field of mathematics only has trivial truths or ones that we define as being true (See Bertrand Russell's paradox and Goedels' Incompleteness theorem). Known is just a short hand way of saying "a very high degree of confidence that this is correct and we will be really surprised if it isn't."
I also emphasise this point of Why because of a pet dislike of mine within management studies - the use of backward causality and the curse of the case study. The fundamental premise is that if successful company X does Y then by doing Y it would mean that I too will be successful. This is normally presented as a case study of Y from a big and successful company which encourages you to follow suit. In my view the bible of this approach was Tom Peter's "In Search of Excellence".
The problem with this popular approach is that it does nothing to improve our model of understanding, it does not give us the Why and it tell us nothing about where we maybe heading. Many examples of those "Excellent" companies such as Kodak and Polaroid are hardly companies most would aspire to today. Unfortunately, the pursuit of Why is often dismissed as too theoretical, too academic and not practical enough. Well, I'll happily agree that the pursuit is difficult but as for being too academic a pursuit ... no.
It's that lack of understanding Why which will almost certainly be behind the highly probable and unnecessary disruption of once great companies such as HP, Dell, IBM, Oracle and SAP by a predictable market change such as cloud computing. These companies by right of their position should never face disruption by a predictable market change. They should only be disrupted by an unpredictable market change as in product vs product disruption such as cable vs hydraulic excavators due to an unexpected change in the value network.
However people talk about cloud computing as though it was unexpected, it wasn't. The What was known, just the When was imprecise but this could have been planned for a long time in advance and weak signals monitored. The issues of inertia, cultural change and the impacts of changing practice could have been managed well in advance. Disruption in this case was entirely preventable. The most interesting thing about these economic states of "war", the punctuated equilibrium where past companies are destroyed is that these are predictable changes and these companies should not have failed. But fail they do, and in general it appears to be because they do not understand the why. They actually think this an unexpected market change. It isn't. It's simply one they failed to prepare for.
Cloud computing wasn't an Unknown (why) Unknown (what) Unknown (when) i.e. an entirely unpredictable change but instead it was a Known (why) Known (what) Unknown (when).
Other changes are however unpredictable, we cannot foresee genesis of the novel and new as it is uncertain by nature. Hence we cannot know that some unpredictable characteristic such as physical size in storage disks will become important. These are cases of the "Innovator's dilemma" where a focus and pre-occupation with existing customers (i.e. our inertia to change) and the unpredictable nature of the change leads to our downfall. But cloud computing is not this.
The probable failure of many of these great companies is due to a predictable change, a preventable disruption combined with the failure of past and in some cases present executives to predict the predictable and plan for it. To put it mildly, many great companies will fail and continue to fail because their executives are out of their depth. They either do not or did not understand the why.
The storm that is cloud computing was foreseeable and was foreseen a long time ago. Companies could have been moved out of its path despite their desire to stay put. This is the role of the CEO and the captain of the ship - to see the storm, to move out of its way and to take advantage of the change. This is the art of strategy. Destruction in this case is lamentable. It's not the employees, it's not the culture, it's not inertia, it's not some random unpredictable change that's to blame but an utter and miserable failure of strategy.
The understanding of Why things change is far from just an academic pursuit.
