How not to fix Government IT

First, my biases.

1) I'm one of the co-authors of the Better for Less paper which had some influence in the change of Government post 2010.

2) Though I'm certainly not a co-author, I had a minor input into the Boiling Frogs paper. It's a wonderful paper that should rightfully influence many in Government and Business. 

Behind these papers exist concepts such as the necessity for iterative processes, to treat things as they are, to focus on user needs and to start small. 

3) I'm a supporter of GDS.  That does not mean I blindly support them, I have taken issues with the tyranny of agile.  However, the work that has been done over the last five years is remarkable. 

So, to say that I'm seething at the rumoured changes to break up GDS is an underestimate. I do hope Bryan is wrong on these rumours. Now, to be clear - I am all for reducing GDS over time by enabling departments to provide services but Government is not ready to do this now.

In 2013, I wrote a paper for the Cabinet Office (called Governance of Technology Change) on how to organise, structure and ultimately distribute services. It was based on iterative principles, focused on user needs and started small. I believe it spent the next couple of years gathering dust. However, I'm going to go through a few of the basic elements in that paper and explain how I think it should be done. The paper contained a number of basic forms of doctrine, derived from the problem of how to continuously manage an evolving landscape. I've picked a few  :-

User needs


The governance system must clearly reflect user needs in all its decision-making processes. The users include not only departmental users but also the wider public who will interact with any services provided. It is essential, therefore, that those users needs are determined at the outset, represented in the creation of any proposal and any expected outcomes of any proposal are set against those needs. 


Coordination


The governance system must provide a mechanism for coordination and engagement across groups including departments and spend control. 


Shared learning


The governance system must provide a mechanism of shared learning – for example, discovery and dissemination of examples of good practice. 


Context


The governance system must accept that there are currently no single methods of management that are suitable for all environments. Hence a mechanism of describing the context of any proposal is required (for example, one appropriate technique is mapping) which allows for the use of multiple methods and techniques (for example, Agile, Lean and Six Sigma according to where each is appropriate). 
This requires situational awareness.

Structure

Within the paper, a structure was derived to apply the doctrine. This structure is shown in figure 1. Now it looks complex but actually, it is rather simple - OCTO (Office of the CTO), IT Supply, Departments, Capability, Decision and Security. We will ignore Decision and Security for this post.

Figure 1 - The structure (with focus and capabilities)

*The above is a simplified view of the structure used, removing the subgroups into capability of a group

Any new project or request to spend significant sums across government first goes to OCTO (spend control). Each request is accompanied with a map of the landscape that is being changed. The first purpose of creating a map is to demonstrate that the department understands what is being built.

The request is examined by spend control (to show compliance to policy) but analysed through an analysis group (also part of OCTO). What we're looking for is areas to challenge, duplication, bias and points of learning in the map. In your typical private company of any size, duplication is rife due to a lack of communication mechanisms. I regularly find hundreds of different groups building the same thing in a single company despite endless committees on architecture and project inventories. If you don't have a mechanism for understanding your landscape then don't be surprised if most of your IT expenditure is duplicated waste.

To begin with, OCTO starts with a single map. You have little information to challenge the request other than to question whether the department is treating activities in the right way or complying to policy.

Figure 2 - First Map.

However, add a few more maps for other requests and then you'll quickly start to discover a common language and duplication between the maps (shown as green dots in figure 3).

Figure 3 - later maps.

Keep on adding maps and you start to build a profile for the landscape. You can start to see how much duplication you have, where you have bias (people custom building that which is commodity) and where items exist that are suitable for provision as common services (i.e. repeated and commodity like).

Figure 4 - Profile

Within OCTO, the co-ordination capability (your internal consultancy) can respond to the department either advising them to use components from another department or a common service or challenging how the project is being built or how to comply with a general policy. OCTO is not only your learning function but enabler. It does so in an iterative manner, improving its understanding of the landscape with each request (and map). Since each map starts with user needs this also reinforces the doctrine of user need throughout Government.

When we spot potential common services this is passed to the IT supply group. This group ensures delivery of common services and provides a registry of common services online throughout Government (G-Cloud, the Ebay of Government services would be ideal). The actual delivery itself maybe by a department (if it has the capability to build a cross governmental service) or an external source (e.g. AWS) or purpose built by its own capability i.e. GDS. Hence bit by bit we cut out the duplication and build a platform of discrete Government services across all of Government. NB. I use the word platform in the sense of a collection of small and discrete Government services (i.e. payment, identity) and not in the sense of a consultant led orgy to build a new Death Star or a coding platform.

There is also a strategy function because the more maps you have, the easier it becomes to exploit the landscape, open source the right components in order to change the environment and anticipate change through weak signals etc. The capability group is involved in developing cell based structures and ensuring we have not only aptitude but the right attitude in place. This is a bit beyond this post other than to say that departments have to demonstrate they have the capability to build the services. This also gives me an opportunity for another gratuitous plug for Boiling Frogs. Please do read that paper.

As I explained in "stopping self harm in corporate IT" the core function of OCTO (and spend control) is learning and hence enabling. Ultimately over time the Departments should be providing services to each other i.e. doing the delivery. This can be achieved in an iterative manner as above. Bit by bit. Service by service. However, to do this in a federated structure you absolutely need a mechanism of communication and learning otherwise you'll just end up (as with most corporates) with hundreds of duplicated examples of the same thing being built.  Just ask yourself,  how many pet IoT or AI projects doing roughly the same thing are actually going on in your organisation right now? If you're of any size the answer is "you don't know" and from experience, it's going to be vastly more than whatever number you just thought of.

Hence, I'm all for GDS over time becoming distributed with departments providing services. But why not carve up GDS into the departments right now?  UK Gov doesn't have the maps, we don't have the analysis functions and we don't have the capability within departments. This isn't the same as saying Departments haven't improved, many have dramatically. You already have examples of emergent behaviours of collaboration between agencies, for example Highways England and High Speed Rail in DfT. However, this is not uniform. Weaken GDS at this time and we are unlikely to achieve our aims but instead lose our centre of gravity, threaten the symbiotic relationship with Old Street (you think it's just coincidence that as Gov improved so did UK tech in London?) and some talent along the way. We run the risk we could well end up in the bad old days of pre 2010 and the only people I can see who would benefit are big consultancy firms and large tech vendors likely to swoop in with their "Let us build a Death Star" mantra. 

This is not progress, this is regression. 

If you want to fix this, you need an iterative process and that learning capability. You'll need those maps. Oh, and before you say "we'll get a consultancy to make the maps for us" ... the only people capable of doing this are the Departments themselves and the only people capable of challenging are GDS and OCTO.

Now this was my view from 2013. It hasn't changed since. I certainly cannot agree to carving up GDS now, nor the inevitable hiring of a big "stratchegery" (well, it's never strategy) consultancy firm to advise on the carve up. In my view it's dreadful strategic play (if you're Government) and not how to fix Government IT.  I do hope Bryan is wrong on the rumours.

Added 1st August

Asked "Doesn't this take a long time?".

Nope. The problem with most large projects (certainly in the corporate world) is that we usually don't have a clear idea of what is involved. Mapping it out can quickly resolve issues of contract structure, organisation, methodology and purchasing. Mapping itself should be quick and can save time and money. To hear more on this, have a listen to Tony Malone from the Highways England (courtesy of LEF).

One you have a map for a project (as a department), then in the above structure you send it to OCTO. The reason for the co-ordination capability in OCTO is to provide a single point of contact. Analysis of a map, comparison to others and response should happen in the background in a matter of hours. In a well run structure you should be talking hours to map and hours to get a response. Days at a push.

The biggest resistance to mapping is usually that it exposes what is there and the assumptions made. Alas some people just don't like being challenged. But when you're talking tens of millions for a project then being challenged should be the norm.

Added 2nd August

Just as a reminder (as if it really needs to be said) but the mapping method is creative commons share alike. I've put together a list of some useful posts if you need it.

Also, I view Wardley mapping as the equivalent of Babylonian Clay Tablets. Someone will find a better method of mapping that is visual, context specific and has position along with movement.

Show me a map!

A Geographical map is a map

A Chessboard is a map

A Wardley map is a map

They are all maps because they share the most basic elements of a map which are visual, context specific, position (which requires an anchor e.g. compass, the board or user) and movement of the components. Some of them have advanced properties like flows of information between components (NB a flow between components is not the same as movement of a component). These maps are all useful for learning, communication and strategic gameplay.

I often visit new companies and they tell me that they use maps. I get excited and ask to look. Unfortunately what they show me are usually box and wire diagrams which lack basic elements of mapping. From a strategic point of view they are next to useless. They then often try to show me their strategy based upon their maps which invariably is the usually endless round of meme copying, consultant blah blah and wasted effort.

Please note, the following are NOT maps. That isn't to say that don't have their use, they do in specific context e.g. improving efficiency on an existing process. They can also be a damn good start towards mapping and are certainly better than nothing. But from a point of strategy or learning then these won't lead you to improved situational awareness.

Business Process map (Lacks movement, not a map)

Mind Map (Lacks anchor, position, movement and is not a map)

Flow diagrams (Lacks position, anchor and movement and is not a map)

Value Stream Map (Lacks movement and anchor is not a map)

SWOT diagrams (Lacks pretty much everything, not a map)

Strategy Map (Lacks position and movement, not a map)

Trend Map (Lacks pretty much everything, not a map)


















Tube like Map with bits added (Lacks pretty much everything, not a map)

Customer Journey Map (Lacks position and movement. Not a map)

What I'd like to see from Brexit - part one of many.

Following on from my post of less of, more of regarding nation states - I'm firmly in the "more of" camp and would like see Brexit taking this direction. As I said I have my own strong biases here.

Figure 1 - Nations States

I thought I'd write a few posts and scribbled thought processes around this over the next few weeks.

Structure & Market

The neoliberalist ideals of less Government, laissez faire, market knows best are quaint but a simple examination of Western society shows that they are both divisive and less effective compared to a more social capitalist system such as China.  Without addressing these issues then the neoliberalist concepts is the road to serfdom as much as national socialism. 

Let us explore one aspect of this discussion, the market. This is not a force for good or evil, it is simply a mechanism of exploitation which can be used for either good or evil. It contains many failures from discounting the future to asymmetric information to externalities and as such it needs management. Hence my preference is towards social capitalism, a view of the use of the market where appropriate as a tool to achieve an end. In the words of Deng Xiaoping "It doesn't matter in the cat is black or white, so long as it catches mice"

For example, industrialised utilities need to be regulated to prevent exploitation but the full force of the market should be allowed to rage in areas of product development. We should be mindful that the market works well when investing in applied R&D (the exploitation of research for a commercial opportunity) but tends to underinvest when it comes to basic R&D. This creates problems for the future (something the market tends to discount) because basic R&D tends to be the source of applied R&D. Accidental discoveries happen more often when experimentation is invested in.

Of course, this is all complicated by the simple fact that things evolve due to supply and demand competition. Hence the system you require to encourage basic R&D is not the same system you need to encourage its exploitation and that is not the same system you need to manage its provision as an industrialised utility. To summarise, I've put the simple path of an activity in figure 2 along with some basic approaches.

Figure 2 - Structure & Market

For example, in the exploration of new fields (a pioneering stage of wonder) then the role of Gov should be directed invested to encourage and enable exploration. It's a gamble on the future but all exploration of the uncharted space is. Large investment programs around basic R&D without the paraphernalia of exploitation (e.g. ROI) is required.

When things are discovered, the market should be allowed to exploit (with potentially the Gov taking a VC position). This is more a settling stage, a more peaceful stage of product development where sustaining development tends to rule, it is a time of refinement and learning. A more hands off free market approach is required (as per Hayek) with light regulation to encourage competition in all directions but to limit obvious market failures.

As the industry develops it will naturally gain inertia due to past success and hence a more directed is required again to force it towards utility forms. Though inertia exists in earlier stage changes e.g. custom to product, none of these are significant or more threatening than the change from product to commodity. That change is also essential for strong componentisation effects and future development. Hence Gov will need to take a pro-active and directed approach to force change usually by encouraging new entrants with more commodity forms into the market and reducing any regulatory barriers to entry.

As the change finds a new equilibrium around a commodity, a more free market around tighter regulatory constraints can develop to be provide well ordered components. This is more the town planning phase and it's here that nationalisation may become necessary if markets cannot behave and attempt to force differentiation on a commodity for reasons of self interest.

Everything evolves through the path but the trick is when does Gov invest or switch given we have a mass of activities evolving?  In the case of basic R&D that should be a constant, set at a high % of GDP. As soon as the market starts to exploit and custom build in a space that previously was being researched then the Gov should cut back in that space and re-invest in future explorations. Evolution during the settling phase is highly unpredictable because each step depends upon individual actors action. The best a Government can do here is in the early stages to encourage the formation of hubs or physical ecosystems with firms located near each other. Obviously this also gives opportunities to cope with other social pressures by dispersing those hubs around the country. However, beyond such encouragement the Gov must step back and interfere only when there are examples of clear market failure - health and safety, monopoly etc.

The inertia phase between product and commodity (a time of "war" as in "war with the past") is broadly predictable because it depends upon aggregate competition - see figure 3. Once companies have started to move across the boundary then defacto standards will emerge and the market encouraged to develop around. Since we can anticipate it, the Government can also effectively direct investment to encourage the change.

Figure 3 - Points of change

Hence using the above (from early 2014), as a Government then I would have been -

1) Focusing on defacto standards and encouraging industry adoption with IaaS / PaaS (i.e. cloud). Hence encouraging growth and development of a commodity market around Amazon, Google and MSFT.  This is something I'm glad to see the UK Gov has been doing with all those companies setting up operations in the UK. Unfortunately even within Government there are some departments (e.g. HMRC) that seem to be stuck in a product mentality (i.e. building rental services such as private cloud). I'd probably be looking at regulating in favour of standards around AWS, Google and MSFT and given free sovereignty introducing laws to enable Gov to declare specific APIs as open standards.

2) Encouraging creation of utility services around Big Data with Gov acting as a VC in this space with significant scale investment but less of them. This field was just at the cusp of entering the "war" and a bit of directed investment by Government to encourage new entrants providing utility forms was required to overcome inertia that many product vendors would have.

3) For the vast majority (IoT, Robotics etc) then a free market is appropriate. I certainly would be looking at providing incentives to create hubs of companies purely due to the effects that local proximity and the movement of staff have in encouraging competition. Over time, some of the markets will need directed investment as inertia barriers are reached and new entrants will need encouragement to create commodity forms.

4) In the area of intelligent software agents, just on the cusp of leaving the genesis stage (the wonder) and entering the stages of custom built and products (known as peace) then I'd probably be planning to cut back on Government funded R&D and directly investing in creating hubs of companies to exploit and develop the space. I'd be acting again as a Gov Investor but more like seed funding with a high degree of uncertainty i.e. lots of very small bets.

5) In certain areas, Materials, Epigenetics, Bio Manufacturing, Hybrid printing then Gov would need strong R&D efforts to encourage exploration. I would certainly be providing disproportionate funding in those areas compared to say Robotics or IoT in which the market players should be investing.

It's probably worth noting in the above that there are different styles of investment from a few big bets in the product to utility "war", to incentives and encouraging hubs (i.e. physical ecosystems) to Government seed funding and directed funding into basic R&D. Do note, the way you invest in the creation of a company around a new activity is very different from the way you should invest in a new entrant building a utility for a pre-existing system. There are very different risk profiles, practices and gameplay that should be followed.

The point of this post is simple, even with something like managing the market then the Gov needs to be adept at applying multiple methods and techniques at the same time - using a Keynesian style approach in one industry whilst using a Hayekian style approach in another. This is, of course, just one small aspect of social capitalism and to operate it requires high levels of situational awareness and gameplay within Government. 

If we don't become better at this gameplay, don't expect other nations to the return the favour.  In particular, China has become very good at directed investment, encouraging markets around hubs of companies and applying appropriate Government methods to the context. The level of strategic play in China vastly exceeds that which I normally see in the West.

---

For reference, there are various post describing the cycle of peace, war and wonder. Two that I particularly like are Revolution and The Best Summary.

More of, Less of and Nation States.

Many years ago, I talked about how organisations evolve caused by a particular cycle known as peace, war and wonder (an economic counterpart to Holling's adaptive renewal cycle). This cycle is caused by an interplay of evolution from competition, co-evolution of practice, inertia caused by past success and network effects creating a punctuated equilibrium.

The most interesting part of the cycle is the "War" part (as in "War with the past"). What happens is underlying activities shift from product to utility initiating the rapid spread of emerging and co-evolved practices which in turn cause new organisational forms. 

Back in 2011, the change in underlying activities (e.g. compute shifting from product to utility) was leading to a change of practice and a new organisational form, described in the diagram below. Firms were either becoming "less of" traditional and "more of" next generation or potentially putting their future at risk. I'm pinching the less of, more of moniker directly from GCHQ's most wonderful document boiling frogs.

Figure 1 - Changing organisational form.

The point of "War" (i.e. industrialisation from product to commodity) is broadly predictable in terms of when and what (general effects) but not whom. This is because the change itself is driven by competition of all market actors whereas the company that initiates it depends upon an individual actor.  In other words we can say roughly when something is likely to evolve to a more industrialised form, we can say what general effects will occur (co-evolution of practice, inertia of those stuck in previous models of products) but not specific effects i.e. what the co-evolved practice will look like or which company will lead the charge. 

For example, we knew that something like Devops would emerge but not what the detailed practices were (until they emerged) nor which individuals would drive it. However, even those small areas of broad predictability can create an advantage because we can prepare for change. Hence back in 2005 at Fotango we knew that compute would become a utility and could prepare for it. Ditto with Canonical in 2008, we knew many of the changes that would occur and could keep an eye out for it and exploit it.

These points of industrialisation (known as "war" as in "war with the past") are calculated from changes in publications and in early 2014 this data provided figure 2. 

Figure 2 - Points of Industrialisation

Now, one aspect I get asked about is "Social Change". Well, I'm expecting significant changes in society broadly 2025-2030. I re-ran the weak signals used in the above in late 2014 and still expect the same.

But whilst I'm expecting change, what that change will be (i.e. what practices will emerge), I don't know. I do expect some nations to resist it (due to inertia caused by past success) but I don't know what they'll resist and which nations will drive it i.e. who will gain the advantage and who will have to adapt just to keep up. Remember, the weak signals in figure 2 are purely driven by changes in publication - so it gives me some information of a point of change approaching but no more than that.

I put this "social marker" in more for myself, as a reminder to examine changing nation states at that time. In much the same way I had anticipated long ago that organisations would change due to the evolution of underlying components from product to utility and waited until late 2010 and early to mid 2011 to test it. I hope the same population techniques I used to create the phenotypic differences of figure 1 should be applicable to nation states. I have no ideal whether the techniques will work, I'll just have to wait and see. But this is an ongoing test for me as to the validity of the cycle.

Whilst I might not know what those characteristics are or who will drive it, I can guess based upon existing shifts in geopolitical power.

I suspect the country that will represent the future nation state is China (hence my interest in researching into China during end of 2014 and 2015). If I was going to make a stab in the dark towards what those characteristics will be, then I'd probably bet here based upon factors which appear to help drive greater competition. Some of the characteristics that I'll pick have already emerged in China but others are somewhat lacking but exist elsewhere (see figure 3).

Figure 3 - Best guess for changes in nation states.

Obviously this is a guess, not based upon data and as such is hugely subject to my own personal biases and wishful thinking. The result (if it occurs) maybe nothing like this. However, this again is more a marker for myself i.e. what I hope advanced nations will become more of which I then can compare to what actually happens (if at all).

Added 28th July 2016

Was asked one of those "China is a communist country, not neoliberal" questions. Actually, that's a really interesting topic and China (from my perspective) is firmly in the social capitalism camp and acting as a venture capitalist. However, the problem here is one of perception, see figure 4.

Figure 4 - Perception and Economic Thought

So, on average it seems when you're talking to someone in the US they often have a difficulty in agreeing with Europeans over what China is (and vice versa) because even within Western philosophy we have different ideas of where economic thought is placed.

What makes a map?

Topographical intelligence is ultimately a mechanism of communication and learning, but what makes a good map? The first thing a map has to be is visual and context specific.

Figure 1 - Visual & Context Specific.

But then any box and wire diagram is visual and context specific. However that isn't enough to learn. In order to understand a context and to learn from it (whether with a chess board or a geographical map) then you need to have position and movement of the pieces on the map. Position is relative to something e.g. position on the board or this piece is north of that piece. That something is the anchor of a map.

In the maps that I've used for the last decade, the anchor has been the user, position is visibility to the user and movement is evolution (i.e. how things change).

Figure 2 - Position and Movement.

With a visual system that is context specific and has position and movement, you finally have something which I would consider is worthy of being called a map. Without position and movement then you have a nice diagram which is not much use for learning, communication or strategic play. However, this is just a starting point. The components of a map don't have to be the same type of thing i.e. there's no reason to limit yourself to activities as you can map practices, data and even knowledge.

Figure 3 - Components and Type

Also within a map, you have flows of risk, information and money. These are also worth investigating. Be careful here, you need a map otherwise it becomes trivially easy to make efficient the most ineffective of things.

Figure 4 - Flow

Finally, with a map you can start to learn climate patterns (the rules of the game), context specific forms of gameplay (strategy) and universally useful patterns (doctrine). The point of a map is you should be able to add on climatic patterns and discuss strategic play around this. You should be able to communicate and challenge assumptions. 

Figure 5 - Climate

You are now in a state in which you can organise with structures like pioneers, settlers and town planners and apply appropriate methods. You can also work out where to attack and from that strategy. 

Strategy is all about the why of movement and this starts with where do you attack? The strategy bit is determining why here over there. This is different from the why of purpose as in "be the best tea shop in Kent". Acting upon your strategic choices (the why of movement) can also ultimately change your goal (the why of purpose) ... and there was I thinking there was something called permanent "core" (circa 2005). It's an iterative loop known as the strategy cycle.

Figure 6 - The Strategy Cycle

Of course, navigating and learning effectively without some form of map is almost impossible. Instead we find ourselves unable to distinguish between that which is context specific and that which is universal. We become the archetypal Themistocles with a SWOT.

There's an awful lot to a simple diagram like a Wardley map. Most of which we don't mark on the map as it becomes intuitive i.e. we don't have to remind people that the anchor is the user or that you have position & movement, that is taken for granted.

Figure 7 - A Wardley Map

However, topographical intelligence in business, the art of strategy based upon situational awareness remains one of those topics which are barely covered in business literature. The overwhelming majority depends upon alchemist tools such a story telling, meme copying and magic frameworks like SWOTs. It is slowly changing though and every day I come across encouraging signs.

So does mapping make an impact? It certainly seems to on a personal level and there exists a tentative link between situational awareness and success at the corporate level. However, all models are wrong but some are useful. It's a tool that I've found useful is the only claim I'll make.

Figure 8 - Personal Impact

Figure 9 - Corporate Impact

If you are completely new to this form of mapping then I've provided a basic introduction.

Observing impact

I've been running an experiment looking at how mapping changes gameplay. I've taken 181 executives through a basic mapping course (i.e. you learn how to map, learn basic economic patterns, basic forms of gameplay) but before I do, I give them a scenario.

In the scenario they are members of the board of a company. They have financial information including P&L, competitor reports, market data, operational reports and a range of strategic options that are presented using SWOT diagrams to business model canvas. They usually work as a small team of 2-4 and are asked to prioritise the strategic options or add their own. Invariably they decide to build some form of digital cloud service, invest in efficiency, expand overseas and invest in product development.

I then teach them how to map and ask them to look at the scenario again. It's worth noting they are giving no additional information on the scenario other than the ability to map. There's a noticeable change in choice with a little more confusion over what to do. The confidence of previous choice is questioned.

I then teach them basic economic patterns and forms of gameplay. Again, they are asked to look at the scenario and no new information is provided to it. At this point a radical change seems to happen. Overwhelmingly after mapping and applying basic patterns then they want to invest in marketing to "pump up" the company in the eyes of outsiders, flog it quickly and rebuild a new company. They've gone from viewing the company as having a great future to viewing it as going over the cliff and hence they need to maximise return.  Their perception has fundamentally changed by simply mapping the environment. 

It's a small sample, 181 executives and we shall see how this develops. Details of the results are provided in the figure below.

What's interesting is they now start to question the basis of previous strategic choices. It's a particular delight to watch someone map something they've done and go "crap, we need to fix this". Makes it all worthwhile to see grizzled executives change.