Wednesday, February 11, 2015

On Government, platform, purchasing and the commercial world.

Whenever you run a large organisation, the first step should always be to get an idea of what you do, how much you do, what each transactions costs and if you're a commercial company then what revenue does each transaction make. This is the most super simple basic information that every CEO should have at their fingertips.

In the case of UK Gov, there are 784 known transactions of which 700 have data on volume and 178 have published cost per transaction. These are shown in figures 1 & 2 (source - UK Gov High Volume Services Transactions)

Figure 1 - Volume of transaction

Figure 2 - Cost per Transaction

In the UK Gov they know that the cost of transaction of a memorial grant scheme claim is £1,085 but a driving license replacement only costs £7.85. Equally they know that we have over 393,000 charity applications, change of details and gift aids but only 19 Human Tissue Authority Licensing Applications. 

Any company of significant size will have a number of core transactions though it probably won't be anywhere near the scale and complexity of UK Gov.  But let us take a hypothetical global Media company with 50 or so different core transactions from the Commissioning of TV programmes to Merchandising to a Games Division to Broadband provision to Mobile Telephony to News Content to Radio to ... etc etc.

Now, each of those transactions should represent the provision of a user need. That is after all where value should be created. Now, the purpose of mapping is not just to force a focus on user needs, improve communication, improve strategic play, use of appropriate methods, provide a mechanism of organisational learning and mitigate risks but also to remove duplication and bias.

If you map out multiple different transactions, you will find that common elements exist between them and that bias exists in the way groups treat those activities. (see steps 4 and 5 on mapping and if you haven't read that post, you need to in order for the following to make sense). 

By aggregating multiple maps together, you can create a view of how things are treated (see figure 3) and then determine (by looking at clusters) how things should be treated (see figure 4)

Figure 3 - An Aggregated view

Figure 4 - How things should be treated.

There are two important parts of figure 4 - first is the duplication (i.e. how many instances of this thing exist in our maps) and second is how it should be treated (determined by looking at the clusters of how it currently is treated) which is a necessity in overcoming bias. It's not uncommon to find 18 different references and examples of compute in your maps and have half a dozen groups arguing that somehow their "compute" is special to them and unlike anybody else's.  If you're up for a laugh, try rules engines (a particular favourite of mine) if you want endless arguments by different groups about how their near identical systems are unique.

Now this aggregate map also has other purposes. Let us suppose you want to build a platform by which I mean not some "monolithic does everything" platform (a guaranteed route to quick failure) but a mass of discrete component services exposed through APIs and often associated with a development environment. When you're building those component services, the last thing you want to build is the uncommon and constantly changing component (i.e. those in the uncharted space). What you want to build is the common and fairly commodity like component services which are therefore well understood, stable and suitable for more volume operations.

Now, fortunately, the aggregated view shows you where these are in all your value chains. Sometimes, you'll find the market has already provided a component service for you to use e.g. Amazon EC2 for compute. There maybe reasons such as buyer / supplier relationship that you may wish to choose a particular route over another but at least the aggregate maps points you to where you should be looking. 

In other cases, you'll find no market solution even though a component is common and commodity like and this is your target for building a component service. For example, if Fraud Analysis is a common component of many of your value chains and assuming it's not provided in the market then this is a component service for you to consider. Now, in the case of Gov that component service could be provided by a central groups such GDS or even a department. For example if DWP was particularly effective at Fraud Analysis then there is no reason why it shouldn't offer this as component service to other departments. Ditto GCHQ and large scale secure data storage.

In fact, Government has a mechanism - known as G-Cloud - where departments could compete with the outside market to provide common components to others. All it requires is some central group such GDS to identify from all the maps of different value chains what the common components should be and to offer the opportunity to a department to provide it (and if they don't accept then GDS could look to provide it itself).

But let us not digress and go back to our Media company. We've taken an aggregated view, we've challenged bias in the organisation, we've identified duplication and determined where we're going to consume services from other groups and what services we're going to provide (hence building our platform of component services), we've even added some strategic play into the mix using open source (see previous post). We now subdivide this environment into contract components (for reasons of risk mitigation), apply the right methods and build a suitable team structure. Our map and understanding of the environment hopefully took only a few hours and now looks something like this.

Figure 5 - Our Map

Now, a key part of this map is it identifies what should be outsourced, use utility services, use other department (or platform) components, built in a six sigma like fashion with a focus on volume operations VERSUS that which needs to be built in-house, using agile techniques and in a more dynamic environment VERSUS that which should be more off the shelf, lean etc.

Obviously, we're aware everything is evolving between the two extremes due to supply and demand competition hence the necessity for a three party systems like Pioneer, Settler and Town Planner (don't get me started on the bimodal gibberish). We're all good.

BUT ... in the same way we need different methods for project management in any large scale project ideally scheduled together with Kanban, then we also need different purchasing tactics (see figure 6).

Figure 6 - Project management and purchasing methods

In other words, as with any large system where one size fits all project methodologies are ineffective, the same is true with purchasing. Any large scale system requires a mix of time and material, outcome based, COTS & fixed contract and unit / utility charging. Each has different strengths and merits as with project management methods. All activities evolve and how you purchase them will change accordingly.

So a couple of points ...

1) Long term contracts for any component is a big effing "no, no!" unless you're talking about a commodity / utility. The act will evolve and so will the method of purchasing.

2) Treating massive systems as a whole and applying a single size method e.g. six sigma everywhere, outsource everything, fixed price contracts everywhere are a big effing "no, NO!" with cream on top. This is almost guaranteed to cause massive cost overruns and failure.

Unfortunately, prior to the coalition, we had a long spell in UK Gov IT where single size methods such as outsource everything, fixed price contracts became the dogma based upon a single idea that if you can get the specification right then it'll all work. 

This idea is unbelievably ludicrous. Take a look at a map of any complex system such as the IT in High Speed Rail (see figure 7).

Figure 7 - Early Map HS2 IT

Whilst some elements of that map are industrialised (suitable for outsourcing to utility suppliers with known specifications), large chunks of that map contain elements that are uncharted i.e. they're novel, uncertain and constantly changing. We DON'T know what we need, the specification will change inevitably, these components are about exploration and no amount of navel gazing will write the perfect specification. If you plaster a single size method across the lot then you'll inevitably incur massive change control costs and overruns precisely because a large chunk of the map is uncertain, it will change!

However, alas we had this single size idea and not only did the single size approach lead to massive cost overruns and a catalogue of IT disasters, it also had the nefarious effect of reducing internal engineering capability. This had serious consequences for the ability of Government to challenge (something we covered in the 'Better for Less' paper and the need for an "intelligent customer" and a Leverage & Commoditise group to ensure challenge). 

In UK Gov, they now have that more "intelligent customer" in the guise of GDS and the Digital Leaders Network. We also have that challenge function in GDS spend control. 

I kid you not, but prior to this I saw an example which was as close enough as it makes no difference to ...

1) Department wants to do something, asked their favourite vendor to do the options analysis 
2) The options came back as Option A) Rubbish, Option B) Rubbish, Option C) Hire us - Brilliant. 
3) The department then asked how much for option C) and haggled a small amount on a figure that was in excess of £100 Million. 

This was for a project that I couldn't work out how you could spend more than £5-£10 Million on. This was not "challenge". This was handing over large fists of cash. That has now fortunately changed. However, you have to be extremely careful to avoid one size fits all methods otherwise it's easy to fall down the same trap. 

So, this brings me to purchasing. Any group involved in this has to know how to use multiple purchasing methods to get the best result. It also has a need which it supplies. That need is "to help the Engineering capability deliver and develop efficiently and effectively". 

This is key. 

Purchasing has to be subservient to Engineering, it has to support it, advise on the best use of contracts and enable it to do the job. If it isn't subservient then you run the future danger of a one size fits all policy being applied and the same mess highlighted above being created.

Now, those various types of work (whether T&M, outcome based, COTS, fixed price or utility) can often be labelled "services" and there is no reason why you can't have a competitive market of these. In the UK Gov case, G-Cloud seems a suitable framework.

So, why do I mention all of this?

First, there was this discussion I had with @harrym on the Digital Services Framework and UK Gov Purchasing.

Then, there was this rather disturbing post by Chris Chant on everything unacceptable with UK Purchasing 

Now, this disturbs me. The real concern is the accusations that CCS is overstepping its mark, not acting as the support function that it is and imposing policy. If that is the case, I can see why Chris argues that CCS should be scrapped or at least (in my view) absorbed into GDS.

Oh, why the commercial world bit in the title?

Let's be honest, I lied at the beginning. When I said this is the "most super simple basic information that every CEO should have at their fingertips" then we all know that the vast majority of large companies in the commercial world have no idea what transactions they do and what volume of transactions they have beyond some basic revenue lines.

Without this, it's fairly farcical to assume large corporations have any concept of user needs, mapping, situational awareness or strategic play beyond simply copying others. This sort of discussion is light years ahead of where most major corporates are (there are a few exceptions like Amazon). The problems that UK Gov faces are problems that most corporates could only dream of. 

I only put that line in because some troll often pipes up and says "UK Gov should be more like the commercial market". Yeah, for the vast majority that is clueless and hopeless, waiting to be disrupted whilst pretending they're master chess players without ever looking at a board. No thanks, this is our Government we're talking about.