If you’re coming here for the latest and greatest in project management techniques, then I’ll stop you there. In this article, I will go through some basic principles that are well over fifteen years old and have been used from heavy engineering to low earth orbit. There will be no new-fangled concepts or terms, just basics that have been tried and tested.

Nor will I spend my time waffling on about the sorry state of many technology projects or outsourcing efforts. Failure rates above 70% have been consistent since the 1990s. They are almost as common as the excuses:- communication issues, mismatched expectations, lack of clarity on purpose, difficulties in managing remote teams and cultural differences.

I will use an example from 2012: the building of HS2 (high-speed rail) in a virtual world. I like examples, and this project, run by James Findlay, was delivered on schedule and under budget. I’m picking it because it’s old, and HS2 doesn’t have a good record in the public eye.

However, before we go through the tips, I want to use that same example to show you what was typical in government in 2010–2014 (when I was more active in the UK government). I don’t know the current situation; you must ask them, not me.

How things used to be done.

The image below is the graph of the system for building HS2 in a virtual world; it’s the sort of diagram you’d often find on whiteboards in engineering departments. It contains multiple connected components with many acronyms, such as PIMS for Project Information Management System or CRM for Customer Relationship Management system.

A systems graph

Now, to “ensure” value for money, it was common in 2010 for organisations (both commercial and governmental) to outsource most of the project. This was often done by breaking the project into smaller groups of components (in this case, I’ve called them LOTs) with components that sounded similar to each group grouped together, e.g. a LOT for infrastructure or a LOT for engineering.

A detailed contract with mountains of specifications was then written for each LOT. People had learned that projects often massively ran over cost because of poor or faulty specifications; hence, a lot of effort went into this. Once the contract was written, these LOTs were put out for tender.

Breaking the project into LOTs

80% of the time, as a rule of thumb, these LOTs would then massively overrun in both time and cost because of poor or faulty specifications despite our best efforts. When you specify what you want in a contract, if you change your mind, that will incur costs. The process for this is known as change control.
People would moan about the specifications as the cost spiralled, often doubling, tripling or more. Those people would then commit to never making the same mistakes again, and next time, the specification would be better.

It wouldn’t.

The same cycle would repeat.

This has been going on for decades.

I was stunned when I first encountered this problem (in a commercial organisation in 2007). It’s a straightforward thing to fix. Since then, I’ve watched billions being wasted unnecessarily.

How to fix things

The first thing you do with any project is to focus on the project users by asking the questions:-

1) “Who are the users?”
and
2) “What are the users’ needs?”

For the above graph, I’ll assume that a good job was done identifying users and their needs, and hence, I’ll create an image with that information extracted.

Users and their needs.

In practice, it’d be a great month if 30% of the organisations I meet did this. You will be stunned at how many multi-million pound projects are undertaken with no clear idea of the actual users or their needs.

Once you’ve identified the users and what they need, you need to ask another two questions.

3) “What capabilities do we need to meet those needs?”
and
4) “What components do those capabilities need?”

Capabilities, users, needs and components are all subtly different forms of … components. The key is understanding the relationship, i.e. this component needs that component. In other words, a USER needs this USER NEED, which needs this CAPABILITY, which needs these COMPONENTS, and on and on. I’ve taken our example from above and expanded the components we are looking at.

A chain of needs

Unsurprisingly, we call this a chain of needs. You should never forget that those relationships can come in multiple forms: A needs B or A needs an absence B, or … there are many.

However, let us keep it simple and stick with A needs B.

This chain of needs is a graph, and whilst it is a significant first step, it doesn’t quite get us where we need to be to fix these contract problems. The next step we need to take is to turn this graph into a map, and we do that by simply asking the following question:-

5) “How evolved are these components?”

That component might be knowledge, data, practice, a value or an activity. We use different labels to describe the evolution of each component type. I’ve provided a table of those labels and highlighted the labels I will use.

Labels for different stages of evolution

Using those labels (the x-axis on the map), going through each component and asking, “How evolved is this?” I’ve turned our graph into a map. This was the high-level map that I created with James Findlay in 2012.

A map of building HS2 in a virtual world, 2012

While creating the map, you’ll often find yourself adjusting the chain of needs — that’s perfectly normal. It is perfectly normal to adjust the map because it is always an imperfect landscape representation. We are not trying to create the perfect map. We are attempting to put our assumptions about a project onto the map so that others can challenge it or tell us what we’re missing or where we are going wrong.

In practice, we’ve been challenging it all along. Go back and look at the questions:-
1) “Who are the users?”
2) “What are the users’ needs?”
3) “What capabilities do we need to meet those needs?”
4) “What components do those capabilities need?”
5) “How evolved are these components?”

That last question is always fascinating. More often than not, I come across corporations custom-building stuff, a commodity. But why? Alas, most organisations tend to run on stories, and the problem with stories is we associate good leaders with good storytellers. When you challenge someone’s story, you say, “You’re not a good leader”. This never ends well.

With a map, I’m saying the map is wrong, not the person. Challenging the assumptions in a map is much easier than challenging the assumptions in a story.

Suppose we do that and end up with a good enough map that has been challenged many times. How does that help fix our contract problem? Using maps, we have learned many technological, economic, social, and political patterns. Few to discuss in a post. However, I will discuss one pattern: “There is no such thing as one size fits all”.

When we look at the map through the lens of project management methods, it turns out that some methods are more robust in different places of the map, i.e. agile / XP works best in the more genesis/custom built phase because it reduces the cost of change and change is the norm. On the other hand, Six Sigma works best in the more industrialised/late product/commodity space because it reduces deviation, and you don’t want deviation with a commodity. In between Lean / SCRUM / MVP work best because we’re still learning about the space. I’ve summarised this in the following image.

Different methods work best in other parts of the map.

Now, we can ask the question:-
6) “How are we managing these components?”

On the map, we can show how we think we should be managing it, i.e. where we should be outsourcing and where we should be building in-house with agile techniques.

Applying different methods to a map

This is precisely what James did, understanding the context and use of appropriate methods. It’s how he delivered the project on budget and on time. He is not the only one; there have been many projects before and since.

Of course, we can go further. We can start breaking the project down into smaller groups and continuing to apply appropriate methods to determine what parts we can turn into a LOT structure for outsourcing to a market through a detailed specification, what we need to build in-house on a time & material basis and where we need to focus on off the shelf products or partnership or maybe even a LOT contract but with a different focus such as outcome rather than specification.

As an example, I’ve done this in the following map. This is your starter for ten, a point of conversation on how we manage something:-

Breaking the project into appropriate contracts

Now that I’ve shown you how to do this let us return to our earlier example and demonstrate what usually goes wrong with projects.

Why do projects go so wrong?

We usually compound this mess with a fatally flawed contract, needing to understand users, their needs, the components involved, the methods required, and failing to challenge what is being planned. This is so common that I would not view 80% of projects failing as a problem but as the norm. Instead, I view the 20% succeeding as a remarkable feat of human achievement. Somehow, they succeed despite every action we take to cause them to fail. Heroic stuff.

To demonstrate the contract problem, let us take one of the LOTs from our original proposed contract structure. In this case, I will take LOT 1 — Engineering.

LOT 1 — ENGINEERING

Now, let us overlay that LOT onto the map we created.

LOT 1 overlaid on the map.

What we are trying to do is to specify in detail the entirety of LOT 1 before putting it out to tender. However, there should be an obvious problem. Part of LOT 1 (the more commodity elements) can be defined and specified, but many of the project components are towards the uncharted space, which means we are still learning. These parts can NEVER be specified precisely because they are changing; they are uncertain by nature.

By trying to create a specification around LOT 1, I have made a cast iron guarantee that there will be massive amounts of change and, hence, cost overruns. No amount of promising to specify future projects better will overcome the issue that you can’t precisely specify the uncertainty. The problem is the contract (see map).

LOT 1 guarantees massive change control costs.

Naturally, this almost guaranteed failure is only compounded by other faults within our purchasing systems. These include:-

1) A tendency to focus on costs alone. 

While it might seem prudent, it will drive vendors to use your projects as training grounds for their people, who are put into more lucrative projects. You can often spot this if you have access to the GitHub of your project by the rapid turnover of people.

2) An assumption of talent. 

For some bizarre reason, the idea often exists that a vendor will have done this thing many times before. That will certainly be true if you’re talking about a commodity component such as computing provided by a utility cloud like AWS. But if your project is a complicated structure spanning components from genesis to commodity, your assumption of talent and expertise can rapidly be faulty. They can easily be less capable than your own people.

3) Shifting the risk to others. 

Whilst using an “approved” contractor and an “approved” process will reduce the political risk for yourself by providing someone else to blame, this does not mean it reduces risk to the company. If you keep failing despite repeating the same process, question whether better specification is the answer.

Wrapping it up

This process I’ve outlined above has been successfully used on several projects for over 15 years. There is no magic here, no secret sauce, and nothing too taxing because those six simple questions are at its heart. Anyone managing a project should be able to provide a reasonable, though never perfect, answer.

These questions are my simple tips for managing any project.
1) “Who are the users?”
2) “What are the users’ needs?”
3) “What capabilities do we need to meet those needs?”
4) “What components do those capabilities need?”
5) “How evolved are these components?”
6) “How are we managing these components?”

Now, as I said, this is a fundamental article. There are many paths to failure, but if you get this lot wrong at the beginning and end up with a contract structure based upon specifying that which cannot be specified, you’re likely heading towards a costly mistake.

Additional notes — Graph versus Map

For those who don’t understand the difference between a graph and a map, the following images should help. The top three images are all graphs, and they are the same. They each contain three nodes connected by two relationships. You can rotate the components by any degree you wish and remain identical.

Graph versus Map

However, the bottom three images are all maps and completely different. If you rotate them, they also mean something else. The distinction between a map and a graph is that in the map, the space has meaning. This is why, with a map, you can’t just move a component (keeping the relationships the same) and expect the map to have the same meaning. It’s also why they are great tools for understanding territorial, economic, social, technological or political landscapes.

Almost everything you’ve ever seen in business that calls itself a map is instead a graph:- mind maps are mind graphs, business process maps are business process graphs, etc. What gives space meaning is the anchor (i.e. the compass or, in our maps, the user) combined with the position of pieces (i.e. this is west or east of this, or in our maps that this is higher up the value chain than that) and consistency of movement (i.e. north is north or in our maps, this is more evolved than that).