Thursday, December 07, 2023

How to organise yourself - the dangerous path to Explorer, Villager and Town Planners

Explorers, villagers and town planners (EVTP) is a system for organising a company that is having to deal with constant change due to an evolving environment. It's over 18 years old, it's a small part of the concept of mapping and it has been used successfully in several companies, BUT it is a difficult path to follow. I used to call this system - pioneers, settlers and town planners - unfortunately the colonialist overtone of those terms made it problematic for many and with good reason.

At the heart of EVTP is the idea that as things evolve then the characteristics of such things also changes e.g. the first ever computer is not the same as a highly industrialised cloud instance of a virtual server. Because of this, the manner in which we manage and deal with those things must also change as it evolves e.g. the techniques by which we manage computing infrastructure at a hyperscale today are not the same as the techniques we used in the past. To effectively cope with an evolving space, therefore, requires different aptitudes (as in skillsets such as engineering or finance) and different attitudes (as in mindsets). The three essential mindsets that we need are explorers, villagers and town planners and they can all be brilliant people.

Explorers are brilliant people. They are able to explore never-before-discovered concepts, the uncharted land. They show you wonder but they fail a lot. Half the time the thing doesn't work properly. You wouldn't trust what they build. They create 'crazy' ideas. Their type of innovation is what we call core research. They make future success possible. Most of the time we look at them and go "what?", "I don't understand?" and "is that magic?". In the past, we often burnt them at the stake. They built the first ever electric source (the Parthian Battery, 400AD) and the first ever digital computer (Z3, 1943).

Villagers are brilliant people. They can turn the half baked thing into something useful for a larger audience. They build trust. They build understanding. They make the possible future actually happen. They turn the prototype into a product, make it manufacturable, listen to customers and turn it profitable. Their innovation is what we tend to think of as applied research and differentiation. They built the first ever computer products (e.g. IBM 650 and onwards), the first generators (Hippolyte Pixii, Siemens Generators). 

Town Planners are brilliant people. They are able to take something and industrialise it taking advantage of economies of scale. This requires immense skill. You trust what they build. They find ways to make things faster, better, smaller, more efficient, more economic and good enough. They build the services that explorers build upon. Their type of innovation is industrial research. They take something that exists and turn it into a commodity or a utility (e.g. with Electricity, then Edison, Tesla and Westinghouse). They are the industrial giants we depend upon.

Now we have the basic idea: why should we implement this, and how do we do that? Well, the answer to the first question is - don't implement it. The process is not easy and you will fail if you don't follow every step. The only reason you would consider implementing the structure is if your organisation is so messed up with infighting, inertia, and inability to change that when compared to your competitors then you are seriously worrying about the organisation's future. In all other circumstances, keep whatever organisational structure you have "as is" because nobody enjoys a re-organisation except leaders who want to feel they are doing something and management consultants who get paid for it. Leave the org alone.

If you really must change the organisation, then these are the steps you have to take.

Step 1 - Principle
Take the following principles listed in the doctrine table (figure 1), gather a diverse group of people in your organisation and ask them, "How good are we at these?". Get them to vote on the table, add "red" or "green" or "blue" circles or squares or stars for things that you are good or poor at. Just make sure you create a label for what the symbols mean (see figure 2)

Figure 1 - The doctrine table.

Figure 2 - A hypothetical filled in table (I'm not sharing actual client's information).

The purpose of filling out the table is just to give you an idea of what state you are in and where you need to focus. In practice, 30-40 people spending 30 minutes filling out the table is enough to give you an idea. It's also worth spending an hour discussing the results with the group. The conversation is usually the most enlightening part. However that means you could have wasted as a group a total of 60 hours work on doing this, which is a lot but it's better to do this first and decide that you're ok and don't need to go further rather than pursuing the rest of the steps.
Step 2 - Improving Awareness.
This is the most time-consuming part. It's where we start at the bottom of the doctrine table, fixing every principle and working your way up. Let us assume that you're hopeless at every one of those principles which is quite common with organisations today. One fast track method of introducing principles is to use Wardley Maps.

Whenever you build a Wardley map, the first thing you need to do is identify the users. This is good for the map and it's also one of our basic principles. The users - whether it's public or the business or government regulators or all three or something else need to be marked on the map (see figure 3) as they are the anchor around which the map is built. Rather unimaginatively, I've added one user to our map called users.

Figure 3 - Know your users.

Now you have your users, you need to work out what their needs are. These you add to the map (figure 4) and fortunately, that's another principle ticked off. I know I'm a bit blase about this and actually getting to understand your user needs involves tasks like talking to them (you'd be gobsmacked at the number of companies who fail at this). I know it takes work.

Figure 4 - Understand your user needs

Of course, meeting those needs requires many components, i.e., an entire supply chain (of physical, technological, data and practice components) is necessary to make it happen. Since those components are all evolving we need to do several things. We need to understand what the components are, the relationships between components, how this connects to use needs, and how evolved those components are. How you manage, purchase, finance and build a totally novel and new concept that has never been done before is radically different from how you manage, purchase, finance and build a highly industrialised commodity. If someone says to you that "we should use agile everywhere" or "we should use six sigma everywhere" then they are either inexperienced or a management consultant trying to flog you their latest fad. Either train them or show them the door.

To recap, we need to understand the details and what is being considered (how evolved this). I've added that to figure 5. This is a Wardley Map.

Figure 5 - Understand the details and know what is being considered.



All maps are imperfect representations of a space but they enable people to expose their assumptions in a manner which allows others to safely challenge. I say safely because most organisations run on stories whilst telling everyone that great leaders are great story tellers. When you challenge someone's story, you're actually telling them that they are "not a great leader". People tend to get defensive at this. By putting the story into an imperfect map, I can challenge the map without challenging the person. I can say, I think this map is wrong. It might be missing a component, missing a relationship or having a component which is considered less evolved than it really is.

In figure 6, I'm challenging where we've placed app certification. You probably won't believe this, but in 2023, we still have organisations custom-building their own private clouds in a world where finally, most people have accepted that the public cloud is a utility.

Figure 6 - Challenge assumptions

When it comes to challenge, the cheat sheet for mapping can be helpful. Simply look down the list of properties, marking off each one for the component and that should give you an idea of how evolved it is (see figure 7)

Figure 7 - Cheat Sheet

Building a map is relatively simple process that gets easier with experience - understand the users, their needs, the components involved (SBOMs of Software Bill of Materials will help), how evolved the components are and then challenge the map. You can even get ChatGPT4v to help.

However, with any large enterprise then you're going to have a lot of maps and those maps are dynamic because they exist in a constantly changing and evolving landscape. If you try to map out the entire landscape before you do anything else - you will fail. It's like painting the Forth Bridge, a never-ending task. Which means we need to take an iterative approach to mapping.

The best way to do this is to create an Intelligence Function (or what I called Spend Control in UK Gov - bad name, my fault, don't repeat that). Let us start by asking a question. If you can map a system in a few hours, then what scale of project do you think it's worth spending those few hours understanding the users, their needs, the components involved, how evolved the components are before starting on the project? Pick a figure - a £1M project, a £100K project, a £10K project or every project? I suggest starting at £100K to begin with, you can change it later. Whatever figure you decide is your limit.

You now need a bit of policy. What you're going to do is create an Intelligence Function in your organisation and introduce the policy that any project over your limit must be mapped by the group building it and that map must be taken to the Intelligence Function for challenging before building of the project starts. People will complain. They will ask why do they need to do this. You just simply need to say "If you're spending £100K on a project, I just would like us to understand who the users are, what their needs are, what components are involved and have someone challenge it before we begin. It's worth the few hours". Many will still grumble. Be firm.

I will warn you now that a lot of politics will be involved. People will try to find exemptions; they don't want to be challenged in any way, and they certainly don't want to expose that they are spending £100M on a project that they have no idea what their users or user needs are. To help reduce the political pressure, make it clear that this is only for new projects, new re-compete, and new upgrades and doesn't affect the existing pipeline. Eventually all that stuff will come through your Intelligence Function but let time work its magic.

The second problem is your Intelligence Function is going to be overwhelmed. That's okay; let them challenge what they can and allow other projects to pass through with a "no comment". Over time, you'll need to staff it up appropriately.

Lastly, it's an Intelligence Function. It will give advice on the project by challenging the map. The group building the project should have the autonomy to ignore the advice if it wishes. But make it clear that this is pre-mortem challenge and all projects will go through a post-mortem learning. If the project fails because the advice was ignored, questions will be asked.

Getting this up and running is not a simple task. Give yourself a year before it is smoothly running. All the time, you'll be improving the situational awareness within the organisation of the landscape you are operating in.

Step 3 - Advice
As more maps are created, and there is less fighting over mapping and the challenge introduced, then you can extend what the Intelligence Function does. With many maps, you can start to identify duplication in projects - don't underestimate how much large organisations duplicate the same thing or the same process, often custom-building it in different ways. The record, to date, is one financial institution that has managed to rebuild risk management over a thousand times in the organisation. We don't quite know how many times because we stopped counting. Duplication of effort on the order of a hundred times or more is quite common. Be prepared to be shocked, that's the horror of looking.

With many maps, you should be able to start to spot duplication. Often you'll find the same component doing the same thing (which can be spotted by relationships being the same on different maps) but called a different name. We love our acronyms and technobabble. Finance tends to be the worst for this btw. A bit of duplication is not a bad thing (for reasons of avoiding systemic failure) but it should a conscious choice. Anyway, the Intelligence Function should be able to start not only challenging on the map itself but also suggesting that one group goes to talk to another group because they are building similar components (see figure 8).

Figure 8 - Spotting duplication.

With enough maps, you can even start to build profile diagrams and spot candidates for common services as opposed to the old-fashioned way of some executive sticking their finger up in the air and declaring we need this as a common service. The maps become both a common language and a source of data for identifying opportunities. That also happens to be two principles but for now, you can leave those until a bit later.

The other quick win you can introduce refers to methods such as project management and contracts. Since all components are evolving and we can actually visualise the components then we can apply appropriate methods to map (see figure 9 and 10). 

Figure 9 - Different Methods works in different contexts

Figure 10  - Applying appropriate methods to a map

This should stop all the management consultant led "lets agile everything" or "lets six sigma everything" or "lets [add some term] everything" fads that they love to create. If you haven't already fired all the management consultants by now, this will be a good time. If anyone grumbles just say use ChatGPT because its cheaper than ChatPPT which is what these management consultants mostly are.

This principle of appropriate methods doesn't solely apply to project management; you'll find you also need different methods for purchasing, finance, operations and even marketing.

The other low-hanging fruit is contract structure. Big outsourcing projects tend to fail because we try to cover an entire project with a single contract structure whilst that project contains multiple components at different stages of evolution. The problem is you can define the industrialised components well and hence use a very structured / rigid form of contract for those components. Unfortunately, the components in the uncharted space (e.g. the genesis of the novel and new to the custom-built) can't be defined. If you attempt to, you will inevitably incur massive change control costs. This is NOT a fault of lack of specification but instead trying to specify what is still evolving and changing. If you look at figure 10, by mapping it and applying appropriate methods, you've already neatly broken it into ten different contracts with defined interfaces for which appropriate contract structures can be applied.

Give yourself another year to get the advice aspects of the Intelligence Function up and running. OK, you might be thinking - "hang on, that's two years" - well, it is. But two years to go from a lack of understanding of users, user needs, components involved, lack of challenge combined with inappropriate methods, massive duplication and flawed contract structures to something which is reasonably sane or at least is on the path to being sane is very fast for a large organisation. Especially one that is used to executives pondering for a year or so on what to do before publishing a fairly meaningless "strategy" with the help of management consultants based on no situational awareness whatsoever.

At this point, you should already have seen visible improvements in the organisation. Just to check, it's worth re-running that doctrine table test (step 1). Given you also now have some level of visibility on the landscape and hence you can observe the environment, it's now worth talking about measurements. There are many measurements you can add such as price per transaction or delivery times or success rates. You might already be using some. Don't overload them, pick a handful and start using that to monitor progress from now on.

Step 4 - Improvement.
Two years in, you should now be pretty good at all the phase I principles in the doctrine table and have at least caught up with those lucky few organisations (and there are few) who were good at this in the first place. The bottom of the table should be looking rosy. Now, you've got the long climb up the table (figure 11),

Figure 11 - Climbing the Doctrine Table

You want to start introducing all these principles into the organisation. You want to recruit against them, reward behaviour that demonstrates them and build up slowly. Complete phase I first (if not already) and then move into phase II and then up.

During Phase II, you'll hit one principle which is Think small teams. At this point, you want to start introducing the concept of small teams built around the maps (see figure 12). 

Figure 12 - Think Small Teams

There's already a lot of good work out there on how to do this including Amazon's Two Pizza model. As a general rule of thumb, teams in the uncharted space of genesis / custom built should be 3 to 5 people but by the time the project becomes more industrialised (i.e. late product, more commodity) then Two Pizza (12 people) or slightly larger is fine. If the team is too large, use the maps to help break it down into smaller components. Use the relationships in the map (lines between nodes) to determine the fitness function (i.e. service delivery, dependencies and metrics) of each team. Do remember, that all the components are evolving. 

After about a year, you should start floating the idea of their being different attitudes (i.e mindsets) and not just aptitudes needed in each team. Don't however, do anything yet.

Step 5 - Explorers, Villagers and Town Planners.
You should be three years in, those bad old days of unknown duplication, no-one focusing on user needs, regular out-sourcing failures, management consultant fads that never worked out and the general all round strife should be past memories. Even issues like tech debt should be slowly clearing out of the organisation. When you run that doctrine test, most principles should look positive and you should be feeling pretty happy about things. If you're not, keep going with Step 4, you're not ready yet.

At this point, we will take the most dangerous and experimental step. We will re-organise the entire organisation to cope with constant evolution and change. Before you do this, consider stopping. If the organisation is doing well against competitors then take a break. Don't inflict any more change.

If you take the path, this means two things.

1) When we populate teams, we not only include the right aptitudes (engineering, finance, marketing, operations etc) but also the right attitudes (i.e. explorers, villagers, town planners)

2) We introduce a mechanism of theft.

The basic characteristics of explorers, villagers and town planners are outlined in figure 13

Figure 13 - Explorers, Villagers and Town Planners (EVTP)

The way it works is as follows. You create a parallel structure to the company which consists of pools of explorers, villagers and town planners. You allow people to select their own attitude and to change if they wish to. Then, when new projects appear, you populate the cells with not only the right aptitude but also attitude (all explorers together or all town planners together, etc). Let that bed down for a little time.

Then you introduce a system of theft. You allow teams of villagers to declare they are taking over some project run by explorers in order to turn it into a better product. You allow teams of town planners to declare they are taking over some product run by villagers in order to turn it into a utility-like service. As this starts to grow, you dismantle the rest of the organisational structure into the pools and introduce guilds across all the pools, i.e. guilds of engineers and guilds of finance (See Figure 14).

Figures 14 - Pools and cells.

There will be a period of readjustment and chaos as you re-organise. This is not for the faint hearted, and if you have any doubts, keep focusing on step 4.  This leap into step 5 should never be done unless you have almost all the principles in place and operating well. If you don't have those principles in place, I can guarantee you will fail.

What you're aiming for can be seen on the map. It's a constant process of team stealing from team, which replicates the evolution that occurs outside the organisation (see figure 15).

Figure 15 - Dealing with constant evolution.


Of the few who have gone down this path, the results have been remarkable and as one exec told me "the organisation found itself naturally falling into the structure". I cannot emphasise enough that there are too few examples to be confident of the structure and there is a litany of failure of those who have tried without principles in place. One of those failures was an organisation that got suckered into Gartner's bimodal nonsense and thought they could get themselves out of the mess by adding another bit. I did warn them. Don't do this. Fix those principles first.

This structure was right on the edge of what was possible in 2005 (when it was first used) and remains at the edge today.

Step 6
There's bound to be something beyond this. I don't know what it is. I've never been beyond EVTP. You will have to experiment to find it, and most of those interesting experiments are occurring in Chinese organisations. So keep an eye on organisations like Haier.