Facebook, the Select Committee and one in a million.

Unless you've been hiding under a rock, you probably know about Facebook's cavalier disregard of informed consent when experimenting on people. Now this alone should be enough for Governments to investigate and I'm glad to see that this is being raised to the Commons media select committee by Jim Sheridan MP.

What we already know :-

1) Facebook has the means to target individuals through numerous dimensions. 

2) It has passively examined moods. All Facebook users in the 100 most populous US cities were examined between January 2009 and March 2012.  Now, this is perfectly normal market research.

BUT ...

3) It has actively attempted to alter 700,000 users moods without informed consent. I understand Facebook claims that the one line in the Data Use Policy is informed consent but I doubt any reasonable person reading those T&Cs would. That the result was relatively minor does not excuse experimenting on people without informed consent. Oh, and by minor the 'test induced negative emotions in tens of thousands of people'.

4) The experiment was claimed to have been approved by the local institutional review board on the grounds that "Facebook apparently manipulates people's News Feeds all the time". So far, no-one has quite yet explained what this other manipulation includes and what other experiments have been conducted. Is this other manipulation simply A/B testing (providing two different versions of the same page) to improve interaction with the site? Or have they've been trying to deliberately alter the state of mind of other users in other experiments?

5) It has undertaken political experiments. In November, 2010 demonstrated it could get people to cast a vote. Now, that the experiment was encouraging people to vote is something that many would see as positive. But, it should be noted that the experiment shows that Facebook can, if it so chooses, have an impact.

So now we have to ask ourselves whether it is conceptually possible for Facebook to target supporters of one party, encouraging them to vote whilst targeting supporters and friends of another party and using 'mood contagion' to spread feelings of sadness or hopelessness and discouraging a tendency to vote? Certainly, it appears to have the capability but is this really likely to happen?

If you ask me, I'd put the probability of this happening at about one in a million. But then again, if you asked me last week what is the likelihood of Facebook conducting psychological experiments to alter mood on nearly 700,000 people without any clear informed consent then I'd have put this at one in million as well.

Facebook needs to answer some really hard questions not just on informed consent but also what it is capable of doing, what it has done and what it is doing.  Given the response of an investor sarcastically dismissing this experiment as some sort of joke and the dismal excuses that consent was given in the T&Cs then I'm not convinced that Facebook can be trusted to police itself. 

In this instance then either legislation or in the worst case a freedom's forge effort to replace Facebook with a European service might be needed. I hope the Commons media select committee takes up the call and digs deep. Facebook has form here and do we really want to allow this to get out of control again until we need to have another Leveson inquiry? Just imagine if the news media were pulling stunts like this and realise this form of personalised control of news is more invidious.

Of course, they'll probably say 'sorry' for now and how they'll never, ever do it again ... well, of course they won't ... not until the next time.

Facebook and informed consent

By now, many people will probably know that Facebook conducted an experiment on around 700,000 users by manipulating their news feeds to see whether it impacted their mental state, their mood. Now the issue is not the size of the experiment nor the merits of the experiment nor even the result. The real issue is informed consent and whether users were aware and agreed to such experimentation.

It would have been relatively trivial for Facebook to have sent a message to a million odd users saying 

'Facebook would like to conduct an experiment on how the content of newsfeed impacts emotions. In order to do this, Facebook is looking for volunteers for such a study. At some point in the next year we will manipulate the feeds of those volunteers either positively, negatively or not at all (the baseline) for a period of one week to observe any change in behaviour. It should be noted that there exists the possibility that some volunteers moods will be altered adversely during that time. The purpose of the research is to determine ways of improving newsfeed content to create a more positive experience and mood. The results of the work will be published and user details will be anonymous. Volunteers may request to see how their feed was altered and any impact on them after the study. Do you agree to volunteering for this study?'

Now the above I wrote quickly in a few minutes and could be vastly improved. However, the purpose of it would be to gain consent and agreement to an experiment to change your mood in a given timeframe for a specific purpose. This is what we call - informed consent. You know what is happening and you agree to it.

In the case of Facebook it didn't inform people specifically because it argues that when you sign up to the T&Cs then you agree to such experimentation. There are two problems with this argument.

1) I doubt if you asked a 100 people to read the Facebook T&Cs that any would agree with the statement that 'By agreeing to the T&C's then you're agreeing to Facebook conducting psychological experiments on you'. Most would probably react with shock and ask 'where does it say that?'

The single line in the Data Use Policy states

we may use the information we receive about you:
...
for internal operations, including troubleshooting, data analysis, testing, research and service improvement.

This is unlikely to qualify as informed consent as the average user would more likely expect this to mean functional changes, speed performance or reliability of the service rather than psychological experiments.

2) You have no option but to agree to the T&Cs or you cannot use the service to communicate with friends. In fact, looking back at earlier T&C's I've yet to find when the above line appeared. If it is the case that this 'research' line appeared in later T&C's then given the widespread use of Facebook to communicate with friends then this is not consent to experimentation but coercion to experimentation e.g. let us experiment on you if you want to continue talking with friends on Facebook.

Whichever way I look at this, I can't see how it can be considered informed consent. At best, it's uninformed consent at worst it's uninformed coercion. Neither is ethical.

But why would you do this? Well, either it's complete arrogance and disregard for users or worse it's that basest of all reasons - money. As anyone who has conducted such experiments before will probably tell you, with the exception of high minded individuals then most people ask 'What do I get for this?' 

Yes, if you're going to experiment on people then they normally expect to be paid. However, avoiding cost is no better an excuse for avoiding informed consent than saying there's a line in the T&C or arguing the results don't mean much. The only partial excuse I can think of is 'we were stupid'.

The experiment conducted is in my opinion truly awful, not because of the results but because of the lack of informed consent. I read a lot of apologists for Silicon Valley saying this is ok. Well, fortunately it's not for them to decide.

I'm glad Jim Sheridan has asked the Commons Media Select Committee to investigate.

-- Update 1st July 2014

When I was looking for when the T&Cs had changed to include the 'research' line, my concern was that this experiment was not just uninformed consent but instead uniformed coercion i.e. users didn't have any real choice but to allow Facebook to experiment on them on threat of losing the service. There was no opt-out. However, little did I imagine that Facebook (or is that FacePalm?) would apparently change the T&C's after they had conducted the experiment.

So, all those apologists for Facebook claiming it was in the T&C's better start trying to think up another excuse. Not only was this not informed consent by any stretch of the imagination, it wasn't even uniformed coercion. It seems to be a new low of Facebook doing whatever it likes without any concern over what it had agreed with, or informed the users of.

Yes, the effects were minor (if you consider tens of thousands of people affected as minor) and yes, the furore might mean Facebook won't publish further research. Which is almost certainly why we will need regulation to stop experimentation on the sly.

No, this isn't simple A/B testing as most users would reasonably expect sites to try different functional pages in order to improve user experience / interaction / ease of use. What users don't expect is for companies to manipulate their personal newsfeed in order to initiate a change of their mental state.

The argument that Facebook was only trying to change your mental state in order to improve your experience with the site is the most tenuous bit of arse covering I've read in a long time. Given the generalised, flimsy and buried line in the T&C's which supposedly permitted this didn't even exit at the time of the experiment then I hope the Commons media select committee hauls Facebook over the coals for this.

Apparently a Facebook spokesperson told Forbes "When someone signs up for Facebook, we've always asked permission to use their information to provide and enhance the services we offer." Well, that's true of many services but I can't see how any reasonable person would interpret this as "When someone signs up for Facebook, they give us permission to run psychological experiments on them including manipulating their newsfeed with the intention of negatively altering their mood".

Oh, and finally people keep on pointing out that advertisers do this all the time. Well, yes they do but that's adverts and we know that advertisers manipulate and we have legislation that even covers this. Few of us trust what advertisers say. But it wasn't adverts that Facebook manipulated it was our newsfeed. It was the collection of messages that we receive from our social network, something we value, something we trust and something which we don't expect to be manipulated in this manner.

We certainly don't expect our newsfeed to be manipulated in a manner designed to effect our mood as part of psychological experiment that we weren't even asked whether we minded being part of and then told we had given 'informed consent' because of an obscure line in the T&Cs  that no reasonable person would interpret as meaning such consent and which didn't even exist when the experiment took place.

Scenario planning and the future ...

Scenario planning in business is not simple, it's complex ... very complex and it takes skill. However, there's a lot of things that you can plan for and rehearse along with many patterns that you can use to determine likely points of change. There's a lot you can do to prepare yourself, a host of tactical plays you can exploit to your advantage but that's all about manipulation of a scenario. The first thing to do is understand the likely scenarios.

So, what does that take? Well, I've summarised the basics in the following figure.

1) Understand your landscape. If you don't have a map of the landscape then you're going to be shooting in the dark. Not a good position.

2) Understand your competitors and potential competitors. This means you have to analyse your map and compare not only direct competitors but possible new entrants caused by evolution of underlying components. No, you can't predict opponents actions or whoever new entrant is but you can certainly do a lot to limit the surprises.

3) Understand common economic patterns and cycles. The economy isn't quite as random as people make out. Some effects (such as competition between all actors driving the evolution of an act) are predictable in terms of what will happen but not necessarily when. The when is often unpredictable because it depends upon individual actors actions but you can still prepare for the what. There's a host of repeatable economic patterns you can use to determine likely points of change.

4) Understand constraints. Critical in gameplay is knowing your constraints (including inertia, resources etc) and those of your competitors. You can often exploit this to your advantage and many tactical plays use the constraints of others.

5) Understand weak signals. There's many techniques and methods for weak signal detection e.g. for the evolution of an act from product to utility then you look for the existence of concept, suitability, technology and attitude. Weak signals don't tell you when something will happen but instead you can use them to determine whether a change is ready to occur.

6) Understand the limits of prediction. Various aspects of change can be predicted in terms of what is going to happen but not when or vice versa. Other aspects are unknowable but there are many ways to cheat the system e.g. use of ecosystems as future sensing engines.

If you have the above then you can make a pretty decent stab at scenario planning and look at how to manipulate the environment to your favour. If you know the landscape, know your enemy and know yourself then you need not fear the result of a hundred battles. Better situational awareness and exploitation of force multipliers is a route to sustaining competitive advantage.

99.1% to 99.5% reduction in costs, outwitting opponents that have vastly more resourcing and funding, stealing the future from others - all of this is possible. Key to this is learning about the environment through the use of maps or equivalent frameworks for organisational learning. Without this ... well, I suppose you could just 'copy everyone else'.

Repeat after everyone else, our strategy is ... agile, digital first, cloud, social media, big data, insight from data, collaborative, ecosystem, efficiency, innovation, open, disruptive, internet of things ... with a few other words scattered in between.  If this is the case then you have no strategy because you have no scenario planning because you have no maps. Pray you don't come up against someone who does.

On doing what you love ...

Over the last two years, on six separate occasions I've been offered eye watering sums of money to shift to Silicon Valley. The most recent, in the last few months, occurred in a particular discomfort of travelling and moving house. Now, these came out of the blue and I'm not going to say I wasn't tempted. The sums involved are instant life changing amounts - we're not talking 10% or 20% differences but order of magnitude.

However, I have a very privileged position which I enjoy. I have access to a vast number of different companies in different fields in which I can conduct research on strategic gameplay & mapping. This alone is worth its weight in gold. 

But there is something else. I am in an extremely privileged position to observe and interact with what I consider to be the single most exciting technology company in the world - the UK Government.  The work that is being done, the calibre of people, the concepts and ideas being pursued, the strategic gameplay, the management approaches are second to none and there has been a dramatic change in recent years. It's not all perfect by a long shot but it's heading in the right direction.

Why do I mention this? Well, everyone tells me that Silicon Valley is the heart of disruptive change and where the future is built. There seems to be an assumption that this is a permanent state. I'm not convinced. 

The work being done in UK Government IT is transformative to an extreme and it is slowly spreading across the world. I know of many others who look at UK Government IT and say 'we need to be more like them'. It'll be interesting to see how these changes impact the surrounding ecosystem and economy over time. 

For me, as an observer, the signs are very positive.

Thoughts on conferences

I've chaired / been an advisory member / been involved in organising many conferences in my time. I was asked recently for some thoughts about conferences. Beyond the usual of having a theme, a story for the day, good speakers, a comfortable environment, an excellent AV crew etc ... there is a particular structure I enjoy.

Morning

a) Scene setting / opening keynotes

Introduction to themes. This should be very much broadcast mode, high level, fast paced and using humour. Often I introduce a group of lightning talks and if necessary a vendor 'red card' system to prevent pitching. Key for me is to leave the audience with a good grounding in the themes.

b) Hegelian dialectics. 

Usually two to three sessions using a format of thesis, antithesis and synthesis on 'hot topics' e.g. "Will robots replace humans?", "Is bitcoin the future of money?", 'Is disruptive Innovation just the latest hype?" etc.  One speaker proposes for 10 minutes (thesis), one speaker opposes for 10 minutes (antithesis) and then both speakers debate and discuss with audience for usually 20 minutes (synthesis). I find this good for sorting out the chaff from the wheat and getting people to realise there isn't usually a simple "answer" to complex problems i.e. thinking caps required. With good to speakers who know their topic well it gives a bit of challenge and often peps up the audience in the bargain.

Afternoon 

c) Practice

After grounding in the subject and discussion on a few controversial hot topics then there's nothing like a bit of practice and real world examples of use. Customer stories, leading users in the field etc. 

d) Res, non verba.

Ok, I never call it this but where possible I like to introduce an element of session choice i.e. two simultaneous tracks.  This is the conference equivalent of A/B testing and tells you a lot about what topics matter but also the composition of your audience. Both want to cover part of the theme and ideally you want them to be both top notch i.e. don't give the audience an easy choice.

e) Audience Panel ('meet the experts')

As we move towards the end of the day, the key thing to focus on is reflection. I find that letting the audience speak to each other about the themes of the day is helpful in building up a real community and interaction within the group. The format I normally use is randomly selected audience members to make the panel (4-5 ppl) and allow an ebb of questions back and forth between panel and audience. This is probably the most difficult part to MC effectively. 

f) Closing thought

The toughest bit, done at the end of the panel, a thank you which summarises the themes discussed and leaves the audience with one key point to think about.

Anyway, that's my preference. Things which I don't tend to enjoy and in my view should be used extremely sparingly (if at all) are  'expert panels', 'fireside chats' & 'product pitches'. 

Full asking price and the end of Caveat Emptor?

Well, I've finally found somewhere to rent until the contract on my new house is exchanged. All looks good. I've also just been informed by the landlord of my old place (2 Moorstock Farm Cottages, Sellindge) that he has managed to sell it for the full asking price. 

I'm pleased for him but I'm a bit surprised by this.

I met the estimator for the property who came up with the initial price range. The house is a good size, the neighbours are lovely and the area is fine. After they came up with a price range, I asked 'Had they considered the defects?' ... they did not seem to think it mattered.

There are a couple of defects to the property which are known to the landlord and which are known to the estate agent I met.  Firstly, about a third of the garden is owned by the neighbours (the land boundaries need sorting). Secondly, the power for next door runs directly under the garden (this has potential building reg issues). Thirdly, the house has no independent water supply from next door and no contract with any water supplier because it has no meter (the road actually needs to be dug up and an independent supply carved off with a meter from the water mains). Fourthly ... oh, now there's a long list of minor inconveniences.

The landlord (the owner) is a nice enough chap but seems to think all these issues are someone else's problem. The water supply is next door's problem despite the property not having a contract with a water supplier. The power issue is next door's problem - it's their power after all. The land issue is ... well, next door's problem.

In my opinion, these are all his problems. All of these in my view should materially affect the value of the property. However, when I met the agent showing prospective buyers around the property and despite him knowing all the defects, I never heard him mention any of these. Maybe they provide this all in a fact sheet but at least one potential buyer I spoke to was completely oblivious to all.

So why does this surprise me? Well, house buying in the UK is no longer under the maxim of caveat emptor or buyer beware. Since 2012 it has been covered by the Consumer Protection from Unfair Trading Regulations (CPRs) of 2008. This means it is no longer the responsibility of the purchaser to "discover" any problems about a property or location but also a duty of the agents to disclose anything which might materially affect the price a buyer might offer.  CPRs expose agents to prosecution if they do not admit knowing something that might influence a price or choice of the buyer.

Having a garden which isn't actually all yours, or a major household power supply running under your garden or no independent water supply are all defects which I would imagine would materially effect the price of a house. Maybe they don't matter ... I'm no property expert. Or possibly, people are still operating under the idea that housing is covered by caveat emptor?

The amount of money involved is not trivial. Back in March / April, I was offered the property for £415,000 but refused unless all the defects were resolved. They weren't and hence I found somewhere else. Maybe the market in housing is so hot that people don't care and are just desperate to get on the ladder? It'll be interesting to find out what happens here.

How to beat Amazon at its own game ...

I was asked a question today of 'What can IBM do against AWS?'

About ten years ago I tried to convince IBM to build a utility service for infrastructure (based upon Fotango's Borg system) and about six years ago I spent several hours of my time explaining to an IBM C-level on the economics of change and how IBM needed to build an AWS clone and create a price war in order to increase demand beyond AWS ability to supply (building data centres is a constraint) and naturally fragment the market in IBM's favour.  None of this happened and so my first response to the question was unfavourably ... "die from inertia?" 

However, given it's a friend who asked the question, I thought I'd spend 30 minutes mapping out the environment, looking for weaknesses and write a quick plan.  So, I've written a map but I don't have time to convert this to an image.

Before I tell you the play, you need to make sure you're familiar with the ILC ecosystem model and why information flows are critical to it. Understanding mapping, evolution, situational awareness, the importance of not just harvesting but nurturing an ecosystem is also essential. For some, this post will seem like gobbledygook.

It's enough to say that key to the play is ...

1) Understand your opponent. The value of AWS is not the utility service but the ecosystem of companies building on top of it. This ecosystem acts as a future sensing engine with everyone innovating and AMZN getting early warning through consumption information. If you're going to take them on then you need to neutralise this or at least get yourself into the flow of information.

2) Understand the landscape. The shift from product to utility is inevitable because of competition. Many 'legacy' companies are struggling to find a route through this. Many new companies are building directly on top of this. Beyond simply getting yourself into the flow of information, you also can exploit others to create a counter force to AWS.

So, what should you do? 

1) Provide some form of proxy service to AWS and call it a brokerage. Enable companies to build on your service (in exactly the same way as they do with AWS) but IBM takes care of the pricing / billing of AWS i.e. IBM plays the game of using reserved instances, spot markets etc on behalf of clients.  Use this to provide all sorts of tailored financial terms suitable to enterprise clients. Yes, you're going to embrace and help grow AWS but overtime you want to position this such that all AWS sees is the IBM 'account' but IBM sees which of its clients are rapidly growing. For a fast track route into this space, go acquire Cloud Options as a starting point. 

2) Once you have a large ecosystem effectively under your observation that is building on AWS through your proxy, start mining that ecosystem to identify new services that are rapidly growing. Acquire them - don't copy. Promote an image that using the IBM proxy service is also a fast route to acquisition along with being vastly cheaper (due to the financial engineering done).

3) At the same time as step 1) you should build a platform play on Cloud Foundry. IBM is already doing this - BlueMix - and that's good but it should build the platform not just on Softlayer but AWS, Azure and GCE. Encourage others to do the same because key to the play is a market of public providers all based around Cloud Foundry which are built on a mix of IaaS including AWS. You want Cloud Foundry to be big on AWS to prevent AWS making moves against it. Don't try and differentiate on features, focus on operational excellence. Help others set up as providers in competition

4) Capture the platform assurance market. With many Cloud Foundry providers then companies are going to want to know who is interoperable and who is reliable. First, you want to make sure that a competitive market forms (unlike the mess in OpenStack) so support Pivotal as a Benevolent Dictator. However, build an assurance platform which monitors all the different Cloud Foundry providers and determines interoperability and compatibility to the core. Once you have market of providers which you have assurance information on then offer a brokerage service for platform. Act as the proxy for the market. This again is a capture play for information from the ecosystem.

5) Now you have a proxy role for both PaaS (CF) and IaaS (AWS) and are capturing information from both ecosystems for acquisition plays.  Being a big supporter of AWS (both through proxy on AWS and use of CF on AWS) then Amazon will find it difficult to react against you. At this point, you can start to look to play substitution games at the PaaS layer i.e. providing price benefits for those using Softlayer.

6) Oh, there's a lot more but my 30 minutes are up and I can't type that quickly.

Ok, I'm not saying the above is the only way forward (and certainly most of the above is late in the day) but what I am saying is that if in 30 minutes I can start to identify routes of attack against AWS then with a bit of effort and gameplay there's a lot that IBM can do to change its fortunes in the market. That IBM hasn't been doing this has been a sorry tale though the first signals they are getting better are appearing. Oh, and the above also goes for HP, Cisco, Oracle and all the rest of the old gang.

My advice to my friend on 'what can IBM do against AWS' is simple ... chose one from "learn to play the game fast" or "build a time machine" or "die from inertia". 

Robots will eat all the jobs ...

In 1896, N.Hawkins wrote the New Catechism of Electricity which included in it a warning on the impacts of electricity on jobs. In 1962, D. Michael wrote Cybernation : the Silent Conquest which included in it a warning on the impacts of computing on jobs.

Throughout history, at every age from industrial, mechanical, electricity, computing, internet and today then people have written warnings that this time, whatever it is will eat all the jobs. Everytime they've been wrong.

To understand why, I'm going to quickly summarise a cycle I've talked about for the last half dozen years - peace, war and wonder. A more detailed version can be found here. 

All our industries consist of value chains comprising of activities, practice, data and knowledge which are evolving due to supply and demand competition. For activities we describe this path of evolution as genesis, custom built, products (+rental services) and commodity (+utility services) and you can visualise the environment through the use of maps (see figure 1)

Figure 1 - Example Map from High Speed Rail overlaying different methods (due to changing characteristics)

Maps are also good for organisational learning and in detecting common patterns in the wider economy. There are many of these. One very important factor to understand is that components (whether activities, practices or data) change in properties as they evolve (see figure 2) due to competition i.e as an activity shifts from left to right on the above graph, its properties change.

Figure 2 - changing properties of evolving components.

This change of properties in combination with competition and inertia creates a common re-occuring pattern known as the peace, war and wonder cycle. The cycle appears both at a local industry level and macro-economic scale (the largest being known as K-waves or 'ages' and is directly attributable to how widespread the underlying components that are changing are in value chains across industries). The cycle itself mimics the adaptive renewal cycle that is found in other systems (see figure 3) and basically involves commoditisation of the past and the rapid creation of the new.

Hence (from figure 3) as an activity A evolves (due to supply and demand competition) through various states A[1] to [A6], it passes through a relatively peaceful state of competition between big vendors (A[1] to A[5]) that all build inertia to change due to past success. It's normally a company that is not encumbered by past models (think bookseller for infrastructure) that shifts the act from a product to commodity (and / or utility services) i.e. A[5] to A[6]. This change is rapid (known as a punctuated equilibrium) and is associated with changing practices (for computing think DevOps) and the rapid creation of higher order system e.g. B[1]. The rapid explosion of higher order systems that are inherently uncertain in value (think age of wonder e.g. utility provision of electricity enabling a mass of new activities), creates new forms of data (think Big Data associated with the shift of computing from product to utility) along with disruption of past models and new organisational forms. You can think of the stages as being equivalent to the adaptive renewal cycle with Peace being equivalent to exploitation and conservation, War being equivalent to release to re-organisation and Wonder equivalent to re-organisation to exploitation.

Figure 3 - Peace, War and Wonder cycle

The interesting part about this cycle, is that during the 'war' phase we get co-evolution of practice, new forms of organisation appearing and disruption of the past combined with a 'wonder' phase where new forms of activities and capabilities appear. This pattern isn't new, I used it when writing Canonical's play in the cloud in 2008. I also used the pattern to determine the new phenotype of organisation that was emerging in 2011 (see figure 4).

Figure 4 - Phenotype of new organisations forming in last cycle (caused by commoditisation of parts of IT)

The economic system turns out to be far more predictable than most people believe but then given most people can't even see the chess board they're playing on then it doesn't surprise me that most think it's random. 

Now, the key part of the peace, war and wonder cycle is that we can see the past that is being industrialised but we cannot see the future (the 'wonder') because it is by very nature uncertain, uncharted. 

Throughout history, the new activities and new capabilities have created new jobs e.g. electricity might have decimated the gas lamp lighters role in life but it created radio operators, TV presenters, computer operators and a host of other roles. However, those new activities are uncertain and hence we don't know what the impact will be.

We know that 'Robots will eat many old jobs' but can we say 'Robots will create many new jobs' ... well, that's the tricky bit. As Jim Stodgill correctly points out, it is a statement of belief either way. This is because of the uncharted (hence uncertain) nature of the new activities. We don't know, we cannot know and there is no crystal ball. The future is actually an uncertainty barrier for information.

Fortunately in the past it always has created many new jobs despite people writing that the change will destroy work. Unfortunately, given a long enough timeline then even Chicken Little will be right.

This is not the data you're looking for ...

Many many years ago I was asked by a hedge fund whether they could use LinkedIn data and other social networks to see which companies were talking to each other with an eye for future acquisitions.

I responded "this is not the data you're looking for".

CEOs are unlikely to like other CEOs on social networks prior to an acquisition. However, CEOs like private jets and ownership of private jets and flight plans are public information. Hence you could simply map the movements of private jets around the US, look for disturbances in the patterns i.e. regular repeated events where CEOs (and other top execs) from different companies landed in roughly the same location, at roughly the same time, ideally where neither companies had HQs (people like to be sneaky, clandestine meetings etc).  The technique proved remarkably useful but these days many companies tend to take efforts to obscure ownership etc.

There are many powerful techniques for examining weak signals to change both at macro-economic level and individual actors (i.e. companies).  When I say many, I mean hundreds - a good competitive intelligence unit doesn't need to resort to the nefarious activities of Duplicity to really understand a competitor's play. This is also my favourite part of mapping - once you get good then mapping your competitors is essential to the most powerful forms of gameplay including using competitor's inertia against them, creating force multipliers through ecosystems etc.

As Sun Tzu once said, 'if you know your enemies and know yourself, you will not be imperiled in a hundred battles'. Understanding the landscape, your own capabilities and the opponent is essential to competition and given most companies barely understand the landscape then you don't need a great deal to take on a market. Correctly used, force multipliers of a x1000 are not uncommon i.e. if you're up against a company with a few billion then you only need a few million and the right play to take them down. 

The point however I want to reiterate, is that in an era of 'big data' the most important things are the models of understanding and not the volume of data itself. Huge amounts of the wrong sorts of data doesn't normally get you anywhere. Of course, if you don't know what you're looking for (i.e. you have no model of understanding) then collect everything. Maybe you'll get lucky.

I'm not a big fan of luck however. I find when you have to start to rely on it (e.g. if you are competing and find yourself needing a lucky break, that next new innovation to succeed) then that's normally when it leaves you. It's normally a sign that your gameplay has sucked.

There are a lot of big companies in that boat at the moment who have failed to understand the landscape, the predictable forces of change and how to manipulate this. A lot of people talk about 'disruptive innovation' when they actually mean "we've been outplayed but just don't want to say this or know how to fix this". 

We should see some huge name casualties in the next 6 years. We have many modern giants that mimic the position of Blockbuster, Atari, DEC and Kodak. But who are they? Well, those companies appear to occupy the bottom left hand corner of this graph. Alas, it'll take a few more years until I have the data to know whether this particular 'weak' signal and model is truly significant or not.

That's the problem with models of understanding, you have to build them and until you have developed models (with correlation, causation and experimental verification) that repeatedly work then you never really know what you're looking for.

It's like the CEO and jets example above. Until we tested it, we didn't really know whether it was relevant or not. Of course, when you find something that works repeatedly - that's where real value lies.

In all this data, those repeatable models are the ones you are looking for.

On capabilities

Jay Fry is doing a grand job of tweeting some choice quotes from the Gartner CIO survey. I've already commented on the 'top ten challenges for I&O leaders' but this is another scorcher.

“The IT org has the right skills and capabilities in place to meet upcoming challenges.” 42% in Gartner CIO survey DISagreed. #gartneriom
— Jay Fry (@jayfry3) June 9, 2014

Ok, here's my problem with the quote. Unless you've mapped out the environment and created a meaningful strategy based upon this and gameplay then how do these companies know what capabilities they'll need? 

I more than understand that most strategy is simply 'backward causality' (i.e. copying others) and hence if you produce a list based on this - digital first, agile, big data, cloud, social media etc - then you can come up with a fairly uniform list of missing skills. But this doesn't mean you actually need those capabilities. It's more than likely that any strategy based upon 'what 67% of other generals do' is the wrong strategy.

In fact, I have quite a bit of evidence to point to this and how lack of situational awareness (a key element to strategic gameplay) is correlated with poor market performance (see figure 1)

Figure 1 - Market Cap impact of Strategic Gameplay

So, how exactly do these CIOs know what capability they need? Certainly the list of top ten challenges doesn't give me confidence or maybe I just inhabit a completely different world.

My list vs Gartner

I was fairly flabbergasted by a recent tweet by Jay Fry providing the top ten challenges for I&O leaders at a Gartner Symposium. There's nothing wrong with Jay's tweet but instead the list. I don't recognise this at all.

In my world, the list is completely different. This is my list broken into two parts :-

Part 1 : Corrective Action

#1 - Stop self harm. 

Introduce measures to bring IT expenditure under control including transparency & use of data, identifying common transactions, cost per transaction, challenge boards for high cost projects, improving internal skills and capability etc. Start building up your internal technical capability to challenge projects.

#2 - Focus on user needs.

Ensure that all projects have a clear focus on user needs, that these are defined and where possible measurable and measured. Start building up your internal technical capability to meet those needs.

#3 - Improve situational awareness

Once you get over the initial hurdle of focusing on user needs then start mapping out complex projects from user needs. Identify how you should treat components encouraging effective project management, risk mitigation, communication and learning from others.

#4 - Learn from next generation

Start getting familiar with next generation practices from use of commodity components to ecosystems to cell based approaches to use of open as a competitive weapon.

#5 - Create meaningful strategy

With improved situational awareness and lessons from next generation, develop meaningful strategy with the business focused on the where and why rather than the usual 'copying everyone else' and 'backward causality' which is rife (hint, you didn't think that everyone's strategy being digital first, agile, cloud, social media, mobile, BYOD, big data, insight from data, ecosystems, efficiency, innovation yada yada was an accident?).

Part 2 : Positive Action

#6 - Execution

Once you created meaningful strategy based upon all the above, now you're ready to do stuff. What this actually means will depend upon your business, your competitors, the environment you operate in, basic economic forces (competition, inertia, buyer / supplier strengths), weak signals etc. So, there's no simple list and hence I stop here.

My list is vastly different from Gartner's which seems to be a tyranny of action (i.e. build a private cloud) with no reference to either user needs, stopping self harm, improving awareness etc. At best it talks about linking business and IT strategies (btw having these as separate is a huge mistake) at #7.

Apparently the list is from a Gartner survey. Obviously we walk in different worlds.