Something for the future

A list of companies in two groups. I'm putting this here in order to return to the list in 2020 and explain why I had the companies listed as such.

It's a prediction test (useful for me, hence I'm putting it somewhere public) but probably not useful for anyone else.

Group 1

1. Amazon
2. Google
3. Samsung
4. BP
5. Baidu
6. China Telecom
7. EMC
8. Lloyds Bank
9. ARM
10. NetFlix
11. Ebay
12. Yahoo
13. Intel
14. Facebook
15. BAE Systems
16. Lenovo
17. Salesforce
18. Time Warner
19. Huawei
20. Canonical
21. Citrix
22. Fastly
23. Bromium
24. Opscode
25. Juniper Networks

Group 2

1. DuPont
2. GSK
3. WalMart
4. Microsoft
5. Berkshire Hathaway
6. Goldman Sachs
7. Barclays Bank
8. RedHat
9. Walt Disney
10. IBM
11. Cable and Wireless
12. Canon
13. SAP
14. HP
15. CouchBase
16. Oracle
17. Cisco
18. Puppet Labs
19. Apple
20. Rackspace
21. Twitter
22. Dell
23. Nokia
24. Zynga
25. PistonCloud

On Open Source, Standards, Clouds, Strategy and Open Stack

On Standards and Open Source

The issue of standards and in particular open standards is a hotly debated topic. The recent UK Government consultation on open standards was embroiled in much of the politics on the subject with even a media expose of the chair of a consultation meeting as a member of a paid lobbyist group. The rumours and accusations of ballot stuffing of ISO meetings with regards to Microsoft’s OOXML adoption as an open standard are also fairly widespread. The subject matter is also littered with confusing terms from FRAND (fair, reasonable and non discriminatory licensing) being promoted as an “open” standard despite it being IP encumbered by definition.

In general, the principle of standards is about interoperability. In practice, it’s appears more of a battleground for control of a developing market. Standards themselves can also create new barriers to entry into a market due to any onerous cost of implementation. There are also 17 different definitions of what an “open standard” is varying from international bodies to governments. Of these the OSI definition is probably the most respected. Here, open standards are defined as those which have no intentional secrets, not restricted by patents or other technology, no dependency on execution of a license agreement, freely and publicly available and royalty free.

When I talk about evolution of systems (whether activities, practices and data) these are all about meeting specific needs such as providing a web site or carbon dioxide emissions of a country. However standards refer to generic needs that apply across many stages of evolution – the need for things to work together and the need to be able to switch between solutions. These generic needs change in implementation as the underlying system evolves.

For example, in the world of software products then standards can provide interoperability between products and hence the potential to replace one product with another. The standard is normally articulated as a principle of working (i.e. how data is transferred and interpreted) in a document and the expression of this standard (i.e. the code itself) is left up to the product vendor.

This process is often imperfect as one vendor’s interpretation and expression of a principle might not match another’s. However since the switching between solutions is not usually time critical for the end user then some level of imperfection is acceptable i.e. you own the product and if you decide to switch you can migrate at your own pace. Though it should be noted that the process of migration is often a fraught one due to the imperfections.

In the world of utility services, the switching time can be critical and immediate such as the termination of a service. Here imperfections in migration are extremely undesirable and effective switching requires semantic interoperability between the services. In other words, any code and data is understood in the same way between the providers. In practice this can only be achieved if both providers are running the same system or strictly conforming to a reference model (an expression of code) rather than their interpretation of a documented principle.

Switching is an essential part of any competitive market and with utility services then the principle in a standards document is not enough and a reference model (i.e. running code, the expression) is required. If that market is to be unconstrained (i.e. free) then that expression needs to be open itself.

Hence open standards in the product world can simply be open documented principles but in the utility world they require open source reference models (i.e. running code). This simple fact is counter to how standards have been used in the past battles between products and hence the confusion, debates and arguments over the terms are unsurprising.

On Clouds, Open Source and Standards

So, we come to the cloud, which is simply the evolution of a range of IT activities from a product and product rental model to one of commodity and utility services. If we are going to see competitive free markets in this space, then open source reference models are essential. But the value to the end user is the market, not whether this system is more open or not.

An example of this is in the IaaS space. If you ask some of the big users of AWS what they want, they often reply with multiple AWS clones and rarely with another IaaS with another API. To create a marketplace of AWS clones you're going to need to start with an open source system that provides the EC2/S3/EBS APIs with multiple implementations (i.e. providers of such).

I first raised this issue at Web 2.0 in 2006 and by 2007 the whole debate of "open APIs" was kicking off and it has raged ever since. The argument today goes that EC2/S3/EBS are not "open". However APIs are simply principles, they cannot be owned only your expression can be. This has been re-affirmed in several court cases over the last few years.

This debate has already started to switch to "But the process of creating EC2/S3/EBS isn't open" ... well neither is the process for the development of Android and the end user doesn't care. As Benjamin Black once said "Solve user problems or become irrelevant".

The user problem is a competitive free market of multiple utility providers offering the defacto standard (which in case you haven't realised is  EC2/S3/EBS) and not a plethora of multiple APIs basically doing the same thing, with super APIs attempting to manage this mess under various cries of being more "Open".

Matt Asay does a good job hitting the point home with his post of "Whose cloud is the open sourciest ... who cares?". Let us be crystal clear here, Matt isn't against open source in the cloud, he (like most of us) understands its essential importance. However, Matt also understands that what needs to be focused on is the user need for competitive markets.

Open source is absolutely essential for creating a competitive free market but the focus should be on solving users need i.e. creating the market.

The focus should not be on differentiation of stuff that doesn't matter whether because of some belief that you can out innovate the biggest ecosystem in a utility space or you view it as an on ramp to your own public services. The focus shouldn't be on protecting existing industries (something which concerns me with the EU communication on Cloud). To reiterate for the umpteenth time over umpteen years, the focus should be adapting to this new world and using open source to create a utility market which meets users needs.

Mark Shuttleworth nailed this point many years ago "Innovation and Open Stack : Lessons from HTTP"


On Strategic Play and Open Stack

So when it comes to playing this game, especially because of the powerful effects that ecosystems can create, then the smart strategic play is to build an open source equivalent to EC2/S3/EBS (i.e. all the core AWS features) and create a market place of multiple providers. The goal is to co-opt, sure you can differentiate later when your ecosystem is bigger but at this moment in time co-opt and build a bigger ecosystem through a competitive market.

But how do we know we can match the APIs. Well, the beauty of competitive markets is that they allow for assurance services and exchanges i.e. there is value in the test scripts which ensure that an API is faithful. So, build the test scripts, use those to build your open source IaaS and allow people to create a Moody / Rating agency style business as your market forms.

Whilst CloudStack and Eucalyptus have made steps in this direction, Open Stack (and more likely the RackSpace people involved) seem more reluctant. "Anything but Amazon" seems to be the watch cry and the adoption of EC2/S3/EBS in Open Stack appears to have been something the community forced upon it.

Differentiation on APIs etc is a pointless hangover to the product world. Adoption of APIs and differentiation on Price vs Quality of Service is the way forward. Being an AWS clone and creating a market around such isn't "giving in" but instead it's "competing on what users rather than vendors want".

I would like to see Open Stack succeed, there are many talented people (and friends involved). But I have my doubts because I feel it has wasted an opportunity, it has been very poorly led in my view despite its success in marketing.

The battle in IaaS is going to heat up between AWS and GCE in the next year or so. Open Stack during that time needs to create a large competing market and an equivalent ecosystem which will only be hampered if it doesn't co-opt AWS.  Hence it has probably twelve months or so to get the technology rock solid, multiple large scale implementations (by large I mean a minimum $500 million capital spend for each installation), overcome the potential collective prisoner dilemma issue (of everyone differentiating rather than contributing to core) and form the competitive market.

If it fails, then my view is the good ship Open Stack will become the Mary Celeste 2.0 and that will be a terrible waste of effort.

In my view, Open Stack needs a strong benevolent dictator (like a Mark Shuttleworth for Ubuntu). An individual, who is willing to do what's in the interest of the end user and community and ride roughshod over others where necessary. Of course, the essential part is benevolence and its easy to fall foul here. With Open Stack, my advice is they need to focus on engineering quality and build the best open source AWS equivalent there is. Anything else (especially when the words differentiate and innovate are used to describe it) should be unceremoniously dumped of a high cliff for the time being. Along with this should go the every API, every Hypervisor and any "be all to everyone" concepts - focus, focus and more focus on one thing e.g. the best AWS clone. At a push, you could make an argument for AWS and GCE as your native APIs ... but I'd advise against it.

I'm glad we have an Open Stack Foundation, as the chances of that benevolent dictator emerging have grown. I'm hoping someone like Lew Tucker will step up to the plate.

However, where would I place my bet? Currently in the three horse race between Open Stack, Cloud Stack and Eucalyptus then the latter two have been playing a better game in my view though this race is far from over. If forced to choose one, well that's splitting hairs between the likes of Eucalyptus and Cloud Stack but I'd probably bet on Cloud Stack. They have a good focus, they're part of the ASF, they have a well funded backer and they have numerous production deployments at scale. It is however, too early to tell ... the next 12 months are critical.

At last ... a great definition for cloud computing

I'm not a fan of the term 'cloud computing' nor the umpteen number of definitions. I don't like NIST's mechanistic definition of 'cloud computing' which misses the nuances and so I prefer to stick with 'computer utilities' (as described by Parkhill in his 1966 book).

A definition of 'cloud computing' has to consider the economic changes due to the evolution of computing infrastructure (a technology activity) to more of a utility but at the same time it has to be mindful of niches and the different organisational and security requirements (resulting in various forms of hybrid environments) during the transition. Many of these won't last (it is a transition after all) but they need to be considered.

Somehow, in all of this time, I've missed this wonderfully simple definition of 'cloud computing' by Ramnath K. Chellappa in 1997 at INFORMS

‘cloud-computing’ is a dynamic computing paradigm where the boundaries of computing are determined by rationale provided by technological, economic, organizational and security requirements

All, I can say is this definition is almost perfect in simplicity and at the same time incredibly sophisticated in nuance (and vagueness). It also happens to be the first known definition of 'cloud computing' (being in 1997) and as far as I'm concerned it has been downhill since then.

Some trivia questions on cloud computing

 ... just for amusement.

Questions

1. Which year was the future of computing being provided by public, private and mixed utilities like "electricity", first published in a book?

2. Which came first, a utility based IaaS or a utility based PaaS?

3. Was Amazon EC2 built on selling Amazon's spare capacity?

4. When did Amazon start working on the concept of EC2?

5. In which year was the idea of future utility markets, federated grids and the role of open source in cloud computing first publicly presented?

Answers

1. 1966, Douglas Parkhill, The Challenge of the Computer Utility.

(see http://en.wikipedia.org/wiki/Douglas_Parkhill)

2. Utility based IaaS. The first utility based PaaS (Zimki) was publicly launched at D.Construct, 14 days after the launch of the most commonly well known utility based IaaS of EC2 on the 25th August 2006.

(See http://radar.oreilly.com/2006/09/zimki-hosted-javascript-enviro.html)

3. No. The myth of Amazon EC2 being built on spare capacity of Amazon is one of the unquenchable and totally untrue rumours.

(see http://itknowledgeexchange.techtarget.com/cloud-computing/amazons-early-efforts-at-cloud-computing-partly-accidental/)

4. 2003, though the implementation of the idea started in 2004. A good bit of background on this can be found on Benjamin Black's post.

(see http://blog.b3k.us/2009/01/25/ec2-origins.html) 

5. Whoot, I'd like to claim that was me in 2006, an earlier version of the talk I repeated at OSCON in 2007 but then that would be completely untrue. (see http://blip.tv/swardley/commoditisation-of-it-419213).

The reality is these ideas was fairly common by 2007 and I don't know when it actually started. Some of the federated ideas certainly dates back to 1960s and many of the concepts above were described in this 2003 paper on Xenoservers by Ian Pratt et al.

(see http://www.cl.cam.ac.uk/techreports/UCAM-CL-TR-552.pdf)

There are many earlier publication, normally around the notion of markets of common service providers (CSPs). You can also bet your bottom dollar that many academics were looking into this issue between 1995-2005.

So I'm afraid this was a trick question and the answer is ... no idea but earlier than people normally think.

Comments

The point I want to get across is that the concepts of cloud computing are much older than many realise, that there still are many myths (such as the Amazon spare capacity story) and we're in our 7th year of commercial applications. Normally, these changes take 8-12 yrs to become widespread and mainstream, hence expect this over the next year or so. If you're just getting upto speed with Cloud Computing then to be honest, you're perilously close to being a laggard.

Don't try to out innovate Amazon

Amazon is famous for it's two factor market and platforms play. The modus operandi of Amazon in general is :-

1. Find an activity which should be so well understood and ubiquitous that is suitable for provision as a commodity (ideally utility service). Example's would be online marketplace, infrastructure, certain data sets etc.

2. Provide a platform to exploit this i.e. expose through public APIs.

3. Enable an ecosystem to build on the platform. This could be either an ecosystem of providers and consumers (two factor market) or consumers of an API (e.g. developers building higher order systems) or ideally both.

4. Mine the meta data of that ecosystem for new information on trends (i.e. things evolving in the market by examining consumption of the API). You don't have to look at the data of others just the consumption rates to do this. Focus the efforts of the company on this activity i.e. use a press release process. In other words, get everyone to write press releases before building anything. Since you can't write a press release for something not invented yet, this has the effect of concentrating people on commoditising pre-existing acts (which you can write a press release for).

5. Commoditise those new trends either through copying or acquisition in order to provide new component services to both feed the ecosystem but encourage it to innovate more.

Rinse and repeat this cycle.

This model is far from new. The basics are - get other's to Innovate, Leverage the ecosystem to spot trends and Commoditise to component services - ILC for short. It enables the company to appear to be highly innovate (everyone else is doing the "innovation" of novel and new), highly customer focused (mining meta data and using a press release process to give people what they want) and highly efficient (economies of scale) all at the same time and the ability to do all three increases as the ecosystem grows.

So, when you come up against Amazon in your industry, here are two simple Do's and Dont's.

Don't try to out innovate Amazon : You're not actually fighting Amazon over innovation but instead you're fighting the entire ecosystem that has built upon its platform and is doing much of the innovation (in terms of new activities). It's worth remembering that some of Amazon's ecosystems can have hundreds of thousands of developers and companies. If you're going to try this alone then you'll need an enormous R&D group to out compete on these terms. If you've not got this then reality is you'll just get spanked. This is despite, if my sources are correct, of Amazon not having a traditional R&D group. It wouldn't surprise me if every time Amazon hears a company say "We're going to out innovate Amazon" then they cross them of their list of competitors to watch and mark them "RIP". The only time its really worth fighting on these terms is when you have an equivalent size of ecosystem (or you're pretty confident of quickly getting one) combined with the ability to exploit it. In which case you're not really trying to out innovate Amazon, you're focused on getting your ecosystem to out innovate their ecosystem.

Do try to co-opt and out commoditise Amazon : Critical to this game is to try and build a bigger ecosystem and one way is to exploit the main weakness of Amazon being a single provider. So, try and build a competing market of equivalent providers enabling customers to easily switch between its members. Co-opt Amazon's ecosystem as much a possible. Provide the systems as open source and don't fall into the trap of all the members trying to differentiate (the collective prisoner dilemma issue). Once your ecosystem is big enough then you can use the ecosystem to out innovate Amazon and its ecosystem.

-- 18th April 2016

Reminded of this today. People are still getting spanked by this. Sad really.

Added rinse and repeat to make it clear, this is a cycle.

Spoiler Alert ... 3D printing

Anyone who knows me, knows I've spent well over a decade keeping tabs on and occasionally being actively involved in 3D printing and printed electronics. My real interest, as it has always been, is the wider economic cycle of change of which 3D printing will be part of the next cycle.

To cut a long story short, 3D printing is about commoditisation of the means of manufacture in the same way that the Internet was about commoditisation of the means of mass communication and Cloud (a term I despise) is about commoditisation of a range of IT activities.

Anyway, I spent rather a dull afternoon in the company of a respected "future"-ologist who clearly has no idea what he is talking about. So, I thought I'd be a miserable old fart and tell everyone what 3D printing is actually going to do.

This is a SPOILER for the future.

1. Commoditisation of the manufacturing process will result in an explosion of new activities (higher order systems) as the means of manufacture become ubiquitous. This will create a time of wonder and new industries (see electricity led to radio, Hollywood, consumer electronics, computing etc) but what those new things are and what new industries form, well we don't know yet because they're uncertain (this is the one bit which you still have to wait and see). However, this won't stop endless pontification on the subject and book writing about the massive transformation underway and "Does Manufacturing Matter?"

2. The manufacturing industry will shift from a state of "peace" where relative competition exists and sustaining change tends to exceed disruptive change to a state of "war". It will become a fight for survival, where many past giants who have created inertia to change due to their past success fail to adapt and subsequently collapse. During this "war" disruptive change will exceed sustaining change and new entrants (not encumbered by past business models) will lead the charge. Among the new entrants will be our future giants.

3. Executives of former giants that are collapsing will start once again blaming culture (i.e. others) for their failure (lack of vision and action).

4. We will see a flood of capital from the past manufacturing industries to these new industries. Some economist somewhere will write a book on Schumpterian economics and 3D printing.

5. As the activity of manufacturing changes (from custom built factories to ubiquitous 3D printers) then practices in manufacturing will change. Practices often co-evolve with activities (e.g. architectural practices have shifted with computing infrastructure moving from product to utility). Someone will come up with a catchy name for this, like "DevOps" with cloud today.

6. The new practices will result in new forms of organisation as per electrical age (Fordism), Internet age (Web 2.0), Cloud (Next gen) and every other age. Some management guru will write a book on these new organisational structures probably calling them the "new Fords" or another equally lame term.

7. At the height of this change someone will write a book about how commoditisation of the manufacturing process (though they won't call it that as we will have created some daft term like "cloud" etc) will lead to mass unemployment etc. This book will probably overlook every other book which has been written about the same phenomenon (Hawkins and the electrical age) and how each time the author failed to anticipate the new activities (higher order systems) and related industries that will form e.g. electrical age led to radio, Hollywood etc etc. The author will of course fail to anticipate because the activities are uncertain (see point 1).

8. In a desperate attempt to save their bacon, past industries will promote the importance of physical DRM to prevent people stealing copyrighted ideas or making dangerous items. Security and the threat of people being able to manufacture things like guns will be used to explain why this change is dangerous for us all and must be stopped. Some lobbyist groups will persuade some Government somewhere that progress is bad.

9. Along with disruption of past giants many secondary industries will discover that their industries will be disrupted due to rapid reduction in barriers to entry. See Newspapers and the "Internet is good for us as it'll reduce our distribution and printing costs ... wait ... bloggers ... oh, no ... stop the Internet" moment. Someone will write a book on how 3D printing is killing our culture / industry / society etc.

10. Because of the reduced barriers to entry and the rise in competitors that were once former consumers (e.g. Amazon vs Infrastructure Providers, Bloggers vs Newspapers), someone will write a book on "Democratizing the means of manufacturing" or something like that. I wish they wouldn't.

11. The trickle of adoption to these new manufacturing techniques and practices will become a flood as the combined forces of efficiency (through commoditisation of the activity), increased agility in building higher order system (componentisation) and future sources of worth (Schumpeter) kick in. This will take everyone by "surprise" especially the analyst who expected the change to occur slowly in a linear like fashion (which it never does).

12. With the new techniques and practices there will be a growth in concepts of Agile Manufacturing followed by endless arguments over Agile vs Six Sigma equivalents. Eventually people will realise that one size doesn't fit all and both approaches have their place.

13. As commoditisation of the manufacturing process spreads and the explosion in higher order systems and new industries accelerates there will be a corresponding explosion in un-modelled data. This is data which we don't know how to model yet but will eventually be modelled as we understand it more. Annoyingly, people will call it unstructured and probably create a term like "Big Data 2.0" ignoring the fact that the current Big Data revolution is about the fifth one we've been through in the last 200 yrs.

14. As hardware becomes more malleable like software, some bright spark will realise that the function of a device consists of digital and physical elements both of which can be described by code and hence a new language will form. In this new language you will describe the function of the thing you want and a compiler works out which bits should be code and which bits should be bits to be printed. They probably won't call it "SpimeScript" but they will come up with an equally daft name.

15. The open meme (e.g. open source, open hardware, open data, open APIs) will happily continue its symbiotic relationship with the Internet and grow rapidly in this space. Former product design companies will get fairly grumpy about this and there will be endless patent arguments. The patent system will become hopelessly outmatched for this world and will become as harmful for manufacturing as it has been in the software industry.

16. Google, Amazon and the like will swoop in with their normal two factor market and platforms plays to grab the developing ecosystems in this space. Many manufacturing, architectural and construction companies will find themselves now competing with the likes of Amazon backed by huge ecosystems of companies selling designs for direct printing (from that new watch to your new home). The results won't be pretty as usual.

17. Some Government official (probably pushed on by lobbyists from past industries) will start to talk about the needs for certification of designers, architects and the like on the grounds of "consumer benefit". Some standards body will jump on this as a potential new revenue stream. It will all go badly wrong.

18 ... oh, it's late. You get the gist. Just go back and replay every other industrial cycle of change.

A dangerous intervention ...

A few years ago, I warned about the dangers of Government intervention and creation of certified cloud providers. After reading the EC communication on Cloud Computing, my concerns are that we will start on this path.

First, some background :-

Cloud computing represents the evolution of a range of IT related activities from a product and product service model to one of commodity and utility provision. As such it has impact in terms of efficiency, rapid generation of higher order systems and new sources of value creation. In much the same way utility provision of electricity spurred on new activities such as radio, telephone, computing and new industries around this.

However, it's not just activities (what we do) that evolve but practices (how we do things). Hence architectural practices related to computing are co-evolving with the activity of computer infrastructure and we see the rise of what is known as "DevOps". This is all normal fare and these cycles of change repeat throughout history.

Each time we undergo such a change both consumers and vendors can have inertia to it. 

In the case of business consumers, some of this inertia is due to the changing architectural practice and the issue that legacy applications are not designed for the cloud. Hence there is often a desire on their part to have the new model operate like the old i.e. to have clouds which actually have the characteristics of more old fashioned data centres, a concept commonly dubbed as the "Enterprise Cloud". This inertia can be exploited to extend old business models.

In the case of vendors, much of their inertia is due to past success i.e. they become habituated to selling customers products and product rental models and are encumbered by their existing business. Which is why this period of change is often initiated by new unencumbered entrants (in the case of computing infrastructure this would be Amazon). The past giants often attempt to continue or extend their business models, believing the change is just a fad. History is not on their side and the result is disruption of the past models.

Today, we have all these issues with cloud - the new entrants providing utility services for computing, vendors and customers often suffering inertia and the attempt to extend past models.

EC communication on Cloud Computing

The EC communication recognises many of these changes and talks of the industrialization of IT. It notes the importance in forming competitive markets of utility providers with easy switching in order to avoid lock-in and other concerns consumers have. This is all reasonable.

The EC communication highlights its intention to help standards form in the industry in order to achieve these goals of competitive markets and choice for consumers. This is also reasonable if the approach is one of adoption of defacto standards (i.e. market chosen) and ensuring their provision both as open standards (i.e. royalty free) and through reference implementations in order to ensure semantic interoperability. These reference models (i.e. an expression of the standard as code) would need to be open sourced in order to ensure a competitive market which is free from constraints or control of specific vendors.

Already, in the infrastructure space, such candidate reference implementations exist with 850+ companies now involved in the Open Stack effort, an entirely open sourced IaaS which is at the heart of an industry movement towards creating competitive, service based markets. It's one of many such potential reference implementations - others including CloudStack, Eucalyptus and Open Nebula. In light of these developments, an EC approach of supporting any defacto provided as open standards with open source reference implementations should suffice in encouraging competitive markets.

Another part of communication discusses the need for voluntary certification and the EC's intention to enable this. If a competitive market of multiple providers forms then as with other utility industries such an assurance / compliance industry should be expected to form. The objection with the communication is that given the development of the market there is no need for intervention of the EC with certification. The danger is the EC risks "blessing" the certification programs it involves itself with.

Such an act can have negative consequences because certification is fundamentally about approval of certain characteristics however the market has not yet come to terms with what characteristics are essential and many business consumers in the market are suffering inertia to change. In these circumstances, certification could be manipulated to "bless" past business models i.e. to create a preference for "Enterprise Clouds" and thereby play upon the inertia that some companies have.  For example, a focus on SLAs can be used to strengthen the case for "Enterprise Clouds" over "Commodity Clouds" despite the latter being the more evolved state that industry will head towards.

The effect of this can be detrimental to industry, as companies adopt services that they wish to believe represent the future and are "blessed" by the EC through supported certifications but then discover the services represent a continuation of old models which take advantage of their inertia to change i.e. the quick fix to adopting cloud is simply to rebadge your existing enterprise hosting service as "Cloud".

A counter argument to this, is that though there is a risk, the trade off to this is encouraging adoption however it assumes that the EC is more capable of deciding what certification is required than the market. Given a situation where there is already significant market efforts to make interoperable systems and competitive markets (such as Open Stack and its 850+ member companies) then this seems premature.

I would agree with the EC effort to support standards, where those standards are designed to encourage a free (as in from constraints) and competitive market and hence based upon defacto (i.e. market chosen), open standard and open source reference models.

I would not agree with the EC effort to support certification programs as these would naturally form in a competitive market and the EC runs the risk of interfering with the process before the market has chosen and the EC cannot be sure that such interference will be beneficial and not manipulated.

The Standards Jungle ...

The EC has released a communication on cloud computing

I've had a brief read, I'm in the middle of other work but I'll note that the general comments about the importance for future industry, competitiveness, good historical comparison (industrialisation of IT, comparison to utility) etc are all fairly standard fare. Though there's nothing particularly interesting it's good for them to repeat it. 

There's also the usual wobbles on cloud reducing cost, alas Jevons' paradox ... any reductions will be temporary as we will end up doing more.

However, regarding the Fragmentation of the digital single market and the Standards Jungle, I'll make a couple of observations.

In the value chain associated with cloud, one of the components is obviously enough the Internet i.e. availability of this commoditised means of mass communication has enabled commoditisation of discrete IT activities towards utility services (e.g. IaaS). Now, the Internet is as much a social as a technology phenomenon inherited from the hacker ethos - anonymous, decentralised, egalitarian, interactive, neutral to end user etc. 

The Internet's existence also allowed for the easy formation of communities such as as various open technology communities (e.g. open source) which themselves have inherited the hacker traits. There's a symbiotic relationship here between the Internet enabling the communities and the communities in effect sabotaging mechanisms to control the Internet (either corporate or Gov) through the technology they build.

Today, open technology is huge business and most of this is unaccounted (see the clothesline paradox). For example the economic contribution of Apache is enormous but not normally reported - O'Reilly has a fantastic report on this.

So, onto this symbiotic relationship between the Internet and the open technology communities, environments which by their very origins are anonymous, decentralised and neutral which have generated huge economic value including spurring the whole cloud industry most of which also depends upon open technology - we want to apply more centralised legislative frameworks with concerns over areas such as security (in principle due to anonymity)? 

Hmmm, this feels like it could be a repeat of how the radio spectrum was divided up due to security concerns for the benefit of a few companies but in the process managed to destroy a much wider ecosystem. However, this time the economic value at risk is huge. I'd be naturally cautious with any such measure.

So, unsurprisingly, I have concerns over areas such as the certified cloud providers / standards approach.  I'd want to look closely at the details of this. i.e. could this end up being a rehash of the OSI vs TCP/IP debacle? 

Overall : my view on this is 50/50. In some parts it is probably uncessary and if badly handled then potentially downright dangerous. I'll write something more on this when I've had the chance to read it properly.

Something I'll be coming back to ...

I'm just finishing my latest research and I thought I'd share a graph with you as I'm going to be using it from time to time.

The graph is a comparison of a reasonable sample of companies across two axis - the level of strategic play vs the use of open as a means of competing against, outmanoeuvring others.

The bubbles size represent the % of two populations - those that consider themselves Traditional and those that consider themselves more Web 2.0.

Basically, companies break down into four groups :-

• Players : companies which think strategically about IT and are willing to use open as a means of competing. (STRATEGY + ACTION)

• Thinkers : companies which think strategically about IT but don't tend to use open as a means of competing. (STRATEGY + DECISION NOT TO ACT)

• Believers : companies which don't think strategically about IT but do use open as a means of competing. (NO STRATEGY + ACTION)

• Chancers : companies which neither think strategically about IT nor tend to use open as a means of competing. (NO STRATEGY + NO ACTION)

There's a whole underlying model (all to do with evolution) of why these groups exist and the impact of them, their success etc. However, I'm not going to go through this now.

I just wanted to say that being in the Chancers or the Believers group doesn't seem to be very healthy. Which is why I've recently been posting about "What's my IT Strategy" and "Open by Default ... No Thanks"

You see, there is method in my madness and there are reasons why I raise these questions.

Open by Default ... No Thanks

Despite being involved in the open source community for over a decade, I am not a fan of open by default. My preferred route for deciding whether to deploy an open technology approach as in open sourcing a project or providing it as open hardware or open data or via an open API is :-

1. Map out the value chain of a revenue stream within the organization. The value chain should contain all the systems (activities, practice and data) needed to create the high level product or service.
2. Apply evolution to the value chain. For example, determine whether the activities are in a state of genesis or more commodity. Care should be taken to ask whether it is widespread and well defined in the industry and not how any specific company considers this activity (as the company maybe suffering from inertia to change). Where a difference exists this should be marked and an inertia barrier added. 
3. Apply forcing functions to the map for where open technology already exists and is driving a system to a more evolved state. 
4. Mark barriers of entry (whether regulatory or requiring high amounts of financial, physical or human capital) and impacts on supplier and buyer strengths.
   
5. Determine your intent (from efficiency to standards game to recruitment) and use this map to determine whether to open something or not.

Wow, this sounds like a lot of work just to answer a simple question of whether to open something or not? Actually, it isn't. Mapping out the value chain of an organisation should take a few hours at most.

The point of the map is it gives you some situational awareness. It can be used to anticipate changes and competitors action. It also forces you to think about what you're trying to achieve with an open technology approach like open source, how you want to manipulate the market, what opportunity exists, what strategy are you following?

What I don't like about "Open by Default" is it often rapidly becomes "Open without Thinking" and alas this degenerates into "Open without Value" i.e. it's the act of thinking about how to manipulate a market which is more likely to mean a business will value it and put resources, effort and focus into an open technology effort.

If it turns out that you can't perceive any value whatsoever in the project then by all means throw it out there with the hundreds of thousands of other open "meh" projects. Someone might find it useful, it might have some form of beneficial effect you hadn't realised or it might help you hire that skilled engineer / data scientist. But at least, you've thought about it.

Thinking about something generally translates into "we value this". Open technology approaches are powerful tactical weapons in the competition between companies and are far too valuable to be simply ignored or launched without thinking.

Unstructured vs Structured

There are many terms I dislike from the misuse of the term innovation to the whole computer utility vs cloud debate. Another example of this is the whole unstructured vs structured data argument.

The terms unstructured vs structured implies there is a difference in the data sets i.e. unstructured data is by its very nature unstructured and unlike structured data. The term implies this is a permanent state of affairs, a set of data which has no structure and therefore cannot be modeled.

However, our entire history of scientific endeavour can be broken down into discovery of data we didn't understand, creation of models to explain the data and finally data we now understand. In other words we constantly move from unstructured to structured via the creation of a model.

Hence I prefer the term un-modeled data vs modeled data. Inherently this implies there is not a difference in the data sets just simply in our ability to model. It implies that there will be a movement from one to another.

What is your view? Am I the only person who dislikes this framing of unstructured vs structured?

The Real Company syndrome

Ten years ago, I went on the Canon Corporate Executive development program at INSEAD and other institutions.

I remember it well because on the first day, in the first meeting we examined the question of the impact of the internet and what Canon's involvement should be - which was the overall theme of the course.

In a group of executives, there was a massive majority opinion which was very strongly voiced that it was just a channel and Canon shouldn't get directly involved. Then there was me, a single voice (ok, I was CEO of a subsidiary) up against this solid wall of opinion of SVPs, VPs from all over the globe.

Fortunately I've never been shy of a fight. Opinions changed by the end of the six month course but that first meeting always stuck in my mind because of one phrase.

The phrase I heard at that meeting was how this or that company wasn't a "real" company. In that meeting it was how Amazon wasn't a real company unlike Kodak.

Today of course, no-one would dream of saying that Amazon wasn't a real company but I keep on hearing that phrase. In the cloud space it was "but are there any examples of 'real' companies using cloud' etc.

I'd say how about Netflix and the response would be "no, 'real' companies not these tech companies'. Ensue argument over what defines a company, how all media companies are tech companies etc etc.

Where does this awful term 'real' company come from? Is it simply an artefact of inertia and a mechanism for manifesting denial over change?