Cloud computing made simple

It has been a truly amazing year since we embarked on our "cloud" journey at Ubuntu, hence I thought I'd review some of the highlights.

We started the journey back in 2008 when Mark Shuttleworth announced our commitment to providing cloud technology to our users. At that time, the cloud world was already in a state of growing confusion, so we adopted an approach of :-

• make the cloud simple.
• focus on one layer of the computing stack (infrastructure) to begin with.
• give our users real technology not promises.
• help drive standardisation (a key requirements of this shift towards a service world) by adopted public defacto standards.
• work with leading partners in this growing industry.
• provide open source systems to avoid lock-in issues.
• actively work to mitigate risks and concerns over cloud by giving our users options.

Hence, in April'09 as part of Ubuntu 9.04 we launched our hybrid cloud strategy.

Our approach was based around the adoption of Amazon EC2 / S3 & EBS as the public defacto standard rather than the creation of some new APIs (there's too many already).

We provided Ubuntu images for use on Amazon EC2 (public cloud) and the technology to build your own private cloud (known as Ubuntu Enterprise Cloud) that matched the same APIs of Amazon. We also added management tools which could cross both public and private domains because of our adoption of a standard API set.

For 9.10 we significantly improved the robustness and ease of setting up a private cloud (Mark built his own several node system in under 25 mins from bare metal). We provided the base for an application store, improved the management capabilities of Landscape and the features of UEC grew extensively. We also launched training, consultancy and support services for the cloud and a JumpStart program to help companies move into the cloud quickly.

During this time we've worked closely with many partners, I'll mention a few (more details can be found on the Ubuntu Cloud site) :-

• Eucalyptus whose open source technology we adopted into the distribution as a core part of Ubuntu Enterprise Cloud.
• Intel's Cloud Builder program to provide best practices on how to create a private cloud using UEC. I'd strongly recommend reading the whitepaper.
• RightScale & CohesiveFT to provide best of breed public management tools alongside our own Landscape system.
• Dell, who will offer a range of pre-built clouds using a series of ‘blueprint’ configurations that have been optimised for different use cases and scale. These will include PowerEdge-C hardware, UEC software and full technical support.

In one year, we've made cloud simple for our users. We've brought our "Linux for Humans" philosophy into the cloud by getting rid of complexity, confusion and myth.

If you want to get into cloud, then we offer :-

• Simple Choices: You can have either private, public or hybrid (i.e. public + private) infrastructure clouds.
• Simple Setup: If you want to build a private cloud, then Ubuntu makes the set-up ridiculously easy. You can be up and running with your own cloud in minutes. Along with with our community documentation covering CD installation and more advanced options, you can also find detailed information on how to build a cloud through Intel's cloud builder program. However, if building a cloud still sounds too daunting then Dell offers pre-built, fully configured and supported private clouds.
• Simple Management: You can use the same tools for both your private and public clouds because we've standardised around a common set of APIs. There's no need to learn one set of systems for private and another for public.
• Simple Bursting: Since we provide common machine images which run on both public and private cloud offerings combined with standardised APIs, then the process of moving infrastructure and combining both private and public clouds is ... simpler.
• Enterprise Help: If you still need help then we offer it, including 24x7 support and a jumpstart program to get your company into the cloud.
• Open source: UEC, the Ubuntu machine images and all the basic tools are open sourced. We're committed to providing open source systems and following through on a genuine open source approach i.e. the system is open source and free and so are all the security patches and version upgrades.

The results of this year have been very encouraging. We recently estimated that there are now over 7,000 private clouds built with UEC, however with 7% of users in our annual Ubuntu User Survey saying that they have built a UEC cloud, the true figure might be very much higher. It was great to hear that almost 70% of users felt Ubuntu was a viable platform for the cloud but there were several surprising statistics including :-

• 54% were using the cloud in some form or another (software, platform or infrastructure). However giving the fuzziness of the term cloud, this can only be seen as a signal of intent to use online services.
• Only 10% had used public cloud providers (such as Amazon) for infrastructure. What was quite remarkable was that given the relatively recent availability of UEC, almost as many people had built private clouds as had used public cloud providers.
• 60% felt that the use of private cloud was more important to their organisation, 25% thought that both private and public was of equal importance whilst only 15% felt that public cloud was the most important.

Whilst this survey was targetted at Ubuntu users, the data we receive from external sources suggest that Ubuntu is becoming the dominant operating system for consumers of the infrastructure cloud space. Even a simple ranking of search terms using Google's Insight around cloud computing show how significant a player Ubuntu is.

Whilst this is great news, what really pleases me is that we're making cloud simple and real for organisations and listening to what they need. We're getting away from the confusion over cloud, the tireless consultant drivel over whether private cloud is cloud computing and the endless pontifications and forums debating vague futures. Instead, we're giving real people, real technology which does exactly what they want.

Over the next year we're going to be tackling issues around creating competitive marketplaces (i.e. more choice for our users), simplfying self-service IT capabilities and orchestration and providing a wide range of open source stacks and platforms to use in the cloud.

We're going to continue to drive down this path of commoditisation by providing common workloads for the cloud (the same as we've been doing for server) and helping businesses to standardise that which is just cost of doing business.

Regardless of any attempts to badge "cloud" as just a more advanced flavour of virtualisation or describe it as "not real yet" by various late vendors, we will be doing our best to bust the various "cloud" myths and push the industry towards competitive marketplaces of computer utilities through defacto standardisation.

Commoditise! Commoditise! Commoditise!

I'm also delighted about our partners successes, with RightScale passing the million server mark, Amazon's continual growth and leadership with the introduction of spot markets, Dell's outstanding move to make cloud mainstream, Intel's push to make cloud easier & Eucalyptus' continued adoption and the appointment of Marten Mickos as CEO.

P.S. if you want to keep track of what's happening with Ubuntu in the cloud, a good place to start is our cloud blog or following our twitter feed, ubuntucloud.

Cloud rant

For the last two posts, I've had a pop at the term "cloud", I'd like to now explain my reasoning in more detail.

First, as always, some background information.

What we know

The transition from a product to a services world in I.T. is very real. It's happening now because of the confluence of concept, suitability, technology and a change it business attitude. Overall it's driven by the process of commoditisation and it is the very ubiquity of specific I.T. activities that makes them potentially suitable for volume operations. I say potentially because volume operations is only viable for activities which are both well defined and ubiquitous.

Fortunately ubiquity has a relationship to certainty (or in other words how well understood, defined and therefore certain an activity is) which is why these activities are suitable for provision on the basis of volume operations through large computer utilities.

Lifecycle

For many years I've talked about lifecycle and the evolution of business activities. Any activity goes through various stages from its first innovation (as per the use of computer resources in the Z3 in 1941) to custom built examples to products which describe the activity. Usually, the activity ends up becoming a ubiquitous and well defined commodity (assuming there are no natural limits or constraints).

During this lifecycle, as the activity becomes more defined in the product stage, service models can often arise (for example the rental model of the managed hosting industry for computing infrastructure or the early subscription like models of electricity provision). As the activity becomes more of a commodity the existence of these service models tends to lead to the rise of utility services (as with electricity provision).

An essential requirement for the growth of the utility model (and the type of volume operations necessary to support it) is that consumers view that what's provided is a commodity. It's little more than a cost of doing business and a standardised version is good enough.

The latter is why a change of attitude is critical in development of the utility service model. If, for example, consumers still view the activity as creating some form of competitive advantage (whether true or not), they are unlikely to adopt a utility model of standard services.

The change in attitude

Over the last decade the attitude of business towards certain I.T. activities has changed dramatically. Recently, a group of 60 odd CIOs & Architects highlighted that many of the I.T. related activities they undertook were commonplace and well defined, particularly across their industry & geography.

Now that doesn't mean they did things the same way, quite the reverse.

Taking just one activity, ERP, then all of these companies agreed that whilst they gain no competitive advantage in ERP, it was an essential cost of doing business. They also agreed that they all had their own customised processes for ERP which they invested heavily in. The shock was their agreement that these different processes provided no differential benefit. It was estimated that for this one activity alone, across this small group of companies, then $600 million p.a. was spent maintaining differences which provided no value.

By reducing customisation through the provision of ERP as standardised services, then each company would significantly benefit in terms of cost savings. Standardisation of processes and removing costs associated with customisation is seen as one of the major benefits of the transition from an "as a Product" to an "as a Service" world.

Let's be clear here, "as a Service" was considered shorthand for "provision of a commodity activity through standardised services via a competitive marketplace of computer utilities". These companies were not looking for an outsourcing arrangement for a highly customised service for their needs, they were looking for a commodity to be treated as a commodity.

The concepts of computer utilities offering elastic and infinite supply on demand, provision of activities through services and commoditisation are all tightly coupled.

The benefits & risks of "cloud"

The benefits of this shift towards service provision via large computer utilities has been discussed extensively for the last 40 years :-

• economies of scale (volume operations)
• focus on core activities (outsourcing to a service provider)
• pay per use (utility charging)
• increased consumer innovation ( componentisation)

However, one critical benefit that gets missed is standardisation itself.

The risks associated with this change (ignoring the disruptive effect on the product industry) can be classified into transitional risks (related to the change in business relationship) and generic outsourcing risks. I've categorised these below for completeness.

Transitional Risks

• Confusion over the new models.
• Trust in the new providers.
• Transparency of relationships.
• Governance of these new models (including auditing, security & management).
• Security of supply

Outsourcing Risks

• Suitability of the activity for service provision.
• Vendor lock-in (& exit costs).
• The availability of second sourcing options.
• Pricing competition.
• Loss of strategic control.

Transitional risks can be mitigated through standard supply chain management techniques. For example with electricity we often combine both public and private sources of provision (a hybrid option). However outsourcing risks require the formation of competitive marketplaces in order to mitigate. Whilst the latter is a genuine concern for companies, even without these marketplaces the benefits of this change are still attractive.

The problem with the term "Cloud"

This change in I.T. is all about standardised service provision of commodity activities through a competitive marketplace of computer utilities. The notion of utility conjures up easily understood and familiar models.

Few would have a problem in understanding how access to a standardised form of electricity through a marketplace has allowed for a huge range of new innovations built upon consuming electricity. Few would have a problem in understanding how the providers themselves have sought new innovative ways of generating electricity. Few would ever consider electricity itself as a form of innovation, to most it comes from a plug and it is critical that it is standardised.

This is a really important point because our companies comprise of value chains that are full of components which are evolving to become commodity and utility like services. This commoditisation enables new higher order systems to be created and new businesses to form e.g. electricity enabled radio, television but also destroys old business. The key here is that whilst commoditisation enables innovation it also destroys the past. The two are different,

The problem with the term "cloud" beyond being fuzzy, is it often used to describe this change in I.T. as something new and innovative. The term helps disguise a fundamental shift towards a world where the bits don't matter and it's all about services. The term allows for all manner of things to be called cloud, many of which have little to do with the standardisation of an activity and its provision through utility services.

You could easily argue the term is misleading as it encourages customisation and distracts the consumer from what should be their focus - standardised services through a competitive marketplace of computer utilities.

Alas, as I said we ALL have to use the term today because of its momentum.

At Ubuntu we focus on commodity provision of activities (common workloads), providing our users with the technology to build a private computer utility (nee "Cloud", as part of a hybrid strategy), the adoption of the defacto standard of EC2 & S3 and we also provide all the technology as open source to encourage the formation of competitive markets.

We use the term "cloud" because it's what customers, analysts and others expect to hear. This of course doesn't stop us from explaining what is really happening and busting the "cloud" myths that exist.

Is your cloud a poodle?

Since we're fond of replacing meaningful concepts such as commoditisation, lifecycle, categorisation and computer utilities with bland terms like "cloud", I thought I'd follow the trend on to its next logical conclusion - Poodle Computing.

The shift of I.T. activities from being provided "as a Product" to being provided "as a Service" through large computer utilities has an obvious next step - the formation of competitive marketplaces. These marketplaces will require standardisation of what is after all a commodity (i.e. ubiquitous and well defined enough to be suitable for service provision through volume operations) and the ability of consumers to switch easily between and consume resources over multiple providers (which in turn requires multiple providers, access to code & data, interoperability of providers and an overall low exit costs)

I won't bore you with the mechanics of this and the eventual formation of brokerages & exchanges, I covered this subject extensively in 2007 when I made my "6 years from now you'll be seeing job adverts for computer resource brokers" prediction.

However, in this future world of brokerages and fungible compute resources (or fungitility as I jokingly called it) the consumer will become ever more distanced from the source of provision. This will be no different to the many other forms of utilities where vibrant exchange markets exist and what the consumer purchases often has gone through the hands of brokers. You don't actually know which power station generated the electricity you consume.

So this brings me to the title of the post. As consumer and the source becomes more distanced, it reminds me of Peter Steiner's cartoon"On the Internet, nobody knows you're a dog".

On that basis, what sort of dog flavour of computing resource will you be consuming?

By introducing the concept of "dog computing" to cover this "cloud of clouds" world (hey, they're both meaningless) then the marketing possibilities will become endless and a lot more fun.

I can see the conversation now, walking into a lean and mean sales organisation and saying to the CEO that they are using "Poodle Computing". Shouldn't they be using our brand new "Pitbull Computing" or at least upgrading to "Springer Spaniel"?

We could always call things what they are (computer utilities & competitive markets of computer utilities") but I suspect we will end up with "cloud of clouds", "cloud exchanges" and an OTC market of ominous sounding "cloudy futures".

What is Cloud?

Before we can discuss this term, a bit of history and background is needed.

Activities

All business activities undergo a lifecycle, they evolve through distinct stages including :-

• the first introduction of a new activity (its innovation)
• the custom built examples replicating this activity (the copying phase)
• the introduction of products which provide that activity ( the product stage, including numerous rounds of feature differentiation which are also unfortunately called product innovation)
• The activity becoming more of a commodity (ubiquitous, well-defined and with no qualitative differentiation). In certain circumstances that commodity can be provided through utility services.

It should be noted that the characteristics of an activity changes as it move through its life-cycle. As a commodity it's of little strategic value (or differentiation) between competitors whereas in its early stages it can often be a source of competitive advantage (a differential).

Information Technology

At any one moment in time, I.T. consists of a mass of different activities at different stages of their life-cycle. Some of those activities are provided through discrete software applications (an example might be ERP), other activities relate to the use of platforms (developing a new system using RoR or provisioning of a large database etc) whilst others relate to the provision of infrastructure (compute resource, storage, networks).

You can categorise these activities into a computing stack of infrastructure, platform and software. Of course you can go higher up the stack to describe the processes themselves and beyond, however for this discussion we will just keep it simple.

What's happening in IT today?

Many activities in I.T. that were once innovations but more recently have been provided as products (with extensive feature differentiation) have now become so ubiquitous and so well defined that they have become little more than a commodity that is suitable for service provision. You can literally consider that chunks of the "computing stack" are moving from an "as a Product" to an "as a Service" world.

This change is the reason why we have the "Infrastructure as a Service" to "Platform as a Service" to whatever else "as a Service" industries. Of course, there are many higher order layers to the stack (e.g processes) but any confusion around the "as a Service" term generally only occurs because we never used to describe these activities with the "as a Product" term.

Had we categorised the previous software industry in terms of "Software as a Product", "Platform as a Product" etc, then the change would have been more obvious.

Why now?

This change requires more than just activities being suitable for utility service provision. It also requires the concept of service provision, the technology to achieve this and a change in business attitude i.e. a willingness of business to adopt these new models. Whilst the concept is old (more on this later), and the technology has been around for some time (yes, it has matured in the last decade but that's about all), both the suitability and change of business attitude are relatively new.

Thanks to the work of Paul Strassman (in the 90's) and then Nick Carr (in the 00's), many business leaders have recognised that not all I.T. is a source of advantage. Instead much of I.T. is a cost of doing business which is ubiquitous and fairly well defined throughout an industry.

It was quite refreshing to recently hear a large group of CIOs, who all spent vast amounts of money maintaining highly customised CRM systems, comment that actually they were all doing the same thing. These systems provided no strategic value, no differential and in reality what they wanted was standardised, low cost services charged on actual consumption basis for what is essentially a cost of doing business. They also wanted this to be provided through a marketplace of service providers with easy switching between them.

This is quite a sea change from a decade ago.

The change from a "as a Product" to an "as a Service" world is happening today because we have the concept, technology, suitability and most importantly this changing business attitude.

An old Concept

The concept of utility service provision for I.T. is not new but dates back to the 1960's. Douglas Parkhill, in this 1966 book - "The Challenge of the Computer Utility" - described a future where many computing activities would be provided through computer utilities analogous to the electricity industry. These computer utilities would have certain characteristics, they would :-

• provide computing resources remotely and online
• charge for the use of the resources on the basis of consumption i.e. a utility basis
• provide elastic & "infinite" supply of resources
• benefit from economies of scale
• be multi-tenanted

Douglas noted that these computer utilities would take several forms as per the existing consumption of other utilities. These forms included (but are not limited to) public, private & government utilities. He also noted that eventually we would see competitive markets of computer utilities where consumers could switch providers, consume resources across multiple providers (i.e. a federated use) and consume all manner of hybrid forms (e.g. private and public combinations)

One final note is the term utility means a metered service where the charge is based upon consumption. That charge might be financial or it could be in any other currency (e.g. access to your data).

The Cloud Term

Between 1966 -2007, the general school of thought grew to be :-

• I.T. wasn't one thing. Many aspects of I.T. created little or no differential value and were simply a cost of doing business (Strassman, 90s)
• There is a correlation between ubiquity of I.T. and its strategic value (differentiation). The more ubiquitous I.T. was, the less strategic value it created. (Nick Carr, '02)
• I.T. activities could be categorised into rough groupings such as software, platform and infrastructure (the actual terms used have changed over time but this concept is pre-80's)
• Certain I.T. activities would be provided through computer utilities as per other utility industries (Parkhill & McCarthy, 60's)
• There were several forms that these computer utilities could take including public, private, government and all manner of combinations in between. (Douglas Parkhill, 1966)
• We would see the formation of competitive marketplaces with switching and federation of providers.
• These computer utilities had certain common characteristics including utility charging, economies of scale, elastic and "infinite" supply etc.(Douglas Parkhill, 1966)
• Whilst all activities have a lifecycle which they evolve along through the process of commoditisation, the shift from "as a Product" to an "as a Service" world would require several factors (i.e the concept, the technology to achieve this, the suitability of activities for service provision and a change in business attitude.)

Back between '05'-'07, there was a pretty crystal clear idea of what was going to happen:-

A combination of factors (concept, suitability, technology and a change in business attitude) was going to drive those I.T. activities which were common, well defined and a cost of doing business from being provided "as products" to being provided "as services" through large computer utilities. The type of services offered would cover different elements of the computing stack, there would be many different forms of computer utility (public, private & government) and eventually we would see competitive marketplaces with easy switching and consumption across multiple providers.

In '05, James Duncan, myself and many others were starting to build Zimki - a computer utility for the provision of a JavaScript based "Platform as a Service" - for precisely these reasons. The concepts of federation, competitive markets, exchanges and brokerages for service provision of a commodity were well understood.

Unfortunately in late '07 / early '08, the term "Cloud" appeared and the entire industry seemed to go into a tailspin of confusion. During '08, the "Cloud" term became so prevalent that if you mentioned "computer utility" people would tell you that they weren't interested but could you please tell them about "this thing called cloud".

So, what is Cloud?

The best definition for cloud today is NIST's. Using five essential characteristics (include elasticity, measured service etc), four deployment models (private, public, government etc) and three services (application, platform, infrastructure) it nearly packages all the concepts of computer utility, the shift from product to services and the different categories of the computing stack into one overall term - "cloud".

In the process it wipes out all the historical context, trainwrecks the concept of a competitive marketplace with switching and federation, eliminates the principle idea of commoditisation and offers no explanation of why now. It's an awful mechanistic definition which only helps you call something a cloud without any understanding of why.

However, that said, NIST has done a grand job of trying to clean up the mess of 2008.

In that dreadful year, all these well understood concepts of computer utilities, competitive marketplaces, the lifecycle of activities, categorisation of the computing stack and commoditisation were put in a blender, spun at 30,000 rpm and the resultant mishmash was given the name "cloud". It was poured into our collective consciousness along with the endless blatherings of "cloudy" thought leaders over what it meant (I'm as guilty of this as many others)

To be brutal, whilst the fundamentals are sound (commoditisation, computer utilities, the change from products to services etc), the term "Cloud" was nothing more than a Complete Load Of Utter Drivel. It's a sorry tale of confusion and a meaningless, generic term forced upon a real and meaningful change.

My passionate dislike for the term is well known. It irks me that for such an important shift in our industry, I have to use such a term and then spend most of my time explaining the fundamental concepts behind what is going on, why this change is happening and undoing the various "cloud" myths that exist.

Being pragmatic, I'm fully aware that this term has enough momentum that it's going to stay. Shame.