Saturday, June 02, 2007

The right tool for the job?

I was reading SaaS blog and Matt Ammerman's article on online IDEs, it's worth a look.

Now there is nothing revolutionary about providing a utility computing cloud through an online development environment - it's just a common sense application of previous ideas.

Since we launched Zimki back in early 2006, we've talked constantly about "yak-shaving", the benefits of simplifying the process of development and for only paying for what you consume etc. We've haven't just been handing out the T-Shirts though, we've been doing this for real. There are some really decent ideas behind it.

Firstly,"it is more efficient to have multiple network services share a common infrastructure that can absorb failures and bursts in client demand than it is to have every service over provision resources to accommodate peak requirements" - see Chaki Ng et al.

Secondly, assuming that every provider uses a common standard or engine then resilience can be achieved through a network of providers greater than any single provider.

The key to this all, is creating the federated grid of common service providers (think of a national grid of electricity providers), an exchange of computing resources and eventual virtualisation over multiple providers and even P2P. The problem has always been, how do you achieve this? You can't just move from one environment to another and expect it to work, there's usually installation, set-up etc, limitations involved.

Hence the reason why open source is such an important part of this puzzle. It's all about building that free and open standard to allow this to happen.

This has been a personal mission for me, as I hate all the stuff which gets in the way of me building something new. I want the freedom to build my application and it's data and release it into a cloud, knowing that it will work. I want my application to move from one cloud to another either because of demand or because I tell it to or because some fault happened somewhere. I want to know that it will still work. I don't want to be shackled with building infrastructure, worrying about capacity planning, demand spikes, disaster recovery, load balancing, test and staging environments and worst of all the cost of it all. All this stuff gets in the way of me building - I want freedom from this "yak shaving" - I just want to build new and interesting things.

Undoubtedly, at the beginning, any standard will come with limitations.

With Zimki you can build whatever you want but you need to write it in JavaScript, which runs on our servers. We chose JavaScript because of web services (XML, JSON etc) and all the other stuff happening on the web - it just seems logical to run the same language on the client that you run in the cloud and relatively straightforward to add persistence to JavaScript on the server.

The plan has always been to open source Zimki to try and create that first standard. Even if it is adopted, Zimki would be just a starting point on this journey. We are going to need others to join us, provide infrastructure clouds which fit with the standard and help develop the standard. Everyone competing and sharing in this federated grid.

But won't that "alienate one crowd that, well, we owe pretty much everything to - software developers and engineers" as Matt says?

That depends upon whom you're talking about - certainly you are going to upset some people because fundamentally it's about commoditisation and turning an often customised but commonly needed item (like a computing environment with CPU resources, storage and bandwidth) into a standard. Commoditisation does means less customisation, but is that a bad thing? Users of a web application have no idea about your infrastructure, they only care that your applications works. In developing an application, I don't care about the infrastructure only that it works reliably, provides my application with the resources it needs, doesn't cost the earth and that if I'm not happy I can move to another environment quickly and easily.

I don't mind if it's two boxes or ten thousand, I'm not interested in how many disks or blue lights there are - I'd rather that this was someone else's problem to solve. I just care that my app works, runs well, cheaply and simply. Of course, some people earn a decent living out of making life complex - I'm sure there were a lot of upset people when the national grid formed, or railway gauges were standardised or TCP/IP became the standard network protocol or when HTTP appeared on the scene.

This always happens whenever you deal with infra-structural goods. There are always some losers, but overall such standardisation is to the greater common good. The upside of commoditisation (forgetting the massive reductions in waste and so forth) is that it allows for new opportunities to develop. It allows for progress.

So yes, it will alienate some people - but for most it will hopefully be liberating, allow for more creativity and free us from the shackles of the old way. All that's needed is an adopted open and free standard. An open sourced Zimki might help us on that road, maybe something better will come along.

I hope so, I've shaved enough yaks in my time.