How Cloud Foundry will save the world from Yak Shaving

I'll make no bones about the fact that I'm a huge fan of Cloud Foundry. It's the right play, by the right people at the right time. Despite all the attempts to dilute the message over the last eleven years, Platform as a Service (or what was originally called Framework as a Service) is about write code, write data and consume services. All the other bits from containers to the management of such are red herrings. They maybe useful subsystems but they miss the point which is the necessity for constraint.

Constraint (i.e. the limitation of choice) enables innovation and the major problem we have with building at speed is almost always duplication or yak shaving. Not only do we repeat common tasks to deploy an application but most of our code is endlessly rewritten throughout the world. How many times in your coding life have you written a method to add a new user or to extract consumer data? How many times do you think others have done the same thing? How many times are not only functions but entire applications repeated endlessly between corporates or governments? The overwhelming majority of the stuff we write is yak shaving and I would be honestly surprised if more than 0.1% of what we write is actually unique. 

Now whilst Cloud Foundry has been doing an excellent job of getting rid of some of the yak shaving, in the same way that Amazon kicked off the removal of infrastructure yak shaving - for most of us, unboxing servers, racking them and wiring up networks is a thankfully an irrelevant thing of the past - there is much more to be done. There are some future steps that I believe that Cloud Foundry needs to take and fortunately the momentum is such behind it that I'm confident of talking about them here without giving a competitor any advantage.

First, it needs to create that competitive market of Cloud Foundry providers. Fortunately this is exactly what it is helping to do. That market must also be focused on differentiation by price and quality of service and not the dreaded differentiation by feature (a surefire way to create a collective prisoner dilemma and sink a project in a utility world). This is all happening and it's glorious.

Second, it needs to increasingly leave the past ideas of infrastructure behind and by that I mean containers as well. The focus needs to be server less i.e. you write code, you write data and you consume services. Everything else needs to be buried as a subsystem. I know analysts run around going "is it using docker?" but that's because many analysts are halfwits who like to gabble on about stuff that doesn't matter. It's irrelevant. That's not the same as saying Docker is not important, it has huge potential as an invisible subsystem.

Third, it needs to provide a mechanism of billing down to the function. One of the things we discovered with Zimki (the first PaaS) was that in a single application it was often discrete functions which generated most of the cost (Zimki had billing at the function level based upon Javascript operations, I/O and network). By having billing at the function, you cause a radical change in the way people write, refactor and refine code. There's nothing like seeing that 70% of your application running cost is being caused by one function to get you to evaluate that function. It also tends to help performance. The existing usage events in Cloud Foundry are fine but not enough. It needs to go deeper (hint to providers - that needs an asynchronous mechanism of logging, things like network taps, it worked a treat in 2005).

Fourth, and most importantly, it needs to tackle yak shaving at the coding level. The simplest way to do this is to provide a CPAN like repository which can include individual functions as well as entire applications (hint. Github probably isn't upto this). One of the biggest lies of object orientated design was code re-use. This never happened (or rarely did) because no communication mechanism existed to actually share code. CPAN (in the Perl world) helped (imperfectly) to solve that problem. Cloud Foundry needs exactly the same thing. When I'm writing a system, if I need a customer object, then ideally I should just be able to pull in the entire object and functions related to this from a CPAN like library because lets face it, how many times should I really have to write a postcode lookup function? 

But shouldn't things like postcode lookup be provided as a service? Yes! And that's the beauty. 

By monitoring a CPAN like library you can quickly discover (simply by examining meta data such as downloads, changes) as to what functions are commonly being used and have become stable. These are all candidates for standard services to be provided into Cloud Foundry and offered by the CF providers. Your CPAN environment is actually a sensing engine for future services and you can use an ILC like model to exploit this. The bigger the ecosystem is, the more powerful it will become.

I would be shocked if Amazon isn't already using Lambda and the API gateway to identify future "services" and Cloud Foundry shouldn't hesitate to press any advantage here. This process will also create a virtuous cycle as new things which people develop that are shared in the CPAN library will over time become stable, widespread and provided as services enabling other people to more quickly develop new things. This concept of sharing code and combing a collaborative effort of the entire ecosystem was a central part of the Zimki play and it's as relevant today as it was then. By the way, try doing that with containers. Hint, they are way too low level and your only hope is through constraint such as that provided in the manufacture of unikernels.

There is a battle here because if Cloud Foundry doesn't exploit the ecosystem and AWS plays its normal game then it could run away with the show.  The danger of this seems slight at the moment (but it will grow) because of the momentum with Cloud Foundry and because of the people running the show. Get this right and we will live in a world where not only do I have portability between providers but when I come to code my novel idea for my next great something then I'll discover that 99% of the code has already been done by others. I'll mostly need to stitch all the right services and functions together and add a bit extra. 

Oh, but that's not possible is it? In 2006, Tom Inssam wrote for me and released live to the web a new style of wiki (with client side preview) in under an hour using Zimki. I wrote an internet mood map and basic trading application in a couple of days. Yes, this is very possible. I know, I experienced it and this isn't 2006, this is 2016!

Cloud Foundry (with a bit of luck) might finally release the world from the endless Yak shaving we have to endure in IT. It might make the lie of object re-use finally come true. The potential of the platform space is vastly more than most suspect and almost everything, and I do mean everything will be rewritten to run on it.

I look forward to the day that most Yaks come pre-shaved. 

Recruiters and daisy chains

The only recruitment firm that I would ever consider using is Superstars. It's run by a dear friend of mine Steve Hutson who I've known for last fourteen years. Steve used to work in the mainstream recruitment industry but began to loathe it because of its practices. He decided to change the way he worked and how recruitment was done. In Superstars, there is no margin, no fees and people are treated as talent, as individuals rather than a commodity shunted about by "human traders" - as one particularly vile firm likes to refer to itself.

There's a lot to dislike about the recruitment industry, so it's good to see Steve succeed in building a new style of firm which changes those practices. One of the worst practices (which I've just caught sight of again, hence the post) is daisy chaining.

Daisy chaining is a based upon the obvious premise that once you find someone a new job then that leaves a hole to fill i.e. their previous job. You can therefore chain a long list of people together and get them to each jump into the next role. This practice is beloved by recruitment consultants as they earn fees on each person jumping, the bigger the chain the more the fee. It also creates a promotion ladder for those in the chain,

I suspect daisy chain is also partially responsible for why I hear large companies complain they can't find big data / IoT talent (when there's plenty out there) whilst real experts in big data / IoT complain they can't find jobs and keep on meeting muppet VPs of big data / IoT who don't understand the basics. The problem is the recruiters want to use the chain (i.e. it's more fees for them) and if you're not part of the chain then you're not very useful. To be part of the chain, you need to have a job so that the recruiter can fill that one to extend the chain and so on.

So basically, just in case it's not clear, the chain works as follows.

Recruiter gets wind (dinners with execs / golf course or whatever) that Company B requires an SVP. Recruiter says they've got the perfect candidate C1 who is currently VP with Company C and offers to make the arrangements, even offering to waive some of the fees because they are friends. Execs agree to meet.

Recruiter phones up C1 and tells them that Company B is desperate to meet with them. 

Recruiter immediately phones up Candidate D1 who is CTO for company D and tells them they have heard that Company C is going to be looking for a new VP. Lines up Candidate D1 for the role.

Recruiter immediately phones up Candidate E1 who is a Engineering Head for company E and tells them that they have heard that Company D is going to be looking for a new CTO. Lines up E1 for the role.

... and so on.

C1 accepts the job of SVP at Company B. The Company B thanks the recruiter for their help - "well, we're partners in this" says the recruiter. Since the recruiter is part of the process, they know roughly when candidate C1 has told their company C that they are leaving.

Recruiter immediately phones up a friendly exec at Company C (those dinners have purpose), chats about how they've heard C1 is leaving. Says they've actually got the ideal candidate D1 who they know is looking and is keen to work there - saves all the hassle of advertising etc. Hint, after laying it on thick about hard recruitment is, many execs are just pleased to hear that a possible replacement can be found quickly. Make introduction. 

 ... and so down the chain you go, every time clocking a % of salary as a finders fee. 

Of course, suppose there is the ideal candidate for one of those roles who is unemployed or not employed in a valuable role that can be easily replaced by another?  Well, they're unlikely to be offered the job because this breaks the chain. Hence if you bother to look, there's oodles of big data / IoT talent out there looking for roles along with companies looking for big data / IoT talent. The problem is your recruitment processes and the companies that you use are broken.