Why the fuss about conversational programming?
(a slightly more upto date version is on medium - I keep these as my original first drafts).
First, there isn't much fuss ... yet. But there will be.
To understand why, I'm going to build on my previous HackerNoon post on "Why the fuss about serverless". That post discussed the historical rise of DevOps associated with changing characteristics of compute (the shift from high to low MTTR), the rise of serverless and the development of an emerging practice built upon serverless. I called that practice FinDev in 2016. In the end we finally got a moniker of FinOps (2018) and subsequently a foundation, a book (O'Reilly Cloud FinOps) and several conferences built around those concepts of visibility into financial value and gluing together component services. This is all good but it's not the end of the story.
One of the questions that I was asked back in 2016 was "What comes after serverless"? I responded "Conversational programming". It's about time that I make clear what I mean and what its impact is going to be. At the same time, it's probably worth discussing platform engineering which is a combination of the useful with the downright harmful. However, before we can get started, you'll need some background information.
BACKGROUND
With these basics in place, let us draw a map of the existing landscape.
THE MAP
Let us start with a basic map of technology, see figure 1.
Figure 1 - A basic map of technology
In the map above, a user has some need which is normally met by an application running on a device. The application is coded in some form of IDE which is built upon some concept of coding practice. That coding practice requires a run-time (e.g. Lamp or .Net) which has composable elements (e.g. libraries) which in turn run in some form of container (whether a virtual container or operating system). These containers runs on some form of compute provided through a concept of architectural practice (e.g enterprise class machines).
A number of components are shown as squares. These represent a pipeline of choice i.e. for applications we have a choice from novel apps to common place apps. Pipelines are used in maps when we have mutliple things with a common meaning i.e. power can mean renewable or fossil fuel or nuclear. Each of the choices that we can make are often independently evolving things. We can also use a pipeline to represent a choice in the evolution of a thing for example when discussing TV series we can talk about the first ever example for a particular format (X Factor) or a more evolved and repeated format (X Factor USA, x Factor UK, X Factor etc).
To explain this more clearly, let us expand out the map to discuss the serverless space.
Figure 2 - The Serverless Map.
In the map above, I've expanded out a number of the pipelines. For example, in compute we had the choice to use servers or cloud circa 2006 and onwards. For architectural practice we had developed best practice for use of servers (capacity planning, scale up, N+1, disaster recovery test) and we developed emerging architectural practices for compute as a utility (cloud). That practice evolved, was given a name DevOps and is currently good practice (there is a convergence in terms of what DevOps means). The "best" practice for compute as a product is these days called Legacy.
Equally, from 2014, the run-time has the option of Lamp / .Net or serverless environment such as Lambda or Azure. The coding practice itself has changed (the subject of the earlier post) with greater use of financial metrics and component services with the code acting more as a glue.
Now, all of these components are evolving, so let us bring it upto date by marking on the evolution and actually date the map. Given we're already discussing discrete components in the pipelines, we can simply remove the surrounding pipelines. This give us figure 3.
Figure 3 - Serverless Map, 2023.
From the map, servers shifted to cloud and enabled a practice called DevOps which is rapidly evolving heading towards best architectural practice for cloud. The legacy practice is actually best architectural practice for compute as a product (i.e. servers) but we call it legacy because it's on the way out. Common libraries are evolving to more component services in the FinOps world of serverless whereas best coding practice for use of Lamp / .Net is built upon the concept of common libraries. It too is destined for a moniker of legacy. The Lamp / .Net world is tightly linked to underlying orchestration tools and containers whereas in the serverless world the underlying architecture is abstracted away. In other words, in the serverless world you don't care about underlying infrastructure.
Of course, there will always be exceptions such as being a major scale provider of a component i.e. AWS worries about racks and physical servers because it provides EC2. It worries about infrastructure because it provides Lambda. However, most of us do not operate at this hyperscale and resistance to using such services is not normally based upon positive ideas of a better service but fear i.e. fear of lock-in, fear of loss of control. In other words, it's normally inertia to change or in some cases a percieved regulatory barrier to adoption. I say perceived because in almost every single instance where I've been told "the regulators won't allow us" - the regulators weren't actually the problem.
This is not to say that lock-in is not a concern but our lack of understanding of physical and digital supply chains including how evolved the components are, the excludability of components, their substitutability and rivalrousness means that we have little to no visibility of the risk in our supply chains. Even the US executive order for SBOMs is only a starting point on a very long journey. The blunt truth is that Microsoft, Google and AWS will almost certainly have a far better understanding (as exhibited by their ability to provide some embedded carbon information) of their supply chains along with greater resilience than your home grown operation. In a global shortage for silicon chips, these hyperscalers are more likely to secure supplies than your "mom and pop" investment bank operation serving a few million customers or your "corner shop" University with a few tens of thousands of students. Anyone in purchasing will tell you tales of how difficult it has been to get hold of computers and peripherals.
WHAT IS CONVERSATIONAL PROGRAMMING?
When you think about the act of writing an application today, it is often an act of gluing together a few discrete component services with some code in a utility run-time environment such as Lambda. Well ... at least, it should be. There are an awful lot of organisation dealing with much lower order components such as racking machines or worrying about container orchestration than there needs to be. That's normal, the Red Queen effect doesn't mean everyone changes at the same time. It's a non linear shift (often called a punctuated equilibrium) and companies will get there eventually. However, if you want to read about what good looks like then I'd suggest "The Value Flywheel Effect".
Even in this serverless world, the act of programming still requires you to think about what component services need to be glued together. That means you have to break down the problem into components, find component services that match, determine what is missing and hence what you will need to build, then build it and glue it all together. That is still a lot of work to be done and to be blunt, it's work that can mostly be automated and achieved through some form of intelligent compiler. This leads us to conversational programming.
Let us think about our IDE (integrated development environment). Today, they are very human centric i.e. built upon an expectation that humans will write the code. In a conversational programming world you tell the system what you want, or least provide it prompts for that. The IDE will be more built around the concept of Human + AI rather than just human. Let us map that out in figure 4.
Figure 4 - Conversational Programming, 2022
The rapid evolution of large language models towards more of a commodity service will enable more conversational styles of programming. If you think this is science fiction then an example of this was provided at AWS RE:Invent in 2019 by Alex. This doesn't mean that the system will build everything for you, there will always be edges that need to be crafted but the majority of what is built today is repetition of code that has already been done.
The modern description of conversational programming is prompt engineering. Examples of which can be seen using large language models such as OpenAI. It's only a matter of time before OpenAI is tightly coupled into Azure's development environment and programming will start to look more like a conversation between an engineer with an AI making recommendations for changes and addition of services. If you wish to see the future then a wondeful example of conversational programming can be found in the marvellous StarTrek Voyager and the "Delete the wife" scene.
Of course, much of this will start with text based system but it's a small jump to voice from there. What is relevant is the conversation itself and not the medium (text or voice). One thing you might note in the map is how I've linked FinOps to conversational programming. Serverless has brought remarkable changes such as refactoring having financial value to the focus on financial visibility within code (including carbon cost of code) and these are unlikely to be lost in a conversational programming world. Again, those decisions are ones which an AI can help with. It's not just the code itself (and reducing duplication) that will matter in these IDEs but the meta data suchy as the cost per function and capital flow within an application whether carbon or dollar or yuan.
We're still waiting for those conversational programming environments to fully form but we're getting close. The technology is there (i.e. large language models), the concept is there (i.e. conversational programming) and the attitude is there (i.e engineers getting swamped by complexity). All the factors needed are in place, it's only a question of how quickly this evolves and which actor launches first - Microsoft or AWS? Of course, whomever launches first and drives this to more of a utility will gain the advantage of the meta data for applications built on top. This is a huge strategic advantage which in the past AWS has thoroughly enjoyed and made use of (see Reaching Cloud Velocity) and it's at the heart of the ILC model (described in that book). Which is why I can't see AWS doing an IBM and letting Microsoft walk away with this show. It'll be an interesting battle.
SO WHEN WILL THIS HAPPEN?
As with all these changes (cloud plus devops, serverless plus finops, conversational programming plus any new moniker for the practice to built on top), there will be the usual gains in efficiency, speed and new sources of value. It will be a punctuated equilibirum (non linear change) which means it will seem to be growing slowly but the doubling rate will catch out most analysts. There will be the usual inertia, the usual crowd of CxOs dismissing it as a fad followed by the usual panic and scramble for skills. There will also be the usual nonsense peddled by large management consultants. It's probably worth listing these :-
1) You'll need less engineers. Nope. See Jevon's paradox. You'll need to retrain to a new world but you'll end up doing more stuff. You'll need those engineers.
2) It'll reduce IT budgets. Nope. See Jevon's paradox again. You'll end up doing more stuff, more cost efficiently.
3) You have choice. Nope. See Red Queen Effect. This is only a question of "when" not "if".
4) It's only for startups. Nope. Startups have less past success and hence lower inertia barriers to change. Large enterprises will resist the change due to the pre-installed capital. Eventually, they will have no choice. See Red Queen Effect.
5) We can build our own. Nope. Well, technically you can but you'll regret it. Doesn't mean that want stop hordes of self interested vendors trying to persuade you to do so for reasons of "security", "lock-in" and "customisation to your needs".
6) I can make a more efficient application by hand crafting the code. Nope. Well, technically you can but the time taken to hand craft it all will be vast (especially if you decide to go down to the level of containers or even worse hardware) compared to the speed at which competitors will move. I'd also suggest reading into Centaur Chess if you think even the most gifted engineer will outcompete an average engineer with an average AI.
7) It'll be the death of DevOps / FinOPs etc. Nope. Well, technically it will be but that takes a very long time. Never underestimate how long legacy (i.e. toxic) IT sticks around. So whilst it'll take 5-8 years to see who the winners and losers in this conversational programming world are, it'll take 10-15 years to become seen as the new norm and anywhere from 30 to 45 years for the old world to truly disappear into very small niches.
A NOTE ON PLATFORM ENGINEERING
If I look at the map above, then all those components on the right hand side can be discussed as building a platform - a cloud platform of utility infrastructure, a serverless platform, a platform of component services and eventually a conversational programming platform etc. In general, it's not a good idea to provide components as services exposed through APIs to others unless those components have become industrialised which is why conversational programming requires large language models to become more industrialised.
There are a number of discrete skills - code respository, toolsets, monitoring - around those "platforms" but in general the main platform principles needed are build discrete components, build WITH discrete components and shift as much of the platform to utility providers. Unfortunately, the term platform engineering seems to have got wrapped up with the idea of building your own platform. This is downright harmful if there are utility providers out there. I've even listened to people talk about their data centres as a platform. I'm afraid, those companies are going to struggle in a world of conversational programming particularly as the training for some of these large language models can run into the hundreds of millions of dollars. I'm sure there will be vendors willing to sell you this but I would pause before spending and think about all those large data lakes you were sold and how much ROI you actually got or think about those private cloud efforts or how much return you're getting on a kubernetes cluster in a world of serverless? Caveat Emptor.
WHAT COMES NEXT?
Oh, that's where the fun really starts. If you look further up the map (figure 4) around the area of application and device this is where we get into the world of Spimes and SpimeScript. Though I suspect we're going to call that CyberPhysical. Anyway, that's another post for another year, we're quite some way from that at the moment.
SUMMARY
In summary, get yourself ready for a world of conversational programming. We're not quite there yet but we should be there soon. When it arrives, embrace it and thank me later.
