In the last post, I explored the idea of code as maps. In this post, we will dig a bit deeper and try to understand the essential role of open source.
To begin with, let us understand what code (or software) is. Code is a set of symbolic instructions. They are symbols that alter the behaviour of the system.
The following symbols ...
#include <stdio.h>
int main() {
printf("Hello world");
return 0;
}
... alter the system's behaviour, making it print the words "Hello world" to the screen.
If you think about where we are heading with conversational programming, we are moving from a world where code is just text and into a world where other forms of code can exist, such as maps. The reason for this is that both enable different forms of discussion to occur around the problem space. In the world of text, the conversation tends towards the rules, style and syntax. In the world of maps, the conversation tends towards objects, relationships and context.
You've probably seen this for yourself. Walk into any engineering department, anywhere and you'll find whiteboards. A very different conversation is happening on the whiteboard compared to what is happening when programming the text on a screen. Both conversations relate to the problem space.
In figure 1, you have the text (code) that creates a map. I've provided an example for you to look at. The conversation we have around the map is very different from the conversation we have around the text.
With earlier versions of ChatGPT (a LLM or large language model), I could provide the code and discuss things such as syntax. With ChatGPT4v (a LMM or large multi-modal model), I can provide the map itself and start a conversation about its context. I'm not saying that we will only use maps to code but instead that the entire set of symbolic instructions that we use to program something will include text and maps. In fact, it already does.
ChatGPT is a system which has been trained on a lot of data. That data consists of symbols and those symbols have changed the behaviour of the system (the model) through training. As a good friend of mine, Adam Bouhenguel, points out, the data is your code and the compiled version is your model weights.
With multi-modal systems (such as ChatGPT4v), the "model" can be trained with images, with sounds, and with a variety of other sources. Maps are just images, and if ChatGPT4v wasn't programmed (i.e. trained) on images, then it wouldn't be able to interpret, analyse and have a discussion with me around the map.
Our world is increasingly one where the symbolic instructions (the code) that change the system's behaviour (i.e. the effect of programming with code) include text, data, images and a vast variety of other things. Text, data and images are all "code" in this world. They create the software that we use. In reality, they've always been "code" (ask any modder), we've just become a little preoccupied with coding in text.
There is a danger here. Over many decades we have fought the open source battle to wrestle control from the hands of the few and give to the many. In our new world, the few have once again brought their proprietary ideas to the space and demanded subservience. Even worse, they've dressed themselves up as being open. They will often open source "code" by which they mean a subset of the symbolic instructions used to program the system. They don't open the training data (which is also part of the symbolic instructions used to program the system), they might not even open the model weights. It's not open source.
Basic freedoms of open source include free redistribution, access to source code, freedom to modify, freedom to create derivative works and non discrimination. Without all the symbolic instructions used to program the system then you can't freely modify it. What if I want to reprogram my system by changing some of the training data ... I don't have access to that. It's not open source even when you hand me the model weights because that's the compiled version. I want the actual code, the training data used. The joke of course is that even the "enlightened" who share model weights can often add clauses on their use. There is nothing remotely open about most of the AI projects out there claiming to be open. It's all openwashing.
To confuse things more, the OSI is looking to redefine what open source means. Why? There is nothing wrong with what open source means. The problem is a whole bunch of people trying to claim that code (a set of symbolic instructions) doesn't include data or images (a set of symbolic instructions) and that somehow software (a collection of symbolic instructions) only includes the text bit. The OSI just needs to make it clear that this is not open source.
Laughably, the UK led an AI safety summit last week. I say laugh but if you're in the UK then you might want to cry especially given so many voices were ignored. You would have thought that handing over national sovereignty in the landscape of technology to a few would be the major safety issue. Apparently not. When it did get mentioned it was often along the lines of the hazards of open source, guardrails and control which is exactly where the lobbyists would take it. A ray of sunshine however came in the form of Oliver Dowden who provides some backing to open source AI. Alas, not all countries are as strategically inept as the UK. The summit was barely underway before China Gov signalled its intended role in open source AI and a number of Chinese enterprises have reinforced that message since.
Fabulous. We get to sit and watch China accelerate, take over the AI space and have the benefits of AI shared amongst its population as we enter this new world of conversational programming whilst we hand over sovereignty to a few as the "great and good" chatter about safety and how they've saved us from some future mythical frontier AI because of a signed note whilst missing the iceberg shaped hole that is sinking the boat. Top job. Trebles all round.
If anyone is laughing, it has to be China.
IN THIS SERIES on "Why the fuss about conversational programming" …
