The seawall

OK. I’m pretty passionate about, and fascinated by, what’s happening with LLMs since GPT-4 (so as not to use another term).

My position is that the coding assistants built with LLMs are going to have a monstrous impact on the way we produce software. By monstrous, I mean the end of the shortage of (good) developers, with at least a 30% drop in demand. Why that number? It’s what I can see in the productivity gain of a developer who uses an LLM tool in their production (and it’s the low end announced by every Amazon/Google/Microsoft for their code-specialized LLMs).

I think this impact is going to be massive and fast, within a few months, the adjustment variable being the speed of adoption by companies. If I believe what I hear around me at large French accounts, Microsoft’s salespeople are doing the job, pitching Copilot X to one major account after another.

However, there are 3 brakes on this adoption, in my view:

• individual contempt
• sending code to a third party
• limitations on context size

These brakes are, for me, “the seawall” that holds back the surge. 6 months after the launch of ChatGPT, the opening of the OpenAI APIs, and 4 months after GPT-4, I think it’s going to give way fairly quickly, for the reasons I set out below.

Contempt

I’m not too worried about this brake, which is above all an individual one. For more than 10 years we’ve been chasing anyone who knows how to line up lines of code at an acceptable level of quality. Inevitably, having a machine that fills 30% of their tasks triggers, for most developers, a few haughty reactions. I think this contempt/rejection will stop after an “Aha moment” from heavy use, or when they’re forced to use them because standard productivity will be assessed with a coding assistant. In short, no choice but to use it.

Sending code to a third party

Here we hit an important brake that affects most organizations. Who would accept that their code, an important asset, be sent somewhere else? The first reaction is: nobody. The second is: “well, we already do it with public cloud, and besides we use Office365, you just need the same contracts.” So while you can understand some organizations blocking this to avoid “rogue” use of ChatGPT, with confidential data sent out in bulk, contracting with Microsoft/Amazon/Google will provide guarantees and this brake will give way fairly easily. This brake is obsolete anyway on a very near horizon: every organization, every individual, will be able to have their own LLM, “on premise”, with no data sent outside. Companies already offer on-premise LLMs (see the table below). But I think the future will belong rather to open-source LLMs, an intuition confirmed by this leaked internal Google memo. In it, the author or authors think that, given the current performance of open-source LLMs, neither Google nor OpenAI actually has any competitive advantage.

Context limitation

As a consequence of the last point, if the current use of coding assistants is limited by context size (they don’t have knowledge of an entire codebase), the arrival of easily deployable open-source models, as capable as a GPT-4, that we’ll be able to fine-tune on our own code, will settle the question.

Seen on LinkedIn, a fairly interesting blog post by Gergely Orosz on the alternatives to Copilot and ChatGPT, with this table in particular:

I note 3 solutions: Tabnine, Codeium and Tabby, which are self-hostable and which can be fine-tuned on the codebase. On the other hand, I didn’t see, in the demos, an integrated Chat that, like ChatGPT, lets you explain code.