The Commoditization of Large Models | #AI23

By now, everyone has understood the significant role foundation models play. Although these Large Models have been around for a while, we haven't seen adoption on a large scale. One reason why the transformation of industries, which we all know this technology is capable of, has not happened yet, has been the limited access to these models. Microsoft, for instance, acquired an exclusive license to GPT-3 early on and tried to secure a competitive advantage, leveraging its preexisting market power. Although access to computing resources remains a deciding factor in building large models, access to these models no longer seems to be power-law-distributed.

Over the last year, a whole ecosystem has emerged, and open-source solutions are becoming a relevant part of the tech stack building up around AI. This development helps to commoditize foundation models and make access to the technology as easy as possible. Small companies not being dependent on proprietary solutions further drives down the cost of intelligence and not only speeds up industries' adoption of large models.  

‍

The economy of OSS-Foundation-Models

One of the most exciting players in this area is Hugging-Face, an open-source platform founded in 2016 by Julien Chaumond, Clément Delangue, and Thomas Wolf. Hugging-Face allows users to build, train and deploy AI models based on openly accessible code. The Transformer library they provide comes along with many pre-trained large models, datasets and a well-designed API that allows programmers to easily interact with other common libraries such as PyTorch or Tensorflow. Hugging-Face makes it easy for every programmer to fine-tune pre-trained transformer models to their needs and apply them to text, speech, vision and tabular data. Moreover, Hugging-Face provides courses that teach you how to use their platform and detailed documentation. This development means a democratization of foundation models, which is good news for every startup aiming to build an AI-first product.

In Addition to platforms like Hugging-Face, the commoditization of foundation models is driven by collaborative efforts to provide open models. Open-GPT-X, for example, is a project to build open large language models tailored explicitly to European use cases and languages. The project is funded by the German Federal Ministry for Economic Affairs and Climate Action and aims to retain the European economy's sovereignty. The idea is that European SMEs, which do not have the resources to train their own foundation models, can share their data in open-GPT-X and collaboratively train a model using federated learning. Afterwards, the resulting models can be fine-tuned to each company's needs. 

‍

Another critical driver of open access models has been the open science project Bigscience where collaborating researchers from all over the world worked together to create Bloom, the largest open multilingual language model. These efforts further democratize the technology since it makes powerful language models accessible for countries where English is not the predominant language. With 176 billion parameters, Bloom has roughly the size of GPT-3 and can generate text in 46 natural languages and 13 programming languages. Regarding image generation, Stability AI plays a significant role in open-sourcing foundation models. With Stable Diffusion, in august 2022, they released a latent diffusion model capable of creating images from texts comparable to Dall-E-2. But, unlike Dall-E-2, Stable Diffusion is entirely open access. 

‍

A new generation of Business

Once the trend of open source and commoditization in foundation models is understood, it becomes apparent how this trend will translate into business trends. From the technical side, building an AI-first company has never been so easy. For this reason, we expect a dramatic increase in companies in 2023, leveraging the trend of open source and fine-tuning large models to build exciting products. The remaining question is, how will successful companies differentiate themselves from others if the same technologies are available to everyone? The winning AI-first companies of tomorrow will be the ones that can rely on a solid ecosystem, are successful in defining the right customer segment and can properly validate their idea before they go to market.

‍