For all the lessons that have been learned in the rapid development of AI in recent years, the most important one is that computer intelligence is, above all, a function of its data. Today’s frontier models and their capabilities exist because a handful of labs built an enormous training-data engine to feed them, on top of an abundance of raw text freely available on the internet. Training data is now a key constraint in the field of robotics as well, the key difference being that robot training data requires tasks to be physically performed and recorded by humans. There is no corpus of training data that robotics developers can scrape yet, and because the training data is that much harder to get, it becomes that much more valuable. In this paper, we lay out why we believe the data layer is the most valuable opportunity in robotics today, and what kind of business is best-positioned to capture this opportunity.
In the seventy years computers have been around, getting one to actually perform a task was usually a task of its own. Someone had to first write out every step in an order the machine could follow, which perfectly suited work that was already built around some sort of fixed, precise procedure. People had been doing math, keeping records, and balancing accounts for centuries, but a computer could do the same sequence faster. Automation reshaped job functions and entire industries but largely remained limited to single tasks, and more importantly, everything a machine did had to be written out by someone first.
Today, machines can write coherent prose, answer specific and open-ended questions alike, and turn plain-language descriptions into code and production-ready applications. This foundational transformation came from changing what kind of instructions a machine is given and how they are given. Though tasks still start with instructions, the capability of a machine when given instructions has significantly evolved. More importantly, the instructions themselves have become far more intuitive and simple, particularly because they could be reduced from detailed specifications to simple sentences.
Importantly, transforming the capabilities of machines required transforming their inputs first. Rather than being given precise instructions on how to do a specific task, machines were given a lot of different examples of tasks being performed, and learned by observing those examples. Rather than teaching a model how to write a sentence, the model processed enough sentences that it learned the task on its own. Instead of teaching a model how to write code, the model processed enough working code to learn how to actually produce it. Instead of teaching a model how to translate two specific languages, the model processed enough translated text that it learned how to do so on its own.
To better illustrate this distinction, consider a calculator and ChatGPT. A calculator performs large and complex operations, provided those operations involve numbers and stay within its defined parameters, but is useless if asked to write an essay. A language model can be used as a calculator, and it can also write the essay, search the web, produce working code, and plenty more. The key difference is the training data. The pattern that lets a machine answer a question it has never seen can only be learned from real examples, therefore the more examples an AI model trains on, the more of the world's structure it captures, and the more it can consequently do. The same mechanism is set to take place in the physical world with robots, so it helps to first understand how it unfolded within software.
In 2020, OpenAI released GPT-3, and a few years later the world was introduced to ChatGPT, which to many was the first application that actually resembled some form of reasoning. ChatGPT answered a wide range of questions, wrote essays, translated languages, and produced working code, often from nothing more than a short prompt. These capabilities struck a chord, as ChatGPT reached one million users in five days and roughly 100 million monthly users within two months of launch, marking the fastest adoption of a consumer application recorded at the time.
Though ChatGPT initially ran on GPT-3.5, the unlock traced back to GPT-3, made possible by the unprecedented scale of its training data. GPT-3 was the third generation of the same underlying GPT model, each generation scaling the parameters of its predecessor by roughly an order of magnitude. For reference, parameters are the building blocks that support a model's core function of identifying and reproducing patterns, so more parameters means more capacity to do so. GPT-1 held around 117 million parameters, GPT-2 around 1.5 billion, GPT-3 175 billion.
Each jump in parameters sought to remove tedious parts of AI model training that were previously thought essential. GPT-1 demonstrated that language ability learned from raw text could transfer to specific tasks with far less task-specific data than starting from scratch. The model was first pre-trained on large amounts of unlabeled text to predict the next word, then fine-tuned on a smaller labeled dataset for each specific task.
Fine-tuning was the tricky part since every new task still needed its own round of fine-tuning and its own labeled examples, therefore GPT-2 tested whether fine-tuning was necessary at all. The idea was that a language model trained on enough varied text is already a multitasker that is capable of handling tasks it was never fine-tuned for, because the training text is full of them. Testing this capability meant scaling up the GPT model’s inputs, so OpenAI raised the parameter count more than tenfold, to 1.5 billion, and trained on a larger, more diverse web corpus. The largest version reached state-of-the-art results on seven of eight language benchmarks with no fine-tuning at all. Scaling the model, on its own, had removed a step that was supposed to be essential.
GPT-3 pushed the same bet even further, training 175 billion parameters on about 300 billion tokens of text (a token refers to a chunk of a word) drawn in weighted proportion from filtered Common Crawl, WebText2, two book corpora, and English Wikipedia. With such a dramatic shift in size, capability improved accordingly. GPT-3 could perform a new task from a few examples placed directly in the prompt, with no fine-tuning required.
Though the capabilities of today’s models have well outpaced GPT-3, the model remains a pivotal moment as it laid the groundwork for GPT-3.5 and ChatGPT, and kicked off an AI boom that has reshaped the industry in the years since.
Frontier labs, the companies and research organizations building the most advanced models from scratch, OpenAI and Anthropic among them, have actually put those models into production and into the hands of hundreds of millions of users worldwide. In doing so, they’ve drawn tens of billions of dollars in investment and are on pace to become some of the most valuable companies in the world. While a lot of their success is often attributed to the researchers they recruit and the vast computing power they own, none of it would be possible without the massive amount of data their models were trained on.
As GPT-3 demonstrated, building anything resembling the capabilities of a frontier AI model requires a great deal of training data. This is where it’s important to note that not all training data is the same. Raw text scraped from the internet is free and abundant, and can teach a model how language works, what words tend to follow others, how arguments are structured, which facts appear where. Models trained on that scraped corpus grew more capable as the corpus grew in size, and eventually began to generalize to tasks no one had built them for. GPT-3 translated between languages, unscrambled words, and did basic math though it was never trained for any one of those specifically. In fact, GPT-3 had actually beaten XLM, Facebook's leading cross-lingual model, on French-to-English translation, despite never being trained on paired French-English text and working from only a few examples placed in the prompt.
There is a reason models aren’t trained only on scraped text, however, as such models often produce answers that sound convincing whether or not they are correct or helpful. Improving accuracy requires human judgment about which of the model's answers are actually good. A person is shown a prompt and two answers the model produced, and either marks which one is better, reads an answer and corrects its errors, or writes from scratch the answer the model should have given. Each of these judgments becomes a training example, and the model is tuned to prefer outputs like the ones humans preferred, a process known as reinforcement learning from human feedback (RLHF). These judgments are a different and more valuable kind of training data than scraped text, and a more expensive one, since each example must be produced by a person rather than collected from the web. The harder the question, the more qualified the judge must be: a model's answer about tax law should only be graded by someone who knows tax law, so suppliers recruit doctors, lawyers, and engineers to evaluate outputs in their fields. Producing this data at frontier scale is difficult, slow, and expensive. Refining a single model involves hundreds of people over months at costs reaching tens of millions of dollars, and frontier labs spend billions annually.)