Table of Contents
Open data is becoming a fault line in AI development
AI2 has released Dolma, a dataset made up of over 8 million English language documents, to the public domain. Researchers can use it to train language models, study the data itself, and explore potential applications. That basic description may sound routine in a field full of model launches and benchmark claims, but the release addresses a much more consequential question: who gets to inspect the raw material behind modern AI systems?
Large language models are often discussed as if the model is the whole product. It is not. The dataset used to train it shapes what it can recognize, repeat, ignore, or get wrong. Decisions about source material, filtering, language coverage, personal information, and document quality can have as much influence on a system’s behavior as the architecture that sits on top of it. Yet those decisions are frequently difficult for outsiders to examine.
AI2 says it hopes Dolma’s availability will encourage more transparent and responsible practices in building and using AI. That ambition matters because openness in AI has often stopped short of the training data. A company may release a model, or some details about its design, while leaving the collection and preparation of the underlying text largely opaque. Researchers can test the final system, but they cannot always determine why it behaves the way it does.
Why training data deserves the same scrutiny as the model
AI2 researchers argue that the dataset used to create a model should be as free to use and modify as the model itself. That is a direct challenge to a common version of “open” AI, where access to a model does not necessarily mean access to the material used to train it.
Luca Soldaini of AI2 explains the choice of sources and the processes used to render the dataset suitable for AI consumption in a blog post. AI2 also says that a more comprehensive paper is in the works. The distinction is important: publishing data without explaining how it was selected, cleaned, or limited only solves part of the problem. A useful open dataset needs provenance and documentation, not merely a download link.
Dolma discloses its sources and processes, including how and why it was limited to original English language texts. That limitation inevitably defines the dataset’s scope. A resource focused on English can be valuable for researchers working on English-language systems, but it cannot stand in for the full range of human language, publishing, or cultural context. Being explicit about such boundaries is better than implying universality where none exists.
AI2 is also optimizing a large open language dataset model specifically for science. The phrasing points to another practical value of an openly described corpus: researchers can assess whether a dataset suits a particular domain rather than treating training data as a black box. Scientific applications have their own demands around accuracy, terminology, and traceability. Those demands are difficult to evaluate when the origins and treatment of source text are hidden.
The closed-data problem is no longer an academic concern
OpenAI and Meta, among other companies, release some crucial information about the datasets they use to construct their open language dataset models. Much of that data, however, remains private. Companies have understandable reasons to protect the secrets of their training processes in a highly competitive AI landscape. Data collection and processing can be a source of commercial advantage, and companies do not generally volunteer the details that competitors might exploit.
That business logic has a cost. When training data remains closed, researchers outside those companies have less ability to study, reproduce, challenge, or improve the datasets and models built from them. Independent evaluation becomes narrower. It is harder to identify the origins of a pattern in a model’s output, harder to test whether a filtering decision created a blind spot, and harder to distinguish a technical limitation from a choice made during data preparation.
The secrecy also fuels concerns about whether material was obtained ethically or lawfully. Critics have raised the possibility that datasets may include pirated copies of numerous authors’ books. In this debate, transparency is not simply a preference for academic neatness. It is closely tied to questions of consent, copyright, compensation, and accountability.
Authors throughout the world have signed a letter calling on AI creators to stop taking books without permission. They emphasize the need to obtain proper authorization before using copyrighted material. Their demands also extend beyond books: they want more transparency around the criteria used to determine text quality and data accuracy, and they insist that personal data be removed from datasets.
Those concerns expose the difficult balance at the center of AI development. A dataset can be technically useful while still raising questions about who contributed to it, whether they had a meaningful choice, and what recourse they have once their work or information has entered a training pipeline. The industry cannot settle those questions merely by saying that large-scale data collection is normal practice.
Openness does not mean absence of rules
Dolma is not presented as an unrestricted free-for-all. Prospective users must submit their contact information and intended use cases. They must disclose any Dolma-derivative creations and distribute those derivatives under the same license. They also must agree not to apply Dolma to prohibited areas, including surveillance or disinformation.
Those conditions reflect a tension that open AI projects increasingly have to confront. Broad access can allow more researchers to verify claims, build tools, and test alternatives to closed systems. The same access can also create opportunities for uses that a dataset’s creators consider harmful. AI2’s approach is to pair access with obligations and stated limits, rather than treating availability as the end of the ethical discussion.
There is also a route for people who believe their personal data may have inadvertently appeared in the database. They can submit a removal request form on AI2’s website. Such a mechanism does not eliminate the challenge of personal data in large datasets, but it establishes a visible channel for people who want to raise concerns. Closed datasets often leave outsiders unsure whether their information was included at all, let alone where to direct a request.
Dolma will not resolve the industry’s arguments over training data on its own. Its significance lies in offering a different standard: sources and processes should be available for examination, limitations should be stated plainly, and access should come with responsibilities. For researchers who want to inspect the material behind a language model rather than only its outputs, that is a meaningful shift.
You can access Dolma through Hugging Face if you wish.
