AI2 open sources text-generating AI models — and the data used to train them

AI2, the research institute founded by Microsoft co-founder Paul Allen, is unveiling new GenAI language models designed to be more open and accessible for developers.

The models, named OLMo (Open Language Models), along with the Dolma dataset used to train them, aim to promote the study of text-generating AI in an open environment.

According to AI2 senior software engineer Dirk Groeneveld, the OLMo framework provides researchers and practitioners with the opportunity to analyze models trained on one of the largest public datasets to date.

Unlike many other text-generating models, OLMo models are truly open, as they come with the code used to produce their training data, along with training and evaluation metrics and logs.

The most capable OLMo model, OLMo 7B, offers a compelling alternative to other models like Meta's Llama 2, depending on the application, and performs well on certain benchmarks.

However, OLMo models have limitations, such as producing low-quality outputs in languages other than English and weak code-generating capabilities.

Despite potential concerns about malicious use, Groeneveld believes the benefits of open models outweigh the risks, as they facilitate research into potential dangers and promote technical advancements for more ethical models.

AI2 plans to release larger and more capable OLMo models in the future, including multimodal models, along with additional datasets, all freely available on GitHub and Hugging Face.