Silo AI releases checkpoint on mission to democratise LLMs

A year has passed since OpenAI unleashed ChatGPT on the world and popularised terms like foundational model, LLM, and GenAI. However, the promised benefits of generative AI technology are still much more likely to be derived by those who speak English, over other languages. 

There are over 7,000 languages in the world. Yet, most large language models (LLMs) work far more effectively in English. Naturally, this threatens to amplify language bias when it comes to access to knowledge, research, innovation — and competitive advantage for businesses. 

In November, Finland’s Silo AI released its multilingual open European LLM Poro 34B developed in collaboration with the University of Turku. Poro, which means reindeer in Finnish, has been trained on Europe’s most powerful supercomputer LUMI in Kajani, Finland. (Interestingly, LUMI runs on AMD architecture, as opposed to all-the-rage LLM-training Nvidia.) 

Along with Poro 1, the company unveiled a research checkpoint program that will release checkpoints as the model completes (the first three points were announced with the model last month). 

Now, the company, through its branch SiloGen, has trained more than 50% of the model and has just published the next two checkpoints in the program. With these five checkpoints now complete, Poro 34B has shown best-in-class performance for low-resource languages like Finnish (compared to Llama, Mistral, FinGPT, etc) — without compromising performance in English. 

Research Fellow Sampo Pyysalo from TurkuNLP says that they expect to have trained the model fully within the next few weeks. As the next step, the model will add support for other Nordic languages, including Swedish, Norwegian, Danish, and Icelandic. 

“It’s imperative for Europe’s digital sovereignty to have access to language models aligned with European values, culture and languages. We’re proud to see that Poro shows best-in-class performance on a low-resource language like Finnish,” Silo AI’s co-founder and CEO, Peter Sarlin, told TNW. “In line with the intent to cover all European languages, it’s a natural step to start with an extension to the Nordic languages.” 

Furthermore, SiloGen has commenced training Poro 2. Through a partnership with non-profit LAION (Large-scale Artificial Intelligence Open Network), it will add multimodality to the model.

“It’s likewise natural to extend Poro with vision,” Sarlin added. “Like textual data, we see an even larger potential for generative AI to consolidate large amounts of data of different modalities.”

LAION says it is “passionate about advancing the field of machine learning for the greater good.” In keeping with Silo AI’s intentions for building its GenAI model and LAION’s overall mission to increase access to large-scale ML models, and datasets, Poro 2 will be freely available under the Apache 2.0 Licence. This means developers will also be able to build proprietary solutions on top. 

Silo AI, which calls itself “Europe’s largest private AI lab” launched in 2017 on the idea that Europe needed an AI flagship. The company is based in Helsinki, Finland, and builds AI-driven solutions and products to enable smart devices, autonomous vehicles, industry 4.0, and smart cities. Currently, Silo AI counts over 300 employees and also has offices in Sweden, Denmark, the Netherlands, and Canada.