Models New
OVHcloud announces Olala mini, a 7-billion-parameter open model trained from scratch on European public supercomputers
Published:
French cloud provider OVHcloud announced on Tuesday Olala, its first family of language models. The first, Olala mini 7A1B, has 7 billion parameters with 1 billion active, spread across 256 experts per layer, and a hybrid attention design combining Differential-TPA and Mamba-3. Its context window is 32,000 tokens.
OVHcloud says the model runs on a GPU with as little as 8 GB of memory, at 160 tokens a second on an RTX A4500, and at 50 tokens a second on a standard processor with no GPU. It was trained on three European public supercomputers, Jupiter, MeluXina and Jean Zay, with computing time the company values at 7 million euros. It will be released on Hugging Face under the Apache 2.0 licence with its weights, base model, technical report and AI Act compliance documentation; the official launch is set for the OVHcloud AI Summit on 19 November.
Search engine Qwant already uses it for its flash answers, which OVHcloud says are 20% faster at a third of the running cost. The company plans a version 1.1 with a 128,000-token context by the end of the year and says version 2.0 will exceed 100 billion parameters.
Caveat. Speed and cost figures are OVHcloud's claims; it published no benchmark scores against other models, and the weights were not yet on Hugging Face when this edition checked.
Sources