Procedia Computer Science · 2025 · 15 citations · 21 references
ComputingComputer ScienceLinguisticsComputational LinguisticsCorpus-based Translation StudiesNatural Language ProcessingLanguage ModelingMachine TranslationMultilingual Natural Language ProcessingMultilingual ProcessingMultilingualismTranslation TechnologyAutomated Data CollectionCross-lingual Language ModelingFilteringMultilingual Machine TranslationStatistical Machine Translation
The quality of open-weight LLMs has seen significant improvement, yet they remain predominantly focused on English. In this paper, we introduce the EuroLLM project, aimed at developing a suite of open-weight multilingual LLMs capable of understanding and generating text in all official European Union languages, as well as several additional relevant languages. We outline the progress made to date, detailing our data collection and filtering process, the development of scaling laws, the creation of our multilingual tokenizer, and the data mix and modeling configurations. Additionally, we release our initial models: EuroLLM-1.7B and EuroLLM-1.7B-Instruct1 and report their performance on multilingual general benchmarks and machine translation.
21
DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · 2023 · 76.3K citations · Full text
Llama 2: Open Foundation and Fine-Tuned Chat Models
Hugo Touvron, Louis Martin, Kevin H. Stone et al. · arXiv (Cornell University) · 2023 · 2.6K citations · Full text
Scaling Laws for Neural Language Models
Jared Kaplan, Sam McCandlish, Tom Henighan et al. · arXiv (Cornell University) · 2020 · 1.5K citations · Full text
RoFormer: Enhanced transformer with Rotary Position Embedding
Jianlin Su, Murtadha Ahmed, Yu Lu et al. · Neurocomputing · 2023 · 1.4K citations
Findings of the 2017 Conference on Machine Translation (WMT17)
Ondřej Bojar, Christian Federmann, Yvette Graham et al. · 2017 · 418 citations · Full text
Computing, Computer Science, Natural Language Processing +14