The paper introduces ${ au}$-bench-LILT, a multilingual extension of the ${ au^3}$-bench benchmark that evaluates conversational agents on realistic tool‑use tasks in German, Korean, Turkish, and Farsi across airline, retail, telecom, and banking domains. It localizes every layer of the benchmark, including culturally appropriate customer databases, native‑language user‑simulator prompts refined by contrastive annotation, and tasks tailored to realistic regional scenarios. Evaluation of eight agent models reveals the most significant performance drops in the Korean and Farsi retail domains, with reductions of up to 61 points in pass^4.
By Kaden Uhlig, Kiarash Mokhtari, An{\i}l Arslan, Yunsu Kim, Joern Wuebker
arXiv:2607. 09424v1 Announce Type: cross Abstract: We present Soofi S 30B-A3B, a sovereign, open-source Mixture-of-Experts (MoE) hybrid Mamba Transformer foundation model for German and English.
By The Soofi-Team, :, Benedikt Droste, David Fitzek, Ruben H\"arle, Lukas Helff, Maximilian Idahl, Alex Jude, Abbas Goher Khan, Maurice Kraus, Timm Ruland, Richard Rutmann, Sebastian Sztwiertnia, Markus Frey, Daniil Gurgurov, Jan Pfister, Tom R\"ohr, Sebastian von Rohrscheidt, J\"org Bienert, Nicolas Flores-Herr, Simon Gottschalk, Andreas Hotho, Kristian Kersting, Joachim K\"ohler, Alexander L\"oser, Wolfgang Nejdl, Simon Ostermann, Jan Plogsties, Patrick Putzky, Mehdi Ali, Michael Fromm, Max L\"ubbering
The paper introduces AssameseLM-E2B, a 2‑billion‑parameter language model tailored for the low‑resource Assamese language, and its instruction‑tuned variant AssameseLM-E2B‑it. Built on the Gemma‑4 architecture, it uses an expanded byte‑level BPE vocabulary that cuts tokenization overhead by nearly 500 % compared with multilingual baselines, and employs a parameter‑efficient Continual Pre‑Training pipeline with Hybrid LoRA followed by supervised fine‑tuning on a balanced, multi‑domain dataset. Evaluation shows strong performance with a Bits‑per‑Byte score of 0.7488 on FLORES‑200, a 96.7 % RAG score, and 100 % refusal accuracy on adversarial prompts, highlighting the effectiveness of parameter‑efficient adaptation for locally deployable models under constrained resources.
By Pragyan Phukan
Multi‑SFT is a new, commercially permissive dataset comprising 502,818 instruction‑response pairs in five European languages, created through translation and regeneration pipelines to address gaps where simple translation is insufficient. Human evaluation shows strong inter‑annotator agreement (≥0.87) across all languages. Fine‑tuning nine models from three families on Multi‑SFT yields performance competitive with larger, proprietary mixes and official instruction‑tuned releases, especially on safety‑related benchmarks.
By Daniela Baiamonte, Kamyar Zeinalipour, Elena Fano, Stefano Simonazzi, Andrea Zugarini
Kolibri is an English‑German mixture‑of‑experts transformer with 78.1 B total parameters and 3.46 B active parameters per token, released under the Apache 2.0 licence. It was trained on 24 T tokens, with German comprising over 20 % of the data, and includes supervised fine‑tuning and reinforcement learning on more than 1.2 M curated tasks. The model activates only 4.4 % of its parameters per token and, among open models evaluated, sits on the Pareto frontier of quality and serving cost for both English and German.
By Aleph Alpha, :, Moritz Althaus, Artur Baranowski, Michael Barlow, Fabien C. Y. Benureau, Felix Berkenkamp, Amrita Bhattacherjee, Lukas Bl\"ubaum, Duncan Blythe, Augustijn de Boer, Bastian Boll, Thomas F. Burns, Bj\"orn Deiseroth, Niko D\"urr, B\'eni Bal\'azs Egressy, Niklas Finken, Niklas Freymuth, Fabian Gebhart, Max Hammer, Bastian Harren, Andreas Hartel, Johann Higl, Steffen Hirschmann, Pablo Iyu Guerrero, Yasser Jadidi, Andreas Jagusch, Mitja Kleider, Markus Klein, Jonas Knupp, Murali Kondragunta, Garwin Lechner, Dominik L\^e-Kellner, Jurek Leonhardt, Jiaze Li, Sohir Maskey, Omar Mehio, Pit Neitemeier, Letitia Parcalabescu, Alejandro Petry Pacheco, Pranav Ragupathy, Martin Reinhardt, Peter Rodenkirch Nfissi, Dylan Rodriquez, Leonard Salewski, Raziye Sari, Jordan Sassoon, Ivo Schaper, Ilhan Scheer, Frank Schneider, Philipp Scholl, Lukas Schott, Eric Schreiber, Pablo Schumacher, Alessio Serra, Philipp Siedler, Prabhu Teja Sivaprasad, Mohamed Soliman, Till Speicher, Volker Stampa, Simon Thel, Serge Thilges, Irina Vidal Migall\'on, Denis Wa{\ss}mann, Samuel Weinbach, Sascha Wirges, Michael W\"uthrich, Alexei Zacharov
We evaluate large language models (LLMs) as language agents playing goal-directed dialogue games in self-play across 30 languages: the 24 official EU languages plus six others. Unlike static or preference-based evaluation, this paradigm is multi-turn, reference-free and programmatically scored, and because the game mechanics are language-agnostic it extends to a new language by localising a fixed set of prompt and word-list files.
arXiv:2601.02186v2 Announce Type: replace
Abstract: Despite continuous advances in medical technology, the global distribution of health care resources remains uneven. The development of large langua...
By Rui Yang, Huitao Li, Weihao Xuan, Heli Qi, Xin Li, Kunyu Yu, Yingjian Chen, Rongrong Wang, Jacques Behmoaras, Tianxi Cai, Bibhas Chakraborty, Qingyu Chen, Lionel Tim-Ee Cheng, Marie-Louise Damwanza, Chido Dzinotyiwei, Aosong Feng, Chuan Hong, Yusuke Iwasawa, Yuhe Ke, Linah Kitala, Taehoon Ko, Jisan Lee, Irene Li, Jonathan Chong Kai Liew, Hongfang Liu, Lian Leng Low, Edison Marrese-Taylor, Yutaka Matsuo, Isheanesu Misi, Yilin Ning, Jasmine Chiat Ling Ong, Marcus Eng Hock Ong, Enrico Petretto, Hossein Rouhizadeh, Abiram Sandralegar, Oren Schreier, Iain Beehuat Tan, Patrick Tan, Daniel Shu Wei Ting, Junjue Wang, Chunhua Weng, Matthew Yu Heng Wong, Fang Wu, Yunze Xiao, Xuhai Xu, Qingcheng Zeng, Zhuo Zheng, Yifan Peng, Douglas Teodoro, Nan Liu
arXiv:2606. 03304v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly evaluated in multilingual settings, yet their inference behavior in low-resource African languages remains underexplored especially under pure prompting without fine-tuning.
By Anuj Tiwari, Terry Oko-odion, Hannah Nwokocha
Cohere, OpenAI, and AI21 Labs have developed a preliminary set of best practices applicable to any organization developing or deploying large language models.
The paper reports a submission to the WMT 2026 Multilingual Instruction Shared Task, where the authors fine‑tune Qwen3‑8B with LoRA on organiser‑provided data, EuroAlpaca, and CrossEuroAlpaca. Their system focuses on open‑ended question answering in ten target languages and shows a 7.8 % relative gain in Gemma‑4 judge score for target‑language QA‑OEG, as well as 41.5 % macro‑averaged ROUGE‑L and 2.9 % BERTScore‑F1 improvements on the Aya Evaluation Suite. However, the gains are task‑ and language‑specific, with some declines in context‑grounded QA and non‑target languages.
By Aleix Sant, Jordi Luque, Carlos Escolano
Large language models (LLMs) are increasingly evaluated in multilingual settings, yet their inference behavior in low-resource African languages remains underexplored especially under pure prompting without fine-tuning. We present a systematic study of prompting strategies for Natural Language Inference (NLI) in Swahili, Yoruba, and Hausa using the AfriXNLI benchmark.
arXiv:2510. 07074v2 Announce Type: replace-cross Abstract: Instruction tuning has become a key technique for enhancing the performance of large language models, enabling them to better follow human prompts.
By Fred Philippy, Laura Bernardy, Siwen Guo, Jacques Klein, Tegawend\'e F. Bissyand\'e