arXiv AI August 11, 2026 By Adrian Marius Dumitran, Theodor-Pierre Moroianu, Mihnea-Vicentiu Buca

MateInfoUB: A Real-World Benchmark for Testing LLMs in Competitive, Multilingual, and Multimodal Educational Tasks

arXiv:2507. 03162v2 Announce Type: replace-cross Abstract: The rapid advancement of Large Language Models (LLMs) has transformed various domains, particularly computer science (CS) education.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI

Aug 18

LLMs Get Smarter from Targeted Synthetic Multilingual Data

arXiv:2608. 15964v1 Announce Type: cross Abstract: Language-specific competency (LSC) is the phenomenon of a language model performing better or worse depending on the language of the prompt.

By Ishika Agarwal, Arkajyoti Charaborty, Tanner Sorensen, Neha Gupta, Andreas Stolcke

llms fine-tuning

arXiv AI

Jun 2

EuraGovExam: A Multilingual Multimodal Benchmark from Real-World Civil Service Exams

arXiv:2603. 27223v2 Announce Type: replace-cross Abstract: We present EuraGovExam, a multilingual and multimodal benchmark sourced from real-world civil service examinations across five representative Eurasian regions: South Korea, Japan, Taiwan, India, and the European Union.

By Jaeseong Kim, Chaehwan Lim, Sang Hyun Gil, Suan Lee

llms multimodal benchmarks

arXiv AI

Aug 11

MCIF: Multimodal Crosslingual Instruction-Following Benchmark from Scientific Talks

arXiv:2507. 19634v4 Announce Type: replace-cross Abstract: Recent advances in large language models have laid the foundation for multimodal LLMs (MLLMs), which unify text, speech, and vision within a single framework.

By Sara Papi, Maike Z\"ufle, Marco Gaido, Beatrice Savoldi, Danni Liu, Ioannis Douros, Luisa Bentivogli, Jan Niehues

llms nlp multimodal benchmarks

arXiv AI

Jun 9

Beyond Pass Rate: A Multilingual, Execution-Grounded Evaluation of Open Code LLMs

arXiv:2606. 08840v1 Announce Type: new Abstract: Code generation models are typically compared using compact execution benchmarks and aggregate pass rates, but such summaries obscure how performance varies across programming languages, problem families, and failure modes.

By Sayed Erfan Arefin

llms benchmarks

arXiv AI

Jul 7

Parameter Efficient Multimodal Instruction Tuning for Romanian Vision Language Models

arXiv:2512. 14926v2 Announce Type: replace-cross Abstract: Focusing on low-resource languages is an essential step toward democratizing generative AI.

By George-Andrei Dima, R\u{a}zvan-Alexandru Sm\u{a}du, Dumitru-Clementin Cercel

llms nlp fine-tuning multimodal

arXiv Machine Learning

Aug 11

Themis: Training Robust Multilingual Code Reward Models for Flexible Multi-Criteria Scoring

arXiv:2605. 00754v4 Announce Type: replace-cross Abstract: Reward models (RMs) have become an indispensable fixture of the language model (LM) post-training playbook, enabling policy alignment and test-time scaling.

By Indraneil Paul, Goran Glava\v{s}, Iryna Gurevych

llms reinforcement-learning benchmarks safety

One email a morning, machine-written

Related stories

LLMs Get Smarter from Targeted Synthetic Multilingual Data

EuraGovExam: A Multilingual Multimodal Benchmark from Real-World Civil Service Exams

MCIF: Multimodal Crosslingual Instruction-Following Benchmark from Scientific Talks

Beyond Pass Rate: A Multilingual, Execution-Grounded Evaluation of Open Code LLMs

Parameter Efficient Multimodal Instruction Tuning for Romanian Vision Language Models

Themis: Training Robust Multilingual Code Reward Models for Flexible Multi-Criteria Scoring