arXiv AI

Multilingual Sentence Embeddings for Linguistic-Integrated Reliability Audit

arXiv:2607. 17466v1 Announce Type: cross Abstract: Multilingual assessment systems commonly rely on translation for scoring and quality-control processes.

arXiv AI

Aug 11

Do Evaluation Metrics Detect Errors in Classical Chinese to English Translations?

arXiv:2608. 08283v1 Announce Type: cross Abstract: Although large language models can translate some historical languages surprisingly well, their usefulness in digital humanities workflows is limited by the lack of reliable evaluation.

By Osvaldo Quinjica, Eric Bennett, Xinchen Yang, Andrew Schonebaum, Marine Carpuat

llms

arXiv Machine Learning

Jul 14

A Durability and Cross-Language Transfer Benchmark for a Validated Teaching-Feedback Classification Protocol

arXiv:2607. 11873v1 Announce Type: cross Abstract: Institutions collect far more open-ended teaching-evaluation feedback than they read.

By Esteban U. Vega Barajas

llms rag benchmarks

arXiv Machine Learning

Jul 22

Translation as Augmentation: Effect of Translated Data on Assessment of Difficulty

arXiv:2607. 19101v1 Announce Type: cross Abstract: Reliable Text Difficulty Assessment is a prerequisite for valid text simplification workflows and personalized learning applications.

By Yiheng Wu, Jue Hou, Roman Yangarber

nlp

arXiv AI

Jun 30

The Heterogeneous Safety Impacts of Benign Multilingual Fine-Tuning

arXiv:2606. 28843v1 Announce Type: cross Abstract: Fine-tuning a large language model is a ubiquitous method for enhancing its capability on a specific downstream task.

By Will Hawkins, Kaivalya Rawal, Jonathan Rystr{\o}m, Stratis Tsirtsis, Zihao Fu, Greta Warren, Ryan Brown, Eoin Delaney, Sandra Wachter, Brent Mittelstadt, Chris Russell

llms fine-tuning safety

arXiv Machine Learning

Jun 25

Shared Doubt: Zero-Shot Cross-Lingual Confidence Estimation for Language Models

arXiv:2605. 31220v2 Announce Type: replace-cross Abstract: Confidence estimation (CE), i.

By Athina Kyriakou, Dennis Ulmer, Ivan Titov

llms nlp

arXiv Machine Learning

2d ago

Language-Specific Gaps in AI Safety Training Datasets

arXiv:2608. 13695v1 Announce Type: cross Abstract: Large language model providers routinely cite multilingual safety benchmarks spanning a dozen or more languages as evidence that their models are safe for non-English-speaking users.

By Chialuka Prisca-Mary Onuoha, Bright Etornam Sunu, Rashidat Sikiru

llms benchmarks safety

arXiv AI

Jun 9

Multilingual Training and Evaluation Resources for Vision-Language Models

arXiv:2604. 18347v2 Announce Type: replace-cross Abstract: Vision Language Models (VLMs) achieved rapid progress in the recent years.

By Daniela Baiamonte, Elena Fano, Matteo Gabburo, Stefano Simonazzi, Leonardo Rigutini, Andrea Zugarini

llms multimodal benchmarks

arXiv AI

Aug 7

Task-Conditional Flow Matching for Balanced Multilingual Text Embedding Adaptation

arXiv:2608. 05785v1 Announce Type: cross Abstract: Multilingual text embedding models are commonly adapted using a single training objective across diverse tasks, despite different tasks requiring fundamentally different optimization strategies.

By Tirth Bhatt, Naren Kumar S, Mayank Singh

rag diffusion benchmarks

arXiv AI

Jul 23

On the Systematic Challenges of Culturally Loaded Machine Translation: Dream of the Red Chamber as the Cultural Lens

arXiv:2607. 20241v1 Announce Type: cross Abstract: Culturally loaded translation poses unique challenges for machine translation (MT), as meanings are deeply embedded in socio-cultural contexts beyond surface linguistic forms.

By Yiming Wang, Jiayuan Di

llms nlp

arXiv Machine Learning

Aug 5

M-GATE: Multilingual Grammar, Accuracy in Translation, and Efficiency Benchmark for Large Language Models

arXiv:2608. 03803v1 Announce Type: cross Abstract: Multilingual language models are deployed across a hundred or more languages, yet most benchmarks test whether a model can perform a task _in_ a language rather than whether it commands the language itself, conflating fluency with proficiency.

By Tom\'a\v{s} Burkert, Angelika Peljak-{\L}api\'nska, David Zelen\'y

llms nlp benchmarks safety

Hugging Face Trending Papers

Jul 13

A Durability and Cross-Language Transfer Benchmark for a Validated Teaching-Feedback Classification Protocol

Institutions collect far more open-ended teaching-evaluation feedback than they read. A prior study introduced a validated protocol for classifying such comments by thematic category and sentiment, built from a documented annotation guide, an intra-annotator reliability measurement, stratified cross-validation, and a held-out evaluation on a Spanish institutional corpus with a frozen-encoder design.

llms rag benchmarks

arXiv AI

Jun 26

SamaVaani: Auditing and Debiasing Multilingual Clinical ASR for Indian Languages

arXiv:2606. 26901v1 Announce Type: cross Abstract: Automatic Speech Recognition (ASR) is increasingly used to document clinical encounters, yet its reliability in multilingual and demographically diverse Indian healthcare context remains largely unknown.

By Subham Kumar, Prakrithi Shivaprakash, Abhishek Manoharan, Astut Kurariya, Diptadhi Mukherjee, Prabhat Chand, Pratima Murthy, Koustav Rudra, Lekhansh Shukla, Animesh Mukherjee

llms fine-tuning multimodal benchmarks safety

Related stories