arXiv AI

I'm Sorry, but I Can't Help with Braille: Revealing Accessibility Failures in State-of-the-Art LLMs

arXiv:2607. 11893v1 Announce Type: cross Abstract: Large Language Models (LLMs) perform strongly on many language tasks, but their capability in structurally constrained, accessibility-critical modalities such as Braille remains unclear.

arXiv Machine Learning

Jul 7

The Truncation Blind Spot: How Decoding Strategies Systematically Exclude Human-Like Token Choices

arXiv:2603. 18482v2 Announce Type: replace-cross Abstract: Standard decoding strategies for text generation, including top-$k$, nucleus sampling, and contrastive search, select tokens based on likelihood, restricting outputs to high-probability regions.

By Esteban Garces Arias, Nurzhan Sapargali, Christian Heumann, Matthias A{\ss}enmacher

llms safety

arXiv AI

Jun 2

TukaBench: A Culturally Grounded Jailbreak Benchmark for African Languages

arXiv:2606. 01322v1 Announce Type: cross Abstract: Safety evaluation of Large Language Models (LLMs) remains heavily English-centric, leaving Low-Resource Languages (LRLs), particularly African ones, critically underexplored.

By Victor Akinode, Senyu Li, Wassim Hamidouche, Waqas Zamir, Inbal Becker-Reshef, David Ifeoluwa Adelani

llms benchmarks safety

arXiv Machine Learning

1d ago

Interpretable Cross-Lingual Alignment in Small Language Models: Probing Cultural and Pragmatic Reasoning in Japanese-English Bilingual LLMs

arXiv:2608. 14896v1 Announce Type: cross Abstract: Large language models work well on English and behave in poorly understood ways on languages typologically far from it.

By Florian Braun

llms benchmarks safety

arXiv Machine Learning

Aug 5

M-GATE: Multilingual Grammar, Accuracy in Translation, and Efficiency Benchmark for Large Language Models

arXiv:2608. 03803v1 Announce Type: cross Abstract: Multilingual language models are deployed across a hundred or more languages, yet most benchmarks test whether a model can perform a task _in_ a language rather than whether it commands the language itself, conflating fluency with proficiency.

By Tom\'a\v{s} Burkert, Angelika Peljak-{\L}api\'nska, David Zelen\'y

llms nlp benchmarks safety

arXiv AI

Aug 11

MCIF: Multimodal Crosslingual Instruction-Following Benchmark from Scientific Talks

arXiv:2507. 19634v4 Announce Type: replace-cross Abstract: Recent advances in large language models have laid the foundation for multimodal LLMs (MLLMs), which unify text, speech, and vision within a single framework.

By Sara Papi, Maike Z\"ufle, Marco Gaido, Beatrice Savoldi, Danni Liu, Ioannis Douros, Luisa Bentivogli, Jan Niehues

llms nlp multimodal benchmarks

arXiv Machine Learning

Aug 11

Embedding Initialization for Unseen Low-resource Languages in Multilingual NMT: A Case Study on Limbum-English Translation

arXiv:2608. 07629v1 Announce Type: cross Abstract: Multilingual neural machine translation models such as NLLB-200 cover 200 languages but leave thousands unsupported, including most Grassfields Bantu languages of Cameroon.

By Samiratu Ntohsi, Neza David Tuyishimire, Anesu Kafesu, Marvin Ogore, Samuel Oluwajunwonlo Babalola, Oche Ankeli

llms rag nlp fine-tuning

arXiv AI

Jul 21

Probing the Difficulty Perception Mechanism of Large Language Models

arXiv:2510. 05969v3 Announce Type: replace-cross Abstract: Large language models (LLMs) are increasingly deployed on complex reasoning tasks, yet little is known about their ability to internally evaluate problem difficulty, which is an essential capability for adaptive reasoning and efficient resource allocation.

By Sunbowen Lee, Qingyu Yin, Chak Tou Leong, Jialiang Zhang, Yicheng Gong, Shiwen Ni, Min Yang, Xiaoyu Shen

llms benchmarks

arXiv Machine Learning

Jul 7

TokSuite: Measuring the Impact of Tokenizer Choice on Language Model Behavior

arXiv:2512. 20757v2 Announce Type: replace-cross Abstract: Tokenizers provide the fundamental basis through which text is represented and processed by language models (LMs).

By G\"ul Sena Alt{\i}nta\c{s}, Malikeh Ehghaghi, Brian Lester, Fengyuan Liu, Wanru Zhao, Marco Ciccone, Colin Raffel

llms nlp benchmarks

arXiv AI

Aug 3

Small Is Enough: Per-User Style Rewriting of AI-Edited Text via LoRA Adapters

arXiv:2607. 29238v1 Announce Type: cross Abstract: InMyStyle is a privacy first, single user system that adapts small language models to rewrite AI-edited text towards an individual user's writing style without an instruction prompt at inference.

By Antorweep Chakravorty

llms rag fine-tuning safety

Hugging Face Trending Papers

Jul 9

Beyond wheelchairs and blindfolds: Investigating disability stereotypes in T2I models with INCLUDE-BENCH

Text-to-image (T2I) models have been shown to exhibit social biases. Prior work has mainly focused on gender, skin tone, and cultural representation within restricted occupational associations, and emerging benchmarks increasingly incorporate these dimensions.

diffusion benchmarks safety

arXiv AI

Jun 9

Multilingual Training and Evaluation Resources for Vision-Language Models

arXiv:2604. 18347v2 Announce Type: replace-cross Abstract: Vision Language Models (VLMs) achieved rapid progress in the recent years.

By Daniela Baiamonte, Elena Fano, Matteo Gabburo, Stefano Simonazzi, Leonardo Rigutini, Andrea Zugarini

llms multimodal benchmarks

arXiv AI

Jun 26

What We are Missing in Multimodal LLM Evaluation?

arXiv:2606. 26348v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) can process diverse inputs, e.

By Po-han Li, Shenghui Chen, Sandeep Chinchali, Ufuk Topcu

llms multimodal benchmarks

Related stories