The paper investigates whether lower stereotype bias scores in multilingual language models truly indicate greater fairness. By auditing ten open models across eight languages, the authors show that lower average scores often arise from translation artifacts rather than genuine bias reduction, as item‑level correlations between English and translated scores are weak. They also identify pipeline errors that can mask bias signals and provide code to replicate their findings.
By Harsh Kumar
The study examined whether brain-language model alignment reflects shared computational mechanisms or merely stable lexical‑semantic correspondences. Using whole‑brain encoding across Mandarin, English, and French, transformer representations predicted activity in a distributed network that overlapped across languages and remained stable across layers. Contextual embeddings and measures of prediction or compression did not outperform static lexical embeddings, suggesting that alignment is robust but not informative about shared computational processes.
By Ni Yang, Rui He, Philipp Homan, Iris Sommer, Davide Staub, Wolfram Hinzen
The paper investigates whether layer-wise visual‑text similarity in multimodal large language models (MLLMs) truly reflects content‑level cross‑modal interaction. By injecting Gaussian noise into the visual stream of 13 MLLMs, the authors show that task accuracy drops sharply while traditional scalar alignment metrics (CKA, SVCCA, MIR, principal‑angle cosine) fail to distinguish corrupted from clean inputs, a phenomenon they term the alignment illusion. They propose the principal‑angle gap (PA gap) as a more reliable geometric diagnostic that correlates better with task performance and reveals when internal geometry diverges from accuracy.
By Hong-Han Wang, Yuntao Wang, Hu Ding
The study investigates whether language models tailored to specific cognitive domains better align with corresponding brain systems. By prompting and fine‑tuning large language models into six domain experts—sensory, spatial, numerical, reasoning, social, and abstract—the authors find that each expert’s representations more closely match the brain region associated with its domain than other experts. This domain‑specific alignment holds across multiple base models and fMRI datasets, while overall prediction accuracy remains largely unchanged, indicating that regional alignment can be obscured when summarizing across the brain.
By Zhivar Sourati, Mengxuan Helen Wu, Nona Ghazizadeh, Jonas Kaplan, Morteza Dehghani, Samuel A. Nastase
arXiv:2602. 12811v2 Announce Type: replace-cross Abstract: When humans and large language models (LLMs) process the same text, activations in the LLMs correlate with brain activity measured, e.
By Laurent Bonnasse-Gahot, Christophe Pallier
IndicQE-APE is a consolidated benchmark that unifies quality estimation (QE) and automatic post‑editing (APE) data for nine Indic language pairs, comprising 126,754 instances with multiple aligned labels such as direct assessment, human post‑edit, word‑level OK/BAD tags, and error explanations. The dataset includes a stratified test set across four difficulty axes and supports training and evaluation of six prompted large language models, three COMET metrics, and three APE systems. Experiments reveal that segments with conflicting holistic and token‑level quality signals are consistently ranked lower, while annotator disagreement shows no effect when controlled for score distribution.
whyItMatters":"The benchmark provides a unified resource for training and evaluating QE and APE across Indic languages, enabling consistent comparison of models and metrics on a shared dataset."
By Diptesh Kanojia, Archchana Sindhujan, Sourabh Deoghare, Daria Sokova, Shenbin Qian, Girish Koushik, Tharindu Ranasinghe, Constantin Or\u{a}san, Chrysoula Zerva, Ricardo Rei, Fr\'ed\'eric Blain, Andr\'e F. T. Martins, Marco Turchi, Matteo Negri, Anoop Kunchukuttan, Mitesh M. Khapra, Pushpak Bhattacharyya
The study investigates how a large language model (LLaMA2‑7B) aligns with human brain activity during naturalistic language comprehension by using SCRIBE, an interpretable semantic representation of 65 neurocognitively grounded properties. Variance partitioning on fMRI data from seven participants reveals that shared variance between LLaMA2‑7B and SCRIBE dominates across the brain, yet both representations also contribute unique variance. Further analysis shows that concrete semantic properties correlate more strongly with shared variance, while abstract properties do not show increased unique variance, and ablation experiments demonstrate that multiple semantic dimensions cumulatively influence both shared and unique brain-predictive variance.
By Camila Kolling, Mariya Toneva
arXiv:2608. 14896v1 Announce Type: cross Abstract: Large language models work well on English and behave in poorly understood ways on languages typologically far from it.
By Florian Braun
arXiv:2610.08159v1 Announce Type: cross
Abstract: COMET reports translation quality as a single number, and that number is routinely compared across target languages written in different scripts. Suc...
By G. L. John Salvin (Indian Institute of Technology Palakkad), Swapnil Hingmire (Indian Institute of Technology Palakkad)
arXiv:2505.12196v2 Announce Type: replace
Abstract: The impressive linguistic abilities of large language models (LLMs) have recommended them as models of human sentence processing, with some conject...
By Yi-Chien Lin, Hongao Zhu, William Schuler
arXiv:2610.32144v1 Announce Type: new
Abstract: When a language model is asked whether its own answer is correct, three languages are involved: that of the question, that requested for the answer, an...
By Jewon Yeom, Jaewon Sok, Seonghyeon Park, Jeongjae Park, Hwiyeong Lee, Taesup Kim
arXiv:2607. 20454v1 Announce Type: cross Abstract: All frontier large language models (LLMs) exhibit response drift -- producing outputs that deviate from expert-validated references -- yet the magnitude and structure of this drift remain uncharacterised by systematic human evaluation.
By Mohammed Aledhari, Ali Aledhari, Fatimah Aledhari, Gowtham Venkat Eathamokkala, Mohamed Rahouti