Large language models

Model releases, architecture work and prompting research on large language models — from frontier-lab announcements to the arXiv papers behind them.

30,670 stories · RSS feed

arXiv Computation and Language
Sep 28

From annotation to reasoning: Culture in language models

The paper proposes a new way to evaluate language models on cultural understanding by focusing on interpretive depth rather than just factual recall. It argues that literary interpretation, where scholars can disagree yet still assess the quality of evidence, provides a useful framework for testing how models handle cultural references, reuse, and transformation across texts. The authors suggest linking evidence-centered benchmarks, preserving scholarly disagreement, and conducting model-development experiments on literary data, with Danish literature as a starting point for broader applications.

By Daniel Hershcovich, Alexander Conroy, Jens Bjerring-Hansen
arXiv Computation and Language
Sep 28

Training-Free Pronunciation Transcription via Text-Constrained Acoustic Rescoring

The paper introduces a training‑free speech‑and‑text‑to‑pronunciation (ST2P) pipeline that combines lexical candidates from G2P tools with acoustic rescoring using frozen pretrained S2P models. By performing a left‑to‑right greedy search over whole‑sequence negative log‑likelihoods, the method achieves a dramatic reduction in character error rate on Japanese corpora, outperforming both baseline G2P/S2P approaches and commercial multimodal LLMs. The approach is also significantly faster—3–3.5× faster than beam search and twice as fast as direct decoding—while maintaining high accuracy across multiple languages.

By Hikaru Asano, Yotaro Kubo, So Kuroki
arXiv Computation and Language
Sep 28

Evaluating Sycophancy in Chinese Large Language Models on Factual Questions Derived from Online Search Queries

The study investigates sycophancy in Chinese large language models (LLMs) by analyzing 364,941 responses from DeepSeek, Qwen, and Doubao to 12,165 yes/no factual questions derived from real-world search queries. It examines how user beliefs, reasoning, and anti-sycophancy prompts affect the distribution of correct, incorrect, and uncertain answers, finding that anti-sycophancy instructions can reduce belief-aligned errors but often increase uncertainty. The results show that preventing agreement with false beliefs does not necessarily preserve factual accuracy, underscoring the need for transition-level evaluation in Chinese-language factual QA.

By Geng Liu, Feng Li, Mengxiao Zhu, Francesco Pierri
arXiv Computation and Language
Sep 28

Modeling Student Sensemaking with LLMs and Knowledge-Graph-Guided Inference

The study explores whether instruction-tuned large language models (LLMs) can analyze collaborative student sensemaking without task-specific training, and whether adding structured knowledge-state information enhances this analysis. Two mid-size LLMs were evaluated on 23 expert-labeled episodes under various prompting conditions, showing that reasoning-enabled prompts better detect unsuccessful sensemaking and that knowledge-state diagnostics improve agreement with experts. No single configuration outperformed others across all sensemaking dimensions, highlighting the task’s multidimensional nature.

By \"Ozge Alacam, Z\"ubeyde Demet Kirbulut G\"une\c{s}, Funda Ekici, Nurcan Turan-Oluk, Dilay Din\c{c}demir, Hakk{\i} Kaday{\i}f\c{c}{\i}, Sevin\c{c} Nihal Ye\c{s}ilo\u{g}lu, Burcu I\c{s}{\i}k, Halil T\"umay, Sinem Gencer
arXiv Computation and Language
Sep 28

CG-Probes: Recovering Guardrail Directions from Patient Query Embeddings

CG-Probes are a method for extracting guardrail directions—Medical Urgency, Psychological Urgency, and Topic Sensitivity—from patient query embeddings. By treating each risk axis as a linear direction in the normalized embedding space, the approach uses difference‑in‑means probing and contrastive pairs generated from clustered Czech oncology search queries. Evaluated on 200 queries graded by oncologists, the probes perform comparably to open‑weight and frontier LLMs while offering lower latency and requiring only search logs, axis definitions, and black‑box embedding access.

By Marko \v{R}eh\'a\v{c}ek, V\'it\v{e}zslav Du\v{s}ek, Martin Rusinko, V\'it Nov\'a\v{c}ek
arXiv Computation and Language
Sep 28

RupeeBias: Auditing Demographic Bias in Indian Economic Guidance from Large Language Models

RupeeBias is a new benchmark that audits demographic bias in large language models (LLMs) when they provide economic guidance in India. It contains 39,150 prompts across four use cases—salary estimation, salary increment estimation, counter‑offer recommendation, and service pricing recommendation—varying 87 India‑specific demographic identifiers such as caste, religion, regional identity, gender, disability, and urban‑rural location. Evaluations of nine LLMs show that outputs differ by an average of 20.2% when only the demographic identifier changes, revealing systematic disparities across all six axes.

By Pavithra P M Nair, Bhavik Talaviya, Shourya Bhushan, Rahul Pankajakshan, Seema Guruvadoo, Avinash Agarwal, Gilad Gressel, Krishnashree Achuthan
arXiv Computation and Language
Sep 28

Stale-Document Poisoning: When Outdated Retrieval Overrides Correct Model Answers

The paper introduces "stale‑document poisoning," a temporal alignment failure where outdated retrieval evidence causes retrieval‑augmented generation models to produce incorrect answers even when the model could answer correctly without retrieval. A benchmark of 317 verified knowledge reversals across medicine, law, software, and platform policy shows that outdated evidence flips 30–37% of Llama and Qwen answers, rising to 66–75% when models are explicitly instructed to trust the document. The study demonstrates that providing explicit validity information and a recency‑aware re‑ranker can substantially reduce poisoning, highlighting the need for models to assess whether retrieved evidence remains applicable.

By Md Shamim Ahmed, Lukas Galke Poech, Richard R\"{o}ttger
arXiv Computation and Language
Sep 28

Strategically Diverse Sampling for Self-Training

The paper introduces two new sampling techniques—GROOT and Verbalized Sampling—to create strategically diverse training data for self-training large language models. By focusing on substantive variation in problem-solving approaches rather than just correctness, the authors demonstrate that models trained on this data outperform those trained on IID samples across competitive programming and Next‑Chapter Prediction tasks. Notably, self‑training with strategically diverse but incorrect traces from Qwen3‑4B surpasses IID distillation from a 235B teacher, challenging assumptions about the importance of correctness and teacher scale.

By Alexander Gurung, Esmeralda S. Whitammer, Mirella Lapata
arXiv Computation and Language
Sep 28

Don't CLAP: Are Music-Text Models Bag-of-Words?

The paper evaluates whether music‑text models truly capture fine‑grained musical meaning by introducing attribute‑swap perturbations that exchange properties such as timbre or order between instruments in a caption. Four contrastive models and one large audio‑language model were tested to see if they would score higher on the original caption than on the perturbed one. The results show that none of the contrastive models reliably distinguish the captions, and the audio‑language model’s advantage stems mainly from language priors, indicating that CLAP scores behave like a bag‑of‑words and fail to reflect attribute bindings.

By Yuan-Chiao Cheng, Alexander Lerch
arXiv Computation and Language
Sep 28

Symbiotic Architecture for Post-Hoc Audio Extension of Frozen Language Models

The paper introduces a symbiotic architecture that equips large language models with audio‑understanding abilities without fine‑tuning their weights. It uses an injector module to write audio‑conditioned vectors into the LLM’s key‑value cache, allowing the model to act as an audio language model while keeping the backbone unchanged. The approach improves scalability—since injection cost depends on the injector width—and preserves the LLM’s original text performance, outperforming conventional frozen‑LLM methods and approaching fine‑tuned ALM results on audio tasks.

By Yotaro Kubo, Qi Sun, Yujin Tang
arXiv Computer Vision
Sep 28

Timo: $\textbf{T}$aming Mult$\textbf{i}$modal Diffusion Transformer for Human $\textbf{Mo}$tion Generation

The paper introduces Timo, a kinematics-aware multimodal diffusion transformer designed for human motion generation. Timo employs fully shared multimodal attention, flow matching, and geometric/rotational-kinematics supervision to better coordinate articulated motion, and uses a two-stage curriculum to align motion with text captions. The authors also present a new benchmark of 40,025 clips from six datasets, showing that Timo outperforms state‑of‑the‑art methods, achieving a 40.8% relative improvement over Kimodo on average.

By Zhao Wang, Jiangtao Hu, Jack Yu, Tao Yu
arXiv Computer Vision
Sep 28

Diagnosing the Sources of Compositional Failure in Vision-Language Models: A Controlled Analysis

arXiv:2609.31456v1 Announce Type: new Abstract: Vision-language models (VLMs) often struggle with compositional reasoning tasks, but the reasons for this underperformance remain unclear. A common hyp...

By Mona Gandhi, Cenk Merih Olcay, Kuan-Chieh Lo, Santiago Castro, Christopher W. Myers, Srinivasan Parthasarathy
arXiv Computer Vision
Sep 28

GraphWrit3R: End-to-End 3D Scene Graph Writing

arXiv:2609.31595v1 Announce Type: new Abstract: 3D scene graphs provide a structured representation of complex environments by encoding objects, their semantic attributes, and the spatial and functio...

By Luka Milivojevic, Nikola Popovic, Sayan Deb Sarkar, Sebastian Koch, Iro Armeni, Luc Van Gool, Danda Pani Paudel
arXiv Computer Vision
Sep 28

PhoenixSR: Generative Heterogeneous Distillation Unleashes Efficient Models for Real-World Super-Resolution

arXiv:2609.30988v1 Announce Type: new Abstract: Real-world image super-resolution (SR) requires recovering perceptually realistic high-resolution images from complex low-resolution observations while...

By Xin Di, Mingyu Shi, Yuanfei Bao, Long Peng, Yue Zhao, Jiaming Guo, Renjing Pei, Xueyang Fu, Yang Cao, Zheng-Jun Zha