Large language models

Model releases, architecture work and prompting research on large language models — from frontier-lab announcements to the arXiv papers behind them.

28,685 stories · RSS feed

arXiv AI
Oct 1

SCB: SpeechConversationBench for Evaluating Multi-Turn Reasoning in Speech-to-Speech Models

SCB: SpeechConversationBench (SCB) evaluates multi-turn reasoning in speech-to-speech models by using 103 sharded GSM8K mathematical problems. The benchmark compares three delivery modes: a full single-turn problem, a concatenated multi-shard version, and an incremental spoken disclosure across turns. Results show that sharded accuracy drops by 5.0–25.3 percentage points for four commercial systems, while the internal LEGO pipeline maintains 77.5% accuracy across all conditions, outperforming GPT‑4o Realtime’s 76.6% sharded accuracy.

By Kanpat Vesessook, Saksorn Ruangtanusak
arXiv AI
Oct 1

Linguistic Loopholes in LLM Unlearning: From a 174-Language Benchmark to Coverage-Aware Unlearning

The paper introduces the problem of cross‑lingual loopholes in large language model (LLM) unlearning, where forgetting a fact in one language can leave it accessible in others. It presents a new 174‑language benchmark, the Cross‑Lingual Unlearning Tensor, and proposes COVER, a method that selects a subset of source languages to maximize unlearning coverage under a language budget. Experiments show COVER reduces residual knowledge by 7.8–27.3% compared to uniform selection and works on both synthetic and real low‑resource news data.

By Tyler Skow, Shravan Chaudhari, Rama Chellappa, Abhay Yadav
arXiv Computation and Language
Oct 1

LoopVL: Recurrent Visual Intelligence

arXiv:2609.38426v1 Announce Type: cross Abstract: We introduce LoopVL to study whether Loop Transformers can be effectively extended to vision- language models. LoopVL combines Module-Loop and Model-...

By Zhe Qian, Ziyang Gong, Zhongxing Xu, Hehan Li, Zhonghua Wang, Fei Luo, Mingxuan Wang, Xue Yang, Shiwei liu, Yanbiao Ma, Junchi Yan, Jungong Han
arXiv AI
Oct 1

Whose Voice Survives the Summary? A Voice-Retention Audit of LLM Employee Listening

The paper introduces a Voice Retention / Representation Ratio metric to assess bias in large language model (LLM) summaries of employee feedback. Using a bilingual corpus of 2,586 responses from a global professional services firm, the study finds that criticism is reported more reliably than praise, and that LLM summaries tend to filter by popularity rather than sentiment—criticism often survives while single-mention concerns, short or German-only content are frequently omitted. The authors argue that prevalence, not sentiment, drives the bias, and provide a metric, field evidence, and a disaggregated voice‑retention card for future audits.

By Thilo Tamme, Anton Hantel, Bijan Khosrawi-Rad
arXiv AI
Oct 1

FIGS: Evaluating Multi-Turn Sycophancy Without Penalizing Empathy

The paper introduces FIGS, a dual‑axis evaluation framework for multi‑turn sycophancy that avoids penalizing empathy. It uses a 10‑turn conversational simulator with 500 diverse scenarios to test whether models stay truthful while keeping praise proportional, and whether they show calibrated validation of user feelings. The study finds that current models either drift toward sycophancy or become overly detached, highlighting an unresolved trade‑off in sustained dialogue.

By Sidharth Pulipaka, Ruta Binkyte, Ivaxi Sheth, Sahar Abdelnabi