Large language models

Model releases, architecture work and prompting research on large language models — from frontier-lab announcements to the arXiv papers behind them.

26,848 stories · RSS feed

arXiv AI
Oct 5

Sentry: Learning to Recover from LLM Agent Failures at Test Time

Sentry is a failure‑management layer for large language model agents that learns from failures at test time. It retrieves relevant lessons from an external playbook when a failure occurs, verifies recovery without task rewards, and stores new lessons only if recovery succeeds, keeping the playbook out of the agent’s context. Across multiple benchmarks, Sentry outperforms both runtime‑intervention and context‑evolution baselines, and its lessons transfer to unseen tasks.

By Changxiu Ji, Amy Lu, Qizheng Zhang, Kunle Olukotun
arXiv AI
Oct 5

FrugalEvo: Towards Cost-Aware LLM-Guided Program Evolution

FrugalEvo introduces a cost‑aware evolutionary framework that pairs a high‑cost, powerful LLM with a cheaper LLM to explore and refine program solutions, respectively. It incorporates a cache‑efficient evolution process that maximizes prefix reuse across steps and defines a Budget‑Aware Area Under the Curve (BA‑AUC) metric to evaluate solution quality within a fixed cost budget. Across ten mathematical and systems optimization tasks, FrugalEvo matches or surpasses state‑of‑the‑art baselines in final solution quality and achieves higher BA‑AUC on nine tasks, while delivering new state‑of‑the‑art performance on circle packing for costs as low as $0.55 to $1.68.

By Hui Chen, Xuan Qi, James Xu Zhao, Zhaopeng Feng, Shilong Liu, Kuang Xu, Pang Wei Koh, Bryan Hooi
arXiv AI
Oct 5

Counterfactual Evidence Audits Predict LLM-Agent Susceptibility to Ranked Context

The paper introduces a counterfactual evidence audit to assess how large language model agents respond to different sets of documents. By exposing agents to two mirrored sets of five documents and measuring changes in six downstream decisions, the authors predict how the agents will behave when presented with a full 45‑document context. Across multiple open‑weight model families and tasks, the five‑document audit accurately forecasts full‑context effects, reducing prediction error by 62% and correctly identifying the direction of most material effects.

By Rana Muhammad Usman
arXiv AI
Oct 5

Sensory-Aware Sequential Recommendation via Review-Distilled Representations

The paper introduces ASER, a pipeline that fine‑tunes a large language model to extract sensory attribute‑value pairs (e.g., color: matte black, scent: vanilla) from product reviews and distills them into a compact sensory bank for each catalog item. During recommendation, a frozen backbone uses a lightweight relational metric over this bank, achieving consistent improvements in HR@10 and NDCG@10 across five Amazon domains and four backbones, with evidence‑grounded extractions validated in 94.8% of cases. The study demonstrates that structured, evidence‑based sensory features enhance ranking quality beyond generic non‑sensory controls.

By Yeo Chan Yoon, Chanjun Park, Kyuhan Koh
arXiv AI
Oct 5

How Do AI Agents Spend Your Money? Analyzing and Predicting Token Consumption in Agentic Coding Tasks

The paper presents a systematic study of token consumption in AI agents performing coding tasks. It finds that agentic tasks are far more expensive—about 1000 times more tokens than code reasoning or chat—primarily due to input tokens, and that token usage varies wildly, with accuracy peaking at moderate costs. Models differ significantly in efficiency, and current frontier models cannot reliably predict their own token usage, often underestimating it.

By Longju Bai, Zhemin Huang, Xingyao Wang, Jiao Sun, Rada Mihalcea, Erik Brynjolfsson, Alex Pentland, Jiaxin Pei
arXiv AI
Oct 5

Where Do Apparent LLM Clinical Triage Failures Arise? Localizing the Multiple-Choice Format Effect

The study investigates why large language models (LLMs) show different triage performance when answering clinician‑authored vignettes in multiple‑choice versus free‑text formats. Using sparse‑autoencoder features on Gemma 3 and Qwen3 models, the authors find that medical information is encoded similarly in both formats, but at the decision token the multiple‑choice scaffold dominates, with over 91% of attribution coming from scaffold‑peaking features. The effect varies by model, and shuffling option order eliminates simple positional bias, suggesting the format influence is tied to answer selection rather than earlier case processing.

By David Fraile Navarro, Berardino Como, Jialei Sheng, Soundariya Ananthan, Shlomo Berkovsky
arXiv AI
Oct 5

Sentence-Level Context Sensitivity as a Training-Free Detector of Unsupported Content, Evaluated Against Trained Verifiers

The paper proposes a training‑free detector that uses sentence‑level context sensitivity to identify unsupported content in retrieval‑augmented generation (RAG) answers. By re‑scoring each sentence with full context, no context, and each chunk removed, the method flags the chunk whose removal most reduces a sentence’s likelihood as the likely source. Evaluated on RAGTruth, TofuEval, and RAGBench, the detector outperforms answer‑level faithfulness scores, achieving AUCs up to 0.745 and matching per‑chunk fact‑checkers while using only a fraction of the compute required by large‑language‑model judges.

By Mohamed Aly Bouke
arXiv Machine Learning
Oct 5

Latent-MOPD: Latent Multi-Teacher On-Policy Distillation

Latent-MOPD is a new on‑policy distillation method that allows a single large language model student to learn from multiple specialist teachers by using both the teachers’ output distributions and their hidden state representations. The approach selects late‑layer targets based on teacher‑student relationships, bridges hidden width differences with a shared projection, and groups updates by domain, enabling gradual shift from representation to token supervision. Experiments show that Latent‑MOPD outperforms token‑only, representation‑only, and uniform‑averaging baselines across nine benchmarks in math, code, and logic, and even surpasses the best individual teacher on most tasks.

By Zhengyu Fang, Seoyeon Hong, Jie Yang, Muyang Li, Koyoshi Shindo, Brandon Joseph Lwowski, Jing Li
arXiv Machine Learning
Oct 5

RAOA: Alternating-Operator Neural Computation with Programmable Radio Propagation

The paper introduces the Radio Alternating Operator Ansatz (RAOA), a recurrent computing architecture that uses programmable radio propagation as a computational depth. RAOA alternates energy‑derived problem updates with mixing updates over a persistent latent state, enabling repeated compositional passes even with the same learned controls. Experiments on discrete optimization, simulation of programmable propagation, and adaptation of pretrained language models show that RAOA can improve solution quality, handle higher‑order interactions, and adapt models with performance close to shallow MLPs, all within a single recurrent framework.

By Toshiaki Koike-Akino
arXiv Machine Learning
Oct 5

Conditional Capacity and Routing in Mixture-of-Experts Particle Transformers

The paper investigates how Mixture-of-Experts (MoE) Particle Transformers perform on the 188-class JetClass-II jet classification task. By varying expert count, routing capacity, top‑K, and auxiliary loss, the authors find that top‑1 MoE models can surpass dense baselines with similar nominal compute, but adding more experts yields diminishing accuracy gains. Activating multiple experts per token improves predictions at higher computational cost, and routing analyses reveal that expert assignments correlate with particle identity and kinematics, though this correlation does not consistently predict performance.

By Kaushik Pendiyala, Haris Zia, Trevin Lee, Timothy Legge, Alejandro J. De Leon, Zihan Zhao, Aaron Wang, Abhijith Gandrakota, Jennifer Ngadiuba, Richard Cavanaugh, Javier Duarte
arXiv Machine Learning
Oct 5

How Far Does a Shared Linear Map Go? Probing Feature-Space Manipulability for Image Editing

The paper investigates how well a simple spatially shared linear map can predict changes in a model’s internal feature space caused by various image manipulations, including geometric, photometric, occlusion, and diffusion-generated semantic edits. Experiments across ConvNeXt, SwinV2, and DINOv3 show that this linear operator often performs nearly as well as more complex, higher‑capacity probes, especially in deeper layers of supervised backbones. The study concludes that a shared linear map is frequently sufficient to capture diverse image edits, with its leading singular components encoding semantic content and higher‑rank components refining details.

By Elias Krey, Nils Neukirch, Nils Strodthoff
arXiv Machine Learning
Oct 5

Power-SMC: Low-Latency Sequence-Level Power Sampling for Training-Free LLM Reasoning

Power‑SMC is a training‑free sampling method that targets the sequence‑level power distribution used to sharpen large language model outputs for reasoning tasks. It maintains multiple candidate sequences, assigns importance weights based on how well each matches the power distribution, and periodically prunes low‑scoring candidates to keep inference latency close to standard decoding. Experiments on MATH500, GSM8K, GPQA, and HumanEval show that Power‑SMC matches or surpasses Metropolis–Hastings sampling in accuracy, preserves output diversity, and can accelerate inference speed by up to 17.6×.

By Seyedarmin Azizi, Erfan Baghaei Potraghloo, Minoo Ahmadi, Souvik Kundu, Massoud Pedram
arXiv Machine Learning
Oct 5

Min-Cost Flow Routing for Evidence Assembly in Long Multimodal Documents

The paper introduces <flowreader>, a method that casts evidence selection for long multimodal documents as a minimum‑cost flow problem over a multimodal content graph. It uses spectral decomposition to identify latent query‑relevant aspects and allocates a fixed evidence budget proportionally to their spectral energy, ensuring aspect coverage without a language‑model planning call. On the VisDoMBench benchmark with Qwen3‑VL‑32B, <flowreader> achieves the highest macro accuracy (68.9%) and outperforms prior systems by 2.7 points, while using fewer content nodes and reader tokens.

By Ambuj Mehrish, Sebastiano Vascon
arXiv Computation and Language
Oct 5

Finding the Move Is Not Winning the Game: XiangqiBench for Closed-Loop Evaluation of LLM Agents

The paper introduces XiangqiBench, an executable benchmark for evaluating large language model agents in Chinese chess. It tracks 8,568 multi‑turn trajectories from 12 frontier LLMs, revealing that metrics such as the Conversion Gap, Consistency Gap, and Simulation Gap overstate true closed‑loop success. The study shows that simply naming the correct move is insufficient; agents must reliably carry a plan through to a verified outcome against an opponent.

By Yekun Chai, Qiwei Peng, Haoyi Xiong
arXiv Computation and Language
Oct 5

Emergent Structure in the Marginal Attention Space of Language Models

The paper investigates how post‑softmax attention weights behave across different language models by marginalizing over query positions to create a joint token‑head space. Across more than 60 diverse LLMs, token‑wise marginal attention reveals a text‑intrinsic signal that is conserved across models, while head‑wise marginal attention yields a model‑private signature that is consistent across documents. This insight enables a practical method for estimating per‑head budgets for key‑value cache eviction, achieving competitive performance on standard benchmarks without per‑document recomputation or training.

By Valentino Maiorca, Walter Nelson, Francesco Locatello
arXiv Computation and Language
Oct 5

Collective Bias Mitigation via Model Routing and Collaboration

The paper introduces Collective Bias Mitigation (CBM), a framework that reduces bias in large language models by learning fine‑grained model behavior and enabling knowledge sharing among diverse LLMs. CBM systematically explores how to select and organize distinct models to produce fairer responses, outperforming standalone baselines. Experiments demonstrate significant bias reduction, with the Committee topology balancing effectiveness and inference cost.

By Mingzhe Du, Luu Anh Tuan, Xiaobao Wu, Yichong Huang, Yue Liu, Dong Huang, Huijun Liu, Bin Ji, Jie M. Zhang, See-Kiong Ng
arXiv Computation and Language
Oct 5

StanceEval 2026: The Second Stance Detection Shared Task

StanceEval 2026 is the second edition of a shared task on stance detection in Arabic social media, where systems must classify a tweet’s stance toward a target as Favor, Against, or None. The event featured two tracks: Track 1 tests cross‑target transfer on thematically related topics (Women Driving vs. Women Empowerment), while Track 2 evaluates cross‑domain transfer to entirely unseen targets (E‑Cars and Trimester System). With 80 registered teams and 30 submissions, top systems achieved $F_{avg2}$ scores of 0.8994 (Track 1) and 0.9400 (Track 2), surpassing baseline performance and highlighting challenges such as target polarization and dialectal nuance.

By Rasha Albalawi, Nuha Albadi, Hamzah Luqman, Asma Yamani, Maram Kurdi, Saad Ezzini, Ahmed Ashraf, Maged Al-Shaibani, Nora Alturayeif
arXiv Computation and Language
Oct 5

Source Preference in the Wild: How LLM Agents Favor Items by Source, and How to Reduce It

The paper investigates how large language model (LLM) agents exhibit source preference when selecting items in end‑to‑end search tasks across three domains. Across 12 agent models, each consistently favors certain sources and avoids others, even when items from disfavored sources better meet the user’s requirements. The study shows that revealing or hiding source identifiers influences selection, and that training rewards or missing information can create or mitigate these preferences.

By Jonghyun Song, Haewon Park, Jeonghoon Shim, Woojung Song, Yohan Jo
arXiv Computation and Language
Oct 5

Single-Pass Uncertainty Heads for Claim-Level Hallucination Detection in Persian Medical Language Models

The paper introduces a single‑pass uncertainty head for detecting hallucinations in Persian medical language models. By adapting the LLM Uncertainty Head framework to Aya‑Expanse‑8B and two Gaokerena backbones, the authors build claim‑level hallucination datasets and train lightweight heads that achieve PR‑AUCs of 0.4820 and 0.4652 and ROC‑AUCs of 0.7852 and 0.7810. These heads operate without retrieval or repeated sampling during inference, offering a cost‑effective approach to hallucination detection in Persian medical LLMs.

By Mehrdad Ghassabi, Pedram Rostami, Hamidreza Baradaran Kashani, Sadra Hakim, Audrina Ebrahimi