Large language models

Model releases, architecture work and prompting research on large language models — from frontier-lab announcements to the arXiv papers behind them.

30,036 stories · RSS feed

arXiv AI
Sep 28

Highlight-Then-Summarize: Learning to Compress Evidence for Long-Context Understanding

The paper introduces Highlight-Then-Summarize (H2S), a two-step approach that first highlights question-relevant evidence in long documents and then condenses it into a compact, question-conditioned summary before generating an answer. The authors built the H2S-Dataset with 6,647 examples spanning 11 benchmark families, and developed H2S-RL to reward evidence selection and summary construction. Evaluated on the H2S-Bench suite, the H2S-14B model outperforms larger open-source models, achieving the highest overall score and maintaining strong performance even with a reduced output budget.

By Zhaoyuan Xia (Peking University, Baidu Inc), Qinghongbing Xie (Tsinghua University), Yung Xiang Hue (Tsinghua University), Jianguang Jiang (Baidu Inc), Gaofeng Lu (Baidu Inc), Zhenyu Jiao (Baidu Inc), Xing Yuan (Baidu Inc), Dai Dai (Baidu Inc), Tong Mo (Peking University), Long Zeng (Tsinghua University)
arXiv AI
Sep 28

Intent2Tc: Automated Intent-to-Traffic Control Translation with Language Models

Intent2Tc is a closed‑loop, language‑model‑driven framework that translates high‑level business traffic‑shaping intents into executable Linux traffic‑control (tc) configurations. It uses an AQM‑based digital twin semantic model, automated metadata extraction, critique‑driven refinement, and Retrieval‑Augmented Generation to improve semantic consistency and configuration reliability. Evaluation on 100 RFC 9315‑compliant intents shows high semantic fidelity and deployment readiness, with Claude Sonnet‑4.6 achieving 0.98 semantic similarity and 0.045 normalized edit distance, while RAG reduces token consumption and latency for compact models.

By Andrea Masini, Sudipta Acharya, Paolo Bellavista, Luca Foschini, Burak Kantarci
arXiv AI
Sep 28

Sorry Robot, Happy Human: Vision-Language Models Read Only One of Two Legible Typographic Layers

The paper "Sorry Robot, Happy Human: Vision-Language Models Read Only One of Two Legible Typographic Layers" reports that vision‑language models (VLMs) struggle to read images containing two overlapping text layers—one with sharp contour lines and one with soft shading. Using the DecoyBench dataset of 300 such images, the authors evaluated six closed‑source VLMs under naive and guided prompting at high and low resolutions. While humans could read both layers accurately, the models reliably read only the contour layer at high resolution and failed to extract the shading layer; at low resolution, neither the models nor humans could read the contour layer, but the shading layer remained readable. "whyItMatters":"The study highlights a consistent limitation of current VLMs in handling typographic structures with multiple spatial frequency layers, underscoring their vulnerability to typographic attacks and the need for more robust text‑recognition capabilities."

By Mert \.Incidelen, Yamen Kashkash, Asya Berker, Murat Aydo\u{g}an
arXiv AI
Sep 28

Towards Mitigating Fabricated Consensus: The Active Provenance Gate for Multi-Agent Debate Synthesis

The paper introduces the Active Provenance Gate (APG), a post‑debate verification layer for multi‑agent debate synthesis that audits debate logs, applies self‑correction, and blocks unsupported claims before publication. Empirical studies show that APG more than doubles provenance fidelity in crisis simulations and that users prefer explicit failure reports over fabricated consensus. The work shifts data origin tracing from passive logging to active conditional blocking, addressing safety gaps in large‑language‑model‑based debate systems.

By Jakub Mas{\l}owski, Jaros{\l}aw A. Chudziak
arXiv AI
Sep 28

PriceBench: A Diagnostic Benchmark for Price, Quality, and Brand Preferences in LLM Booking Agents

PriceBench is a diagnostic benchmark that extracts price, quality, and brand preferences from large language models (LLMs) by analyzing their hotel booking choices. Using a logit choice model, the study evaluated 28 LLMs from eight providers across 3,600 booking tasks involving 179 New York City hotels. Results show that more capable LLMs exhibit stronger, more consistent preferences, while weaker models either lock onto a single position or show near-indifference, with significant variation in price sensitivity and price/quality trade-offs across providers.

By Pavel Kireyev
arXiv AI
Sep 28

Evaluating Cultural Awareness of LLMs for Haitian Creole

The paper presents the first systematic evaluation of cultural awareness in large language models (LLMs) for Haitian Creole, a low‑resource language. Using a benchmark of culturally salient prompts curated by native speakers, the study assesses four dimensions—specificity, bias, diversity, and variation—in a text‑infilling setting. Results show a clear gap between Haitian Creole and higher‑resource French, with Haitian performance more uneven and more affected by French linguistic interference; story generation reveals stereotypical portrayals of Haitian characters even in positive contexts.

By Christelle Clervilsson, Yanzhu Guo
arXiv AI
Sep 28

Statistical attribute alignment for black-box generative AI via output post-processing

The paper introduces algorithms for aligning the attribute distribution of outputs from black-box generative AI models with a user-specified target. It focuses on minimizing the number of queries needed to achieve exact or approximate alignment, proving optimality as the number of requested outputs grows. Experiments on text-to-image and geocoded persona generation demonstrate that the proposed post-processing methods improve statistical attribute alignment beyond prompting alone.

By Kevin Jiang, Morgane Austern, Edgar Dobriban, Jason M. Klusowski
arXiv AI
Sep 28

The Plot Twist: Jailbreaking Unified Multimodal Models with a Three-Act NarrativeAttack

The paper introduces NarrativeAttack, a jailbreak framework that exploits unified multimodal models (UMMs) by embedding a malicious query within a self‑contained three‑act visual narrative. The attack uses the model’s own image generator to create setup and resolution images, hiding the malicious event as a hidden climax, and concludes with an image‑based guessing game that forces the model to select the relevant answer. Experiments demonstrate that NarrativeAttack outperforms previous methods, achieving up to 88.25% attack success rate on Gemini‑2.5‑Flash, revealing a significant safety vulnerability in UMMs.

By Shaoxiong Guo, Tianyi Du, Lijun Li, Yuyao Wu, Jie Li, Jing Shao
arXiv AI
Sep 28

Testing the Utility of Using Large Language Models to Create Personalized Networks From Therapy Session Transcripts: A Proof of Concept Study

The study presents a pipeline that uses large language models to automatically generate personalized client networks from therapy session transcripts, aiming to aid case conceptualization and treatment planning. Researchers annotated 8,028 utterances from 77 transcripts, then used binary and multi-label classification to identify clinically relevant processes and dimensions, clustered these processes, and generated connections between clusters. Evaluation showed general support for the model’s utility, though interrater agreement varied, and qualitative analysis indicated the networks were coherent and interpretable, suggesting feasibility while highlighting the need for further research on outcome improvement.

By Clarissa W. Ong, Hiba Arnaout, Kate Sheehan, Estella Fox, Eugen Owtscharow, Iryna Gurevych
arXiv AI
Sep 28

Stepwise Intrinsic Rewards for Reasoning in Large Language Models

The paper introduces Stepwise Marginal Information Gain (MIG), an intrinsic process reward that evaluates how each reasoning step of a large language model (LLM) or vision-language model (VLM) improves the likelihood of the reference answer. MIG rewards only new likelihood maxima, preventing duplicate credit, and is combined with outcome, format, and self‑distillation objectives to guide training. Experiments on eight benchmarks show that this method outperforms outcome‑only reinforcement learning and improves accuracy by up to 4.8 points over binary‑reward training, including a 12.6‑point gain on MathVerse and a 12.9‑point advantage on vision‑language transfer at 7B parameters.

By Xiangwei Wang, Wei Wang, Ken Chen, Nanduni Nimalsiri, Sachith Seneviratne, Saman Halgamuge
arXiv AI
Sep 28

Latent Generative Solvers for Generalizable Long-Term Physics Simulation

The paper introduces the Latent Generative Solver (LGS), a neural PDE solver that combines a Physics VAE, a Pyramidal Flow-Forcing Transformer, and input noising to achieve generalization across twelve PDE families and stable long-term rollouts. LGS matches or surpasses deterministic baselines on one-step predictions, outperforms them on 5- and 10-step rollouts, and significantly reduces long-horizon error while cutting compute costs. It also adapts efficiently to unseen higher-resolution systems, demonstrating strong empirical performance on 2D regular-grid PDE simulations.

By Zituo Chen, Sili Deng
arXiv AI
Sep 28

When to Think Fast and Slow? AMOR: Adaptive Entropy Gate for Hybrid Models

The paper introduces AMOR, an Adaptive Metacognitive Output Router that selectively activates attention in recurrent-attention hybrid models based on predictive uncertainty. By gating attention blocks with an entropy threshold derived from a running batch median and scaled standard deviation, AMOR reduces attention usage to about 40% of positions while achieving superior common-sense reasoning performance across multiple scales. The approach also improves retrieval accuracy over pure recurrent models and maintains long-context robustness, outperforming fixed-schedule hybrids under distribution shift.

By Haoran Zheng, Chen Shani
arXiv AI
Sep 28

Combee: Scaling Prompt Learning for Self-Improving Language Model Agents

Combee is a new framework that scales prompt learning for self‑improving language model agents by enabling many agents to run in parallel while learning from their combined traces. It uses parallel scans, an augmented shuffle mechanism, and a dynamic batch size controller to maintain quality and reduce delay. Experiments on AppWorld, Terminal‑Bench, Formula, and FiNER show up to 17× speedup over prior methods with comparable or better accuracy at similar cost.

By Hanchen Li, Runyuan He, Qizheng Zhang, Changxiu Ji, Qiuyang Mang, Xiaokun Chen, Lakshya A Agrawal, Wei-Liang Liao, Eric Yang, Alvin Cheung, James Zou, Kunle Olukotun, Ion Stoica, Joseph E. Gonzalez
arXiv AI
Sep 28

Agentick: A Unified Benchmark for General Sequential Decision-Making Agents

Agentick is a unified benchmark for sequential decision‑making agents that evaluates RL, LLM, VLM, hybrid, and human agents on 37 procedurally generated tasks across six capability categories, four difficulty levels, and five observation modalities via a single Gymnasium‑compatible interface. It includes a Coding API, oracle reference policies, pre‑built SFT datasets, a composable agent harness, and a live leaderboard. An evaluation of 27 configurations and over 90,000 episodes shows no single approach dominates, with GPT‑5 mini leading overall, PPO excelling in planning and multi‑agent tasks, and the reasoning harness boosting LLM performance by 3‑10×, while ASCII observations outperform natural language.

By Roger Creus Castanyer, Pablo Samuel Castro, Glen Berseth
arXiv AI
Sep 28

AcuityBench: Evaluating Clinical Acuity Identification and Uncertainty Alignment

AcuityBench is a new benchmark that tests whether language models can correctly identify the urgency of medical care needed from user presentations. It unifies five public datasets—user conversations, online forum posts, clinical vignettes, and patient portal messages—under a shared four-level acuity framework, providing 914 cases for evaluation. The benchmark supports both explicit four-way classification and free-form conversational responses, revealing that models vary widely in accuracy and that conversational formats reduce over-triage but increase under-triage, especially for high-acuity cases.

By Robin Linzmayer (Department of Computer Science, Columbia University, Department of Biomedical Informatics, Columbia University), Georgianna Lin (Department of Biomedical Informatics, Columbia University), Di Coneybeare (Department of Emergency Medicine, Columbia University Irving Medical Center), Jason Chu (Department of Emergency Medicine, Columbia University Irving Medical Center), Trudi Cloyd (Department of Emergency Medicine, Columbia University Irving Medical Center), Manish Garg (Department of Emergency Medicine, Columbia University Irving Medical Center), Miles Gordon (Department of Emergency Medicine, Columbia University Irving Medical Center), Elizabeth Hartofilis (Department of Emergency Medicine, Columbia University Irving Medical Center), Benjamin Hong (Department of Emergency Medicine, Columbia University Irving Medical Center), Ashraf Hussain (Department of Emergency Medicine, Columbia University Irving Medical Center), Eugene Y. Kim (Department of Emergency Medicine, Columbia University Irving Medical Center), Oluchi Iheagwara King (Department of Emergency Medicine, Columbia University Irving Medical Center), Ross McCormack (Department of Emergency Medicine, Columbia University Irving Medical Center), Erica Olsen (Department of Emergency Medicine, Columbia University Irving Medical Center), John K. Riggins Jr (Department of Emergency Medicine, Columbia University Irving Medical Center), Mustafa N. Rasheed (Department of Emergency Medicine, Columbia University Irving Medical Center), Dana L. Sacco (Department of Emergency Medicine, Columbia University Irving Medical Center), Vinay Saggar (Department of Emergency Medicine, Columbia University Irving Medical Center), Osman R. Sayan (Department of Emergency Medicine, Columbia University Irving Medical Center), Amit Shembekar (Department of Emergency Medicine, Columbia University Irving Medical Center), Janice Shin-Kim (Department of Emergency Medicine, Columbia University Irving Medical Center), Wendy W. Sun (Department of Emergency Medicine, Columbia University Irving Medical Center), Bernard P. Chang (Department of Emergency Medicine, Columbia University Irving Medical Center), David Kessler (Department of Emergency Medicine, Columbia University Irving Medical Center), No\'emie Elhadad (Department of Computer Science, Columbia University, Department of Biomedical Informatics, Columbia University)
arXiv AI
Sep 28

CODESKILL: Learning Self-Evolving Skills for Coding Agents

CODESKILL is an LLM-based framework that learns to extract, evolve, and maintain procedural skills from coding-agent trajectories. It treats skill extraction and skill-bank management as a learnable policy trained with reinforcement learning, using a hybrid reward combining rubric-based skill quality and verifiable execution feedback. Experiments on EnvBench, SWE-Bench Verified, and Terminal-Bench 2 demonstrate that CODESKILL raises average pass rates by 11.03 over a no-skill baseline and by 5.10 over the strongest prompt-based or memory baseline while keeping a compact skill bank.

By Yanzhou Li, Yiran Zhang, Xiaoyu Zhang, Xiaoxia Liu, Yang Liu
arXiv AI
Sep 28

Energy Efficiency of Locally Deployed LLMs: A Preliminary Quantitative GPU Power Benchmark on Consumer Hardware

The paper reports a reproducible GPU power benchmark for 18 open‑source LLMs (0.5B–7B parameters) run on a single consumer RTX 4060ti GPU using the Ollama inference engine. Energy metrics such as mean/peak power, total energy per prompt, energy per output token, and throughput were measured, revealing that model architecture and quantization strategy, rather than parameter count alone, drive energy efficiency. The most efficient models were qwen2.5:0.5b and tinyllama:1.1b, while the 7B‑Mistral model consumed up to 8.6× more energy per token, and qwen3.5:0.8b(on) showed unusually high per‑prompt energy due to extended internal reasoning.

By Philipp M. Z\"ahl, Elja Dalipaj, Anika Hennig, Timon Bayer
arXiv AI
Sep 28

Agentic AI-driven Immersive Simulation: A Knowledge-Aware Virtual Training Platform for High Dose Rate (HDR) Brachytherapy

The paper introduces an agentic AI-driven immersive simulation for High Dose Rate vaginal cylinder brachytherapy, combining Virtual Reality, mobile computing, and a knowledge-aware assistant powered by Retrieval-Augmented Generation. The system offers a high‑fidelity, risk‑free training environment that provides natural language interfaces and real‑time guidance during complex medical maneuvers. A prototype using Meta Quest 3 and a GPU‑accelerated backend demonstrates acceptable latency and high context precision, answer completeness, and relevance in the pedagogical support.

By Ronghua Xu, Kepha Barasa, Manoj Kumal, Xinyun Liu, Weihua Zhou, Xin Qian
arXiv AI
Sep 28

T-LoopFormer: Token-Level Elastic-Depth Looped Transformers for Latent Reasoning with Dynamic Routing

T-LoopFormer introduces token-level elastic-depth looped transformers that allow each token to decide its own number of loop iterations based on its hidden state, improving token generation accuracy. It also adds a recursion-wise key‑value cache so tokens at different depths only attend to their corresponding cached states, speeding up autoregressive decoding. Experiments demonstrate strong performance on language modeling and zero‑shot reasoning, achieving the lowest decoding latency among comparable models.

By Mingqian Yu, Wenpeng Zhang, Shaobo Cui, Peilin Zhao
arXiv AI
Sep 28

LEAD: An EEG Foundation Model for Alzheimer's Disease Detection

LEAD is a gated temporal‑spatial Transformer foundation model designed for EEG‑based Alzheimer's disease detection. It was trained on the world’s largest EEG‑AD corpus of 2,238 subjects and uses a subject‑regularized strategy and medical contrastive learning across 13 datasets. LEAD outperforms existing methods on five downstream AD datasets, achieving the best average ranking across 20 evaluations.

By Yihe Wang, Nan Huang, Nadia Mammone, Marco Cecchi, Xiang Zhang