Large language models

Model releases, architecture work and prompting research on large language models — from frontier-lab announcements to the arXiv papers behind them.

28,308 stories · RSS feed

arXiv AI
Oct 2

AVSD-Scenes: A Dataset for Audio-Visual Description of Urban Scenes

AVSD-Scenes is a new dataset of 12,291 audio‑visual scene descriptions for urban environments, built from the TAU Urban Audio‑Visual Scenes dataset. The descriptions are generated by first creating modality‑specific text with Qwen2‑Audio‑7B and Qwen2.5‑VL‑7B, then merging them with large language models (Qwen3‑14B, Mistral‑Small‑3.2‑24B‑Instruct‑2506, Gemma‑3‑27B‑it) to produce multimodal narratives that combine auditory and visual cues. Benchmarks show that these multimodal descriptions improve semantic alignment, cross‑modal retrieval, and scene classification accuracy (up to 95.4%) while remaining discriminative even without explicit scene labels.

By Dhanunjaya Varma Devalraju, Arshdeep Singh, Mark D. Plumbley
arXiv AI
Oct 2

Mapping the RAG Landscape: A Four Axis Taxonomy of Efficiency, Defense, Interactivity, and Reasoning

The paper surveys recent advances in Retrieval Augmented Generation (RAG), a technique that integrates external retrieval into language model generation to reduce hallucinations and keep knowledge current. It introduces a four‑axis taxonomy—efficiency, defense, interactivity, and reasoning—to organize contemporary RAG research, covering retrieval methods, fusion strategies, embedding optimizations, and reinforcement learning policies. The survey also reviews evaluation practices, domain‑specific applications, and architectural variants, while highlighting ongoing challenges such as retrieval quality, reliability, domain adaptation, scalability, and explainability.

By Meghana Sunil, Shravya V, Shravan Venkatraman, Joe Dhanith PR
arXiv AI
Oct 2

Counterfactual Auditing of Bias in Open-Source Large Language Models for Clinical Triage

The study evaluates counterfactual bias in ten open‑source large language models (LLMs) for pediatric Emergency Severity Index (ESI) prediction. By creating paired clinical vignettes that differ only in demographic or socioeconomic variables, the authors measure shifts in acuity assignment, finding that counterfactual sensitivity varies widely across model families and sizes. A fine‑tuned Qwen2.5‑7B model exhibited the lowest sensitivity, while larger or medical‑domain models sometimes showed greater shifts, highlighting the need for fairness assessment before clinical deployment.

By Manar Aljohani, Brandon Ho, Kenneth McKinley, Dennis Ren, Xuan Wang
arXiv AI
Oct 2

Causal Memory Policy: Making Memory Utility Identifiable by Intervening on Retrieval

The paper introduces Causal Memory Policy (CMP), a framework that identifies the utility of memories in memory‑augmented language models by intervening on retrieval rather than on storage. CMP reserves fixed context slots for memories sampled with known propensities and estimates utility using self‑normalized inverse propensity weighting, providing unbiased estimates and exact variance. Experiments show that CMP improves discrimination between required and non‑required memories and reveals that identified utility alone is insufficient for retention decisions across unseen queries.

By Arman Behnam, Binghui Wang
arXiv AI
Oct 2

Counting Moves, Weighing Voices: Bayesian Dialectical Argumentation for Calibrated Multi-LLM Councils under Persistent Adversaries

The paper introduces Bayesian Dialectical Argumentation (BDA), a method for aggregating answers from multiple large language models (LLMs) in a council setting. BDA treats each LLM’s typed moves—proposals, challenges, and concessions—as evidence in a classical annotator model, estimating per-agent reliability even when some agents are persistently unreliable. By weighting evidence according to these inferred reliabilities, BDA produces calibrated posterior probabilities for candidate answers and can invert unreliable agents instead of merely outvoting them, achieving superior calibration and robustness on both binary and multi-class benchmarks without extra LLM calls.

By Ionel Eduard Stan, Paolo Napoletano
arXiv AI
Oct 2

SPHERE: Adaptive VR Indoor Scene Generation via LLM-Enhanced Spatial Preference Learning and Human-in-the-Loop RL

SPHERE is an adaptive VR indoor scene generation framework that turns isolated 3D synthesis into continuous human‑AI co‑creation. It learns persistent spatial preferences from multimodal user interactions, abstracts these into hierarchical constraints for geometric resilience, and employs a human‑in‑the‑loop reinforcement learning loop to refine retrieval policies. A mixed‑design study with 42 participants and offline ablation show that SPHERE reduces corrective edits and physical effort while avoiding bias toward shallow object‑level traits, producing geometrically resilient, profile‑aligned layouts.

By Hyeonmin Lee, Zheng Wei, Kyungmin Kwon, Jumin Seo, Jiwon Park, Hayoung Oh
arXiv AI
Oct 2

Mimir: Physics-Grounded LLM Agents for Long-Horizon Irrigation Control

Mimir is a physics‑grounded large language model agent designed for long‑horizon irrigation control. It operates on two timescales: a fast scale that uses a structured physical interface and deterministic simulator to validate and refine LLM proposals before execution, and a slow scale that consolidates recurrent failure patterns into persistent contextual principles. Across multiple sites, crops, and years, Mimir achieves the lowest aggregate control cost and reduces irrigation usage by about 51% compared to historical schedules, while ablation studies confirm the importance of forward simulation, verified revision, and persistent context.

By Yimeng Liu, Mi Zhang, Younsuk Dong, Zhichao Cao
arXiv AI
Oct 2

External Observers May See More Clearly: Cross-Model Span-Level Hallucination Detection in Large Language Models via Hidden State Probing

The paper proposes a hidden‑state probing method for detecting hallucinations at the span level in large language model outputs, moving beyond token‑wise binary classification. By examining layer‑wise activation patterns, the approach identifies the exact onset and continuation tokens of hallucinations, achieving higher precision‑recall AUC than random baselines despite class imbalance. Additionally, the authors introduce a cross‑model detection framework where one model observes another’s internal representations, showing that an external observer can match or surpass the generator’s own self‑detection of hallucination onsets, even when the observer is smaller.

By Kingshuk Gupta, Davide Buscaldi
arXiv AI
Oct 2

VISTA: A Visual Harness for Reasoning in an Interactive World

The paper introduces VISTA, a visual harness that equips a general-purpose multimodal model with long‑horizon vision and a lossless visual memory. VISTA enables the model to directly perceive and actively retrieve past observations, allowing it to reorganize visual input during reasoning. On the ARC‑AGI‑3 benchmark, VISTA boosts Claude Opus 5.0’s Relative Human Action Efficiency from 40.68 to a perfect 100.00, completing all 25 public games with 57.4% fewer actions than first‑time human participants, and it also outperforms baselines on three additional visual game and puzzle benchmarks.

By Qiushi Han, Keya Hu, Linlu Qiu, Cathy Wu, Kaiming He
arXiv AI
Oct 2

The First Token Is Not the Verdict: Hidden Costs of Reading LLM Judges Without Generating

The paper demonstrates that reading a large language model (LLM) judge’s verdict from the logits of its first generated token—an approach used in constrained decoding and likelihood‑scoring evaluation—introduces a significant distortion in position bias. Because judges do not always start with a verdict token (12–49% of cases for Qwen3 judges and <3% for Llama‑3.1‑8B and Phi‑3.5‑mini), this readout often returns the first response rather than a true judgment, inflating position bias by up to 42 points while barely affecting judge accuracy. The authors recommend reporting the frequency with which a judge leads with a verdict token to provide a more accurate assessment of position bias.

By Gnaneswar Villuri, Hashmath Shaik, Alex Doboli
arXiv AI
Oct 2

BudgetSchemaBench: A Budget-Swept Diagnostic for Schema Context in Text-to-SQL

BudgetSchemaBench is a diagnostic tool for evaluating how different schema‑context budgets affect text‑to‑SQL systems. It automatically derives relevance labels from gold SQL, tests four budgets across 80 databases, and compares three schema representations while keeping table rankings fixed. The study shows that increasing the budget improves execution accuracy, especially for lexical retrieval, and that dense retrieval already captures most needed tables at low budgets.

By Chen Shen
arXiv AI
Oct 2

Safety in Self-Evolving Agents: A Survey

The article surveys safety concerns for self‑evolving agents that continually update their internal state, such as model parameters and memories, from new interactions. It introduces the SAVER framework, which tracks reusable influence, adaptation, violations, exposure, and response to assess whether safety properties persist as agents evolve. The survey finds that legitimate state can become unsafe when its persistence, authority, or scope expands beyond its original conditions, and highlights gaps in current research on descendant repair and longitudinal evaluation.

By Jiahao Chen, Zhou Feng, Oubo Ma, Yichen Yan, Ruixiao Lin, Hangtao Zhang, Linkang Du, Hengyu An, Yong Yang, Jun Liu, Junhao Li, Naen Xu, Chunyi Zhou, Yuan Su, Zehao Jin, Qianli Ma, Leyi Qi, Yiming Wang, Zhe Ma, Yuwen Pu, Mengyao Du, Yuanyi Song, Enhao Huang, Zhihui Fu, Jun Wang, Jinfeng Li, Yuefeng Chen, Hui Xue, Yiming Li, Tianyu Du, Shouling Ji
arXiv AI
Oct 2

When the AI Leaves the Tailorshop: Measuring What an LLM Advisor Leaves Behind in Complex Problem Solving

The study investigates how large language model (LLM) advisors affect complex problem‑solving in a simulated clothing‑factory setting. Two preregistered experiments (N=200 and N=198) found that participants with AI support reported higher confidence and understanding, expended less effort, and in some cases achieved better performance or avoided bankruptcy. Within the AI‑supported group, more frequent changes to the AI’s recommendations were linked to improved unaided performance and knowledge.

By Robin Welsch
arXiv AI
Oct 2

White Men Without Degrees Receive the Lowest Ratings from Large Language Models

The study reports that white men lacking an undergraduate degree receive the lowest average ratings from large language models across credit, hiring, and rental application scenarios. Using 18 models and 32 demographic profiles, the researchers found that these individuals scored 75.87 in credit, 92.71 in hiring, and 86.62 in rental housing, while black women with degrees scored highest. The findings show that white men without degrees consistently rank lowest or second‑lowest in most model‑scenario combinations, underscoring a distinct disadvantage not captured by broader racial or gender categories.

By Maxim Chupilkin
arXiv AI
Oct 2

Decoding the Disaster: Multi-Task Geospatial Reasoning with Vision-Language Models and Crowdsourced Imagery for Disaster Mapping

The paper introduces GRDisaster, a multi-task geospatial reasoning framework that leverages vision‑language models to interpret, geolocalize, and assess damage in crowdsourced disaster imagery. It builds on a new benchmark dataset of 26,340 images from PhotoMappers, linking volunteer geographic information, street‑view imagery, and remote sensing data across multiple disaster events from 2018 to 2024. GRDisaster combines deterministic and probabilistic cross‑view geolocalization with multi‑view fusion, and introduces spatial reasoning indicators to validate cross‑view matches and quantify disaster severity using expert‑verified annotations.

By Wenping Yin, Fabian Desuer, Ziqi Liu, Naixia Mou, Weijia Li, Pedram Ghamisi, Xiao Xiang Zhu, Hao Li
arXiv AI
Oct 2

Proof-Gated Signing: Solver-Checked Transaction Guards that Hold Under State Drift for Onchain AI Agents

Proof‑Gated Signing (PGS) is a method for securing on‑chain AI agents that control wallets by simulating proposed transactions, extracting their effects, and using an SMT solver to verify a declarative value‑and‑permission policy across a range of oracle‑uncertainty prices. PGS then compiles on‑chain post‑conditions—such as wallet balance bounds, payee receipts, allowance caps, and ownership checks—ensuring that any execution satisfying these conditions also satisfies the policy, even when the chain state drifts due to front‑running, contract upgrades, or token‑parameter changes. In a testbed of 260 scenarios, PGS prevented 93.6 % of harmful cases and passed 97.5 % of benign ones, outperforming simulation‑only checking and static allowlists, while incurring about 41 k gas and 0.1–0.2 s per check.

By Bravish Ghosh
arXiv AI
Oct 2

Deep Learning for Anomaly Detection in Railway Systems: A Structured Survey

This survey reviews deep learning methods for detecting anomalies in railway systems, organizing them by taxonomy of anomaly location, data representation, sensing modality, and temporal traits. It categorizes approaches—convolutional, recurrent, attention-based, autoencoders, GANs, transformers—into classification, prediction, reconstruction, and hybrid paradigms, and discusses data challenges, evaluation, metrics, and deployment issues such as edge‑cloud architectures and hardware constraints. The paper also offers a decision‑oriented framework linking anomaly characteristics, data properties, and operational constraints to guide the selection and deployment of suitable detection solutions.

By Ammar Bouketta, Smail Niar, Hamza Ouarnoughi
arXiv AI
Oct 2

A Comparative Explainability Framework for DeBERTa-v3 in Zero-Shot Medical Abstract Classification

The paper introduces a comparative explainability framework for auditing DeBERTa‑v3 in zero‑shot medical abstract classification. It evaluates five explanation methods—SHAP, LIME, occlusion, Input × Gradient, and Attention × Gradient—using a natural language inference engine on a balanced corpus of 1,000 abstracts per diagnostic category. The study finds that explanatory stability aligns with predictive certainty, identifies three systemic failure mechanisms, and recommends combining multiple explanation methods and quantitative agreement metrics for transformer‑based medical text classifiers.

By Javier Diaz Esteban-Herreros, David Mu\~noz-Valero, Raquel Mart\'inez-Espa\~na, Jose M. Juarez, Juan Moreno-Garcia
arXiv AI
Oct 2

VisionQ: VLM-as-a-Judge Taxonomy, Dataset and Benchmark for Qualitative Analysis in Computer Vision

VisionQ is a new benchmark for qualitative analysis in computer vision that evaluates vision‑language models (VLMs) on criterion‑conditioned visual discrimination. It is built from over 1,800 peer‑reviewed comparison figures in CVPR and ICCV papers, linking each image crop to author‑stated visual claims through 3,911 hand‑annotated data points. The benchmark includes a 51‑leaf taxonomy of visual criteria, a protocol that hides method identities and reports accuracy per criterion, and a DPO‑tuned Gemma‑4‑E4B judge that improves accuracy on a held‑out test set.

By Vu Dinh Xuan, Duc-Hai Nguyen, Minh-Dung Dao, Vu Quynh Giao, Quang Hong Nguyen, Binh-Son Hua, Barry O'Sullivan, David Murphy, Hoang D. Nguyen
arXiv AI
Oct 2

Are AI Coders Snitches? An Empirical Study of Pretraining Data Detection on Code Large Language Models

The paper investigates whether code large language models (CodeLLMs) inadvertently reproduce proprietary or sensitive code by evaluating seven state‑of‑the‑art training data detection (TDD) methods on eight CodeLLMs. It introduces CodeSnitch, a benchmark of 9,000 function‑level code samples across three languages, each labeled as included or excluded from training data, and applies mutation strategies based on the Type‑1 to Type‑4 code clone taxonomy to test TDD robustness. The study offers a systematic assessment of current TDD techniques for code and suggests directions for developing more effective detection methods.

By Tianlin Li, Yunxiang Wei, Zhiming Li, Aishan Liu, Qing Guo, Xianglong Liu, Dongning Sun, Yang Liu