Large language models

Model releases, architecture work and prompting research on large language models — from frontier-lab announcements to the arXiv papers behind them.

32,584 stories · RSS feed

arXiv AI
Sep 12

Fork Where the Model Changes Its Mind: Belief-Shift Branching for Tree-Structured Reinforcement Learning

The paper introduces belief‑shift branching, a method for placing forks in tree‑structured reinforcement learning rollouts by detecting where a model’s answer belief changes most sharply. Unlike traditional fixed‑length or entropy‑based forking, this approach uses a lightweight probe or learned activation direction to identify pivots in the value curve, reducing unnecessary sampling. Experiments show that belief‑shift forking consistently outperforms baseline methods across multiple models and benchmarks, yielding significant gains in mathematics and code tasks.

By Bin Lei, Yu Li, Prafulla Kumar Choubey, Jiaxin Zhang, Becky Xiangyu Peng, Qinyuan Ye, Kartik Narayan, Caiwen Ding, Silvio Savarese, Chien-Sheng Wu
arXiv AI
Sep 12

An AI-Powered Culturally Aware Chatbot for Stress Detection and Wellness Support among Pakistani University Students Using NLP and Machine Learning

The paper presents an AI‑powered, culturally aware chatbot designed to detect stress and provide wellness support for Pakistani university students. Using a Random Forest model trained on 1,100 responses across 20 psychological, physiological, academic, environmental, and social features, the system achieves 89.09% accuracy and a macro F1‑score of 0.89 across three stress severity levels. The model’s outputs are fed to an open‑source large language model via OpenRouter, enabling culturally tailored conversations in English, Urdu, and Roman Urdu, with teacher‑student relationship identified as a key stressor.

By Muhammad Fahad Bashir, Muhammad Afzal
arXiv AI
Sep 12

Exploring Diffusion Transformers for Cross-Modal Augmentation in Multimodal Brain State Decoding

The paper introduces CoMA-DiT, a bidirectional cross‑modal Diffusion Transformer that uses paired modalities as mutual generative supervision for latent augmentation rather than just inputs for fusion. By conditioning velocity prediction on the paired modality through cross‑modal attention and injecting variation via a reliability‑gated residual mechanism, CoMA‑DiT improves multimodal brain state decoding. Experiments on auditory attention decoding and emotion recognition show consistent gains over 20 baselines, with absolute accuracy and macro‑F1 improvements of 4.28% and 6.70% respectively, and extensive analyses confirm its robustness and interpretability.

By Ziwei Wang, Xingyi He, Hongbin Wang, Tianwang Jia, Bohan Fang, Dongrui Wu
arXiv AI
Sep 12

LLMs as Post-hoc Auditors of Physiological Plausibility in Symbolic Regression: A Clinician-Evaluated Case Study

The paper investigates using Large Language Models (LLMs) as post-hoc auditors for symbolic regression models produced by genetic programming. By having three LLMs analyze and rank four evolved expressions across multiple runs, the study compares the LLM-generated rankings and interpretations with assessments from three clinicians. Results show that LLMs provide more favorable comparative rankings than isolated term-level interpretations, yet they also generate physiologically and mathematically questionable explanations, suggesting they are best used under expert oversight rather than for autonomous validation.

By Jorge L\'opez-Varela, J. Ignacio Hidalgo, Jos\'e-Manuel Mu\~noz, Omar Costilla-Reyes, Esther Maqueda, Jesus Moreno-Fernandez, Tom\'as Gonz\'alez-Vidal, J. Manuel Velasco, Oscar Garnica
arXiv AI
Sep 12

RouteRepair: Instance-Level Failure Diagnosis and Targeted Repair in LLM-Based Automated Heuristic Design for Routing Optimization

RouteRepair is a method that diagnoses specific weaknesses in large language model (LLM)-generated routing heuristics by evaluating performance at the instance level and then applies targeted modifications to the heuristic components that are failing, while preserving components that already perform well. It combines routing evidence, solver behavior, and program context to set bounded repair objectives and validates each change through matched parent-child evaluation of failure recovery and collateral degradation. Experiments on the traveling salesman problem (TSP) and capacitated vehicle routing problem (CVRP) show significant reductions in optimality gaps and route costs, demonstrating that failure-aware, evidence-constrained refinement can improve routing heuristics on difficult instances while maintaining performance on easier cases.

By Binghao Ji, Di Huang, Jiahui Fang, Zhiyuan Liu
arXiv AI
Sep 12

ActMap: Single-Pass Uncertainty Quantification from Generation-Time Activation Maps

ActMap is a new white‑box representation that compresses the entire hidden‑state trajectory of a language model during generation into a fixed 12 × 32 × 128 tensor. This compact 96 KiB map can be captured with no overhead and is read by a lightweight Vision Transformer to estimate answer correctness in a fraction of a millisecond. In experiments on short‑answer QA, math, and summarization, ActMap outperforms sampling, token‑probability, attention, and embedding baselines and matches a larger ACT‑ViT detector while achieving lower calibration error on most test pairs.

By Jacopo Dardini (University of Bologna), Roberta Calegari (University of Bologna)
arXiv AI
Sep 12

Prompt Revision as a Source of Cultural Bias in Text-to-Image Systems

The paper "Prompt Revision as a Source of Cultural Bias in Text-to-Image Systems" investigates how commercial text‑to‑image models silently modify user prompts before generating images, a step that is often hidden from users. Using the multilingual benchmark WORLDVIEW, the authors audit the revision layer in DALL‑E‑3, Imagen‑4, and GPT‑Image‑1.5, finding that non‑Western and non‑Anglophone contexts are disproportionately marked, reduced to narrow vocabularies, and stereotyped. The study demonstrates that the revision layer itself is a previously undocumented causal source of cultural stereotyping, underscoring the need to audit deployed systems rather than just the underlying models.

By Aleksandra Urman, Elsa Lichtenegger, Salima Jaoua, Azza Bouleimen, Robin Forsberg, Corinna Hertweck, Stefania Ionescu, Nicol\`o Pagan, Ancsa Hannak, Joachim Baumann
arXiv AI
Sep 12

Making Alternative Data Work: Context-Augmented LLMs for Financial Forecasting

The paper explores how large language models (LLMs) can forecast a firm’s future financial performance by integrating alternative data—such as consumer transactions, web traffic, and prediction markets—with traditional financial information. A two‑agent framework is proposed: one agent identifies which alternative data channels are relevant for each firm, and the other uses firm‑ and channel‑specific context to predict revenue. Experiments across four commercial alternative data channels show that incorporating alternative data in context improves LLM forecasts over using either data source alone and outperforms standard forecasting baselines.

By Jihoon Kwon, Lawrence Liu, Daekyung Park, Sumin Kim, Haverty Jack, Hoyoung Lee, Katherine Bjorkman, Josh McKenney, Peter Laurelli, Nicole Kagan, Zach Golkhou, Thorsten Neumann, Edward Tong, Pete Petersen, Yoon Kim, Alejandro Lopez-Lira, Yongjae Lee, Chanyeol Choi
arXiv AI
Sep 12

MAPLE: Memory-Augmented Planning with Language and Evolution

MAPLE is a new agent that maintains and updates optimization problems through successive natural‑language requests, combining language‑based problem construction with mathematical programming and evolutionary search. It preserves the optimization program, accepted plans, earlier updates, and candidate solutions for future requests, enabling rapid adaptation to changing business constraints. In a benchmark of 15 trajectories and 180 updates across various operational domains, MAPLE completed all trajectories with high online scalar quality and Pareto hypervolume ratio, and maintained update validity and useful search information across substantial revisions.

By Kesheng Chen, Yamin Hu, Wenjian Luo
arXiv AI
Sep 12

Geospatial AI, Dataverse Metadata, and the Study of Place-Based Government

The paper presents a knowledge graph built from Harvard Dataverse’s public data, linking 102,650 datasets to 215,985 nodes and 528,003 edges that include keywords, publications, subjects, journals, and locations. About 43,991 datasets contain geospatial metadata, and 7,654 are identified as policy‑relevant, with elections and legislatures forming the largest cluster. The authors highlight the challenge of place resolution—disconnected nodes representing the same location—and propose the graph as a testbed for AI‑driven metadata enrichment and entity resolution, noting a bias toward American city‑level data.

By Danny EBanks, Devika Jain
arXiv AI
Sep 12

Beyond Verified Answers: Solver-Informed Self-Distillation for Bootstrapping Operations Research Language Models

The paper introduces SOLID, a framework that enables operations research language models to self-improve without relying on verified answers or external evaluators. SOLID uses solver-generated artifacts from the model’s own rollouts to create pseudo-references, clustering objectives and applying group-relative advantages for dense self-supervision. Experiments on multiple OR benchmarks show that SOLID enhances solution accuracy for both general-purpose and OR-tuned models compared to outcome-only training.

By Rui Zhu, Minglong Cao, Chenyu Zhou, Jianghao Lin, Dongdong Ge
arXiv AI
Sep 12

Beyond Static Guarantees: Measuring the Static-Pass Dynamic-Fail Gap in Security-Sensitive and LLM-Generated Python Code

The paper introduces the Static‑Pass Dynamic‑Fail (SPDF) phenomenon, showing that static analysis can miss vulnerabilities that are exploitable at runtime. Using a three‑stage pipeline—static scanning, LLM‑driven CWE reasoning, and autonomous exploit verification—it evaluated 1,355 Python samples and found that 14.53% of samples that passed static checks were actually exploitable. The study highlights that static‑analysis success and runtime security are distinct assurance layers, especially for AI‑generated and security‑sensitive code.

By Jessica Pourleyli, Maitreyee Das Urmi, Glaucia Melo
arXiv AI
Sep 12

Evaluating Scaffolding-Oriented Multi-Agent Large Language Model System for Clinical Interview Training

The study introduces a scaffolding-oriented multi-agent Large Language Model (LLM) AI Standardized Patient (AI-SP) platform designed to train medical students in patient interviews. In a randomized controlled trial with 100 students, the multi-agent system—comprising a patient agent, a Socratic tutor agent, and a turn-level evaluator—did not improve diagnostic accuracy but significantly enhanced overall OSCE scores, especially in communication, empathy, and history-taking. The authors also release a richly annotated dataset to support further research in AI-supported clinical reasoning training.

By Luming Yang, Haoxian Liu, Siqing Li, Rong Jia, Yue Xiao, Guanhua Chen, Li Lu
arXiv AI
Sep 12

BenchShield: Formal Model-Backed Instrumentation for Reward Integrity in LLM-Agent Evaluation Infrastructure

BenchShield is a formal, model-backed instrumentation layer designed to protect reward integrity in large language model (LLM) agent benchmarks. It uses a finite lifecycle model of reward-relevant events to run a static, phase-aware taint analysis that flags potential reward-hacking paths before execution, and a runtime analysis that attributes concrete agent actions and provides evidence-backed claims. The system was evaluated on a corpus of 456 adjudicated trajectories from over 31,000 public agent runs across three benchmarks, showing significant improvements in recall, coverage, and cost efficiency compared to a baseline hackability scanner.

By Shenghan Zheng, Zonglin Di, Yimin Liu, Kyoung Whan Choe, Jiankai Sun, Heguang Lin, Penghao Jiang, Yifeng He, Xiao Cheng, Jicheng Wang, Wenbo Chen, Alex Yates, Yinzhe Zhao, Bingran You, Yuan Gao, Ayush Munot, Shubham Gaur, Zhe Ye, Hao Wang, Xiangyi Li, Dawn Song, Christophe Hauser
arXiv AI
Sep 12

AI Soccer Analyst: Stage-Aware and Verifiable Human-AI Collaboration for Soccer Data Analysis

AI Soccer Analyst is a mixed‑initiative system that guides analysts through six revisable stages—Data Understanding, Problem Definition, Structured Planning, Execution, Evidence‑Grounded Reporting, and Interaction and Refinement—to produce soccer data analyses. A formative study with five analysts shaped design goals around automation, verifiability, human control, and accessibility, while a task‑based evaluation with 16 participants showed that 33 of 48 tasks met completion criteria and participants reported high quality, reliability, and verifiability of outputs. Interaction logs revealed that domain knowledge emerged during clarification, planning, and refinement, demonstrating that stage‑aware human‑AI collaboration can produce inspectable, revisable, and verifiable analyses while keeping domain experts in control.

By Calvin Yeung, Keisuke Fujii
arXiv AI
Sep 12

Discovering Temporal Structure: An Overview of Hierarchical Reinforcement Learning

The paper reviews hierarchical reinforcement learning (HRL) as a method for enabling agents to explore, plan, and learn in complex, open-ended environments by uncovering temporal structure in experience streams. It discusses the unclear definition of what makes a structure useful, the benefits of HRL for decision‑making challenges, and its impact on AI agent performance trade‑offs. The authors survey various HRL methods—from online learning to offline datasets and large language model integration—and outline the challenges and suitable domains for temporal structure discovery.

By Martin Klissarov, Akhil Bagaria, Ziyan Luo, George Konidaris, Doina Precup, Marlos C. Machado
arXiv AI
Sep 12

An Agentic Evaluation Framework for AI-Generated Scientific Code in PETSc

The paper presents PETSCAgent-Bench, a multidimensional benchmark and agent-based framework designed to evaluate AI-generated scientific code that uses the PETSc library. It combines deterministic checks with LLM-based assessments across five categories—correctness, performance, code quality, algorithmic appropriateness, and library-specific conventions—using a 14-evaluator pipeline. The framework demonstrates that while current large language models produce readable, well-structured code, they often fail on correctness and library conventions in realistic PETSc problems, revealing gaps that simple pass/fail tests miss.

By Hong Zhang, Barry Smith, Satish Balay, Le Chen, Murat Keceli, Lois Curfman McInnes, Junchao Zhang
arXiv AI
Sep 12

Some hypotheses on how chatbots work in problem-solution-driven conversations: Large Language Models as confirmation of the Innovation Illusion

The article examines chatbots as partners in problem‑solving conversations, arguing that basic chatbots—comprising a large language model (LLM) and a simple interface—are multifaceted but cannot match human cognitive flexibility. Drawing on Aggregation Dynamics, Cognitive Linguistics, Neuropsychology, and Psychology, the authors describe how LLMs encode artificial metaphorical problem propagations from training data, which only partially imitate human thinking. They conclude that further LLM development will not yield true thinking partners, yet chatbots are widely used, making their understanding socially and politically important.

By S. F. M. van Vlijmen, H. D. Lethe jr
arXiv AI
Sep 12

DCO: Dynamic Cache Orchestration for LLM Accelerators through Predictive Management

The paper proposes DCO, a dynamic cache orchestration scheme for multi-core AI accelerators that uses application-aware policies and dataflow information to guide cache replacement, bypass decisions, and thrashing mitigation. Using a cycle-accurate simulator, the authors demonstrate up to 1.80× speedup over conventional cache architectures and validate the approach with an analytical model and RTL implementation. The design occupies 0.064 mm² on a 15 nm process and operates at 2 GHz, showing that a shared system-level cache can simplify programming while boosting performance for large language model workloads.

By Zhongchun Zhou, Chengtao Lai, Yuhang Gu, Wei Zhang
arXiv AI
Sep 12

Towards a Deterministic Math Solver for Clinical Language Models

The paper explores a new approach to improve arithmetic reliability in clinical language models by having the model generate case‑specific Python code that a local executor runs deterministically, rather than performing arithmetic directly. Experiments on the MedCalc‑Bench Verified dataset show that this Program‑Solve interface yields modest gains for larger models (Qwen2.5‑32B) but not for smaller ones (Qwen2.5‑7B), and it does not replace the need for verified formulas or accurate variable extraction. The study highlights that adding an executor can help some open‑weight models but is not a universal solution.

By Felipe Ocampo Osorio, Sebasti\'an Andr\'es Cajas Ordo\~nez, Maximin Lange, Rafi Al Attrach, Sahil Kapadia, Zakaria Laouabdia Sellami, Angelo Antonio Talio, Leo Anthony Celi