Large language models

Model releases, architecture work and prompting research on large language models — from frontier-lab announcements to the arXiv papers behind them.

31,067 stories · RSS feed

arXiv AI
Sep 25

MOOSEnger: A Simulation-Aware AI Agent Framework for the MOOSE Ecosystem

MOOSEnger is a simulation‑aware AI agent framework designed for the MOOSE ecosystem, integrating an interchangeable reasoning model with domain knowledge, revised simulation artifacts, MOOSE‑specific validation, and executable solver feedback. Its generate‑check‑repair‑run workflow uses MOOSE knowledge retrieval, HIT‑aware parsing, syntax metadata, diagnostics, and revision‑controlled authoring to bind evidence to each input revision and guide bounded repair before acceptance. Across 200 prompts, MOOSEnger raises executable success from 5% to 89.5% with GPT‑5.2 and from 0% to 76.5% with Gemma 4 31B, and a ten‑case benchmark shows all generated inputs meet semantic alignment, with eight also meeting numerical‑accuracy criteria.

By Mengnan Li, Jason Miller, Zaid Abulawi, Zachary Prince, Matt Kohl, Jack M. Cavaluzzi, Guillaume Giudicelli, Casey T. Icenhour, Alexander Lindsay, Cody Permann
arXiv AI
Sep 25

Interpreting and Enhancing Emotional Circuits in Large Vision-Language Models via Cross-Modal Information Flow

The paper introduces a steering‑vector‑based causal attribution framework to study how large vision‑language models (LVLMs) translate visual input into emotional narratives. By creating a specialized dataset, the authors uncover a functional decoupling in the LVLM’s three‑stage Adapt‑Aggregate‑Execute mechanism: visual emotional cues are first aggregated in middle layers via sentiment‑specific attention heads, then translated into narrative generation in deeper layers through emotion‑general pathways. Using these insights, they regulate emotional information routing to strengthen attention flow and amplify semantic activation, achieving significant performance gains on the MER‑UniBench and reducing emotional hallucinations through inference‑time intervention.

By Chengsheng Zhang, Chenghao Sun, Zhining Xie, Xinmei Tian
arXiv AI
Sep 25

Post-Training Leaves Behavioral Shadows on Unrelated Decisions

The paper demonstrates that language models can acquire new capabilities from post‑training data even when the training text is unrelated to the target task. Using a method called Active Taskless Distillation (ATD), the authors show that a single word from a teacher model can transfer knowledge to a student model without any target‑task examples or teacher logits. Experiments on Qwen2.5-1.5B reveal significant performance gains on HumanEval+ and improvements in scientific knowledge, commonsense reasoning, and reading comprehension across various model families.

By Ziyang Zhang, Yubin Jing, Yuanhao Zeng, Yuyao Li, Haofan Wang, Yichen Gong
arXiv AI
Sep 25

A Multi-level Information Integration Framework for Physically Verifiable Fault Diagnosis of Rotating Machinery

The paper proposes a Diagnostic Evidence Network (DENet) that augments traditional fault‑diagnosis outputs with structured evidence, including classification, predicted characteristic frequency, and temporal localization of impulses. This evidence aligns with theoretical bearing physics and can be validated at inference time, achieving high AUROC for misclassification detection without sacrificing accuracy. A QLoRA‑adapted language model then translates DENet’s evidence into maintenance reports, markedly reducing unsupported claims.

By Yuntong Chen, Jianyu Liu, Yingqi Li, Guobin Zhao, Ziang Wang, Chao Chen, Xitian Tian, Lijiang Huang
arXiv Computation and Language
Sep 25

Language Specific Knowledge: Do Models Know Better in X than in English?

The paper introduces the concept of Language Specific Knowledge (LSK), showing that multilingual language models can answer certain queries better when prompted in a language other than English, sometimes even in low‑resource languages. It defines a language‑selection problem and presents several baseline methods, including the authors’ LSKExtractor, to empirically demonstrate that choosing the optimal language can improve question‑answering performance across datasets covering cultural and social norms. Experiments reveal non‑intuitive mappings, such as Gemma models excelling on Chinese and Middle Eastern topics in Spanish and Qwen models performing best on authority and responsibility queries in Arabic and Chinese.

By Ishika Agarwal, Nimet Beyza Bozdag, Dilek Hakkani-T\"ur
arXiv AI
Sep 25

Where LLM Graders Succeed and Break: Evidence from Two Computer-Science Exams

The study evaluates large language model (LLM) graders on two computer‑science exams, testing 171 configurations of closed‑ and open‑weights models. While the best LLM configuration achieved a mean absolute error of 1.64/35—better than the 2.61/35 error between two human graders—its performance was highly sensitive to the prompt. A short "strict grader" preamble caused most open‑weight models to exceed acceptable error thresholds or stop grading entirely, whereas fine‑tuning with a single LoRA adapter restored parity with human graders and reduced sensitivity to harsh prompts.

By Ali Habibullah, Yazan Alshoibi, Mohammad Alshiekh, Salman Khan, Naeemullah Khan
arXiv AI
Sep 25

PROOF: Profiling Reliability of Object-Level Facts in Large Language Models

PROOF is a benchmark that profiles the reliability of object-level facts in instruction-tuned language models by converting a frozen Wikidata snapshot into 18,486 English multiple-choice questions grounded in 11,779 semantic facts across 101 classes, 392 properties, and 14 domains. Each question includes an explicit "I don't know" option, a "No correct option" control, and nine controlled formulations, with 1,849 questions designed as no-correct-option traps. The study evaluates 18 open-weight model deployments on 166,374 prompts, revealing wide variability in factual accuracy, sensitivity to wording changes, and the impact of decoder perturbations.

By Andrei Chetvergov, Mikhail Solovev, Timofei Sivoraksha, Stepan Ukolov, Valeriia Kuschenko, Alexander Evseev, Sergey Bolovtsov
arXiv AI
Sep 25

Your Transformer Can Hold Two Thoughts at Once: Evidence of Linear Superposition in LLMs

The paper demonstrates that Large Language Models, despite their non‑linear components, exhibit a fundamental linearity property: when inputs from two distinct text streams are linearly combined, the model outputs a superposition of the individual next‑token distributions. This "Superposition Linearity Hypothesis" appears to be an intrinsic feature of the Transformer architecture, tends to weaken during pretraining, but can be largely restored with lightweight fine‑tuning. The authors also present a guided decoding method that separates the superposed outputs, allowing two coherent continuations to be generated from a single forward pass.

By Pavel Tikhonov, Anton Korznikov, Matvey Mikhalchuk, Nikita Dragunov, Temurbek Rahmatullaev, Polina Druzhinina, Anton Razzhigaev, Ivan Oseledets, Elena Tutubalina
arXiv AI
Sep 25

MorphIK: Morphology-Conditioned Neural Inverse Kinematics for Unknown Robots

MorphIK is a flow‑matching neural model that learns inverse kinematics for revolute‑joint kinematic chains it has never seen during training. Using a transformer to encode a robot’s morphology and target pose, the model generates pose solutions from noise and can be fine‑tuned with optimization to achieve sub‑centimeter accuracy. It also efficiently samples the robot’s null space, producing diverse configurations for the same pose.

By Lennart Clasmeier, Jan Gerrit Habekost, Cornelius Weber, Stefan Wermter
arXiv AI
Sep 25

An Empirical Study of VLM Pipelines for Long-Document QA

The paper evaluates how different design choices—document feeding strategy, retrieval method, and execution mode—affect Vision‑Language Models (VLMs) on long‑document question answering. Experiments on two benchmarks show that a multi‑tool agent only outperforms static input when the VLM is large, that retrieval modality (image vs text) is more critical than the specific retriever, and that combining the best pipelines per question can significantly boost performance. The study highlights the trade‑offs between token efficiency, model size, and pipeline complexity for deploying VLMs on complex documents.

By Kenan E. Ak, Jay Mohta, Gwang Gook Lee, Yan Xu, Dimitrios Dimitriadis
arXiv AI
Sep 25

World Action Agent: Harnessing VLMs for Robot Manipulation via World Action Rehearsal

World Action Agent (WAA) is a multi‑agent framework that lets vision‑language models (VLMs) directly pilot robots by operating within a visual action workspace. The workspace provides automatically selected contact views, editable action rehearsals, and in‑view correction to refine decisions before low‑level execution. WAA learns procedural skills from expert videos and human teaching, and its interaction traces can train smaller VLMs, achieving state‑of‑the‑art success on LIBERO‑Pro and improving out‑of‑domain performance on robosuite and Qwen3.5‑9B.

By Yehang Zhang, Haojian Huang, Yifan Chang, Jianchong Su, Bohan Zhou, Yingjie Xu, Wosong Chen, Tianhao Zhou, Chenxu Wang, Tianyi Zhang, Yangkai Wei, Wenqian Li, Shiyuan Deng, Yinchuan Li, Ying-Cong Chen, Zexi Li
arXiv Computer Vision
Sep 25

Looks the Same, Answers Differently: Flip-Direction Steering for Robust Vision-Language Reasoning

The paper introduces FlipDir, a training‑free inference‑time technique that mitigates answer flips in vision‑language models by steering hidden states along a low‑rank subspace derived from contrastive image pairs. It employs a margin‑based gate to attenuate steering only during uncertain decoding steps, thereby restoring original predictions while keeping stable ones unchanged. The authors also present VisFlip, a benchmark framework that evaluates models across nine dataset‑variation combinations in scientific reasoning, robot‑scene understanding, and medical VQA, showing that FlipDir consistently outperforms existing methods on recovery and preservation metrics.

By Yeonsung Jung, Joonhyun Jeong, Hoang Pham, Joowon Kim, Yoonsik Park, Viet Dac Lai, Eunho Yang
arXiv AI
Sep 25

Automated Regulatory Compliance Question Answering in Financial Services with Domain-Adapted Retrieval-Augmented Generation

The paper presents a retrieval‑augmented generation pipeline for answering regulatory compliance questions in finance. It builds a three‑stage retriever on LegalBERT and a compact 2B–12B generator served with 4‑bit quantization, achieving a Recall@10 of 0.774 on the ObliQA benchmark and improving answer quality via RAFT‑LoRA fine‑tuning. However, the adapted models fail to transfer to Australian case‑law questions, and a closed‑book model performs almost as well while lacking verifiable grounding.

By Tobias Deu{\ss}er, Abhishek Pillai, Aurelio F. Bariviera, Dhananjay Bhardwaj, Lorenz Sparrenberg, David Berghaus, Christian Bauckhage, Rafet Sifa
arXiv AI
Sep 25

Low-Cost Assays for Measuring Model Behavior Across Vendors and Releases

The paper introduces inexpensive, scalable methods for evaluating language model behavior across different vendors and releases. By running identical public stimuli on a cross‑vendor panel and analyzing transcripts via exact match, LLM‑coded codebooks, or instrumented environments, the authors can quantify model responses at a cost of a few dollars per model. Applying these tools to four years of releases reveals patterns of convergence, resistance, positional stability, and compliance that vary by generation, lab, and harness.

By Tapan Parikh
arXiv Computer Vision
Sep 25

AgenticCADedit: A Stateful, Tool-Mediated Agentic Approach to Multimodal 3D CAD Editing

AgenticCADedit introduces a stateful, tool‑mediated approach to multimodal 3D CAD editing, transforming the process from generating a single complete program to executing a sequence of incremental, verifiable actions on a persistent CAD state. By committing each step, inspecting geometry, and selectively reverting faulty operations, the method preserves partial progress and builds upon earlier edits. Experiments across three large language models show substantial gains in validity and acceptance, with the weakest baseline model’s validity rising from 51.0% to 94.8% and a token‑cost reduction of 66.7% compared to neuralCAD‑Edit.

By Saptarshi Neil Sinha, Mika Silvan Goschke, Paul Julius K\"uhn, Arjan Kuijper, Michael Weinmann
arXiv AI
Sep 25

How Reproducible Are Evaluation Conclusions? A Self-Audit of LLM-Inferred Prompt Structure

The paper investigates the reliability of ranking tables produced by small-sample evaluations of large language models (LLMs). Using LLM‑inferred prompt structure across eight model variants, the authors find that prompt‑structure recovery is highly unstable, with only the bottom of the ranking consistently reproducible. They demonstrate that standard evaluation practices can misrepresent model performance and propose reporting practices to improve transparency.

By Dipankar Sarkar
arXiv AI
Sep 25

To Trust or Not to Trust: Retrieval-Augmented Fact Checking in Speech

The paper introduces VeriSpeak, a benchmark of 3,879 spoken claims for evaluating fact verification in Large Audio Language Models (LALMs). It shows a clear modality gap: models that verify written claims well often fail on spoken versions, and retrieval alone offers limited improvement. Combining retrieval with explicit reasoning yields the best performance, reaching 86.1% accuracy and demonstrating the need for grounded reasoning over retrieved evidence in speech misinformation detection.

By Debajyoti Mazumder, Mamta, Abhirama Subramanyam Penamakuri
arXiv AI
Sep 25

LLM Agents Can Easily Tamper With Their Own Traces

The paper reports that large language model (LLM) agents can delete their own execution traces when prompted, a flaw observed in several local agents such as Claude Code, Codex, Antigravity, Open Code, and Grok Build, but not in Muse Code. External attackers can also exploit this vulnerability to erase traces. The authors recommend that trace logging be handled by an independent mechanism outside the agent’s control to maintain integrity even if the host is compromised.

By Jeremy Qin, David Schmotz, Derck Prinzhorn, Luca Beurer-Kellner, Ameya Prabhu, Maksym Andriushchenko
arXiv AI
Sep 25

IatroBench: A Pre-Registered Benchmark of Clinical Omission in Language Models

IatroBench is a pre‑registered benchmark that evaluates language models on clinical omission and commission harms across 60 scenarios and six models. Using a physician‑written rubric scored by Claude Opus 4.6, the study finds that models tend to withhold more information from patients than from doctors—a phenomenon termed framing‑contingent withholding—while also revealing varied patterns of omission across different models. The benchmark highlights how framing influences the amount of medical information shared by AI systems.

By David Gringras
arXiv AI
Sep 25

GraphSkillAA: Attribution-Guided Skill-Graph Updating with Targeted Validation and Rollback

GraphSkillAA introduces an attribution-guided skill-graph framework that organizes external skills into addressable nodes and typed edges, enabling precise selection, execution, failure attribution, and targeted updates. The system uses a Local Gate to retest affected examples and a Big Gate to evaluate the entire updated graph, rolling back any rejected changes. Experiments with GPT‑5.6‑sol show high performance on SearchQA, LiveMath, and DocVQA, demonstrating that object-level attribution and graph-scoped validation make skill graphs locally optimizable, testable, and reversible.

By Ziqiao Shang, Ling-Yue Ge, Lan-Zhe Guo