Omni-InteractScope is a new benchmark comprising 1,479 human‑annotated interactions and 1,790 temporal targets across eight task groups, designed to evaluate how well omnimodal large language models (Omni‑LLMs) connect continuous audiovisual streams to user needs and deliver timely, sustained interventions. The benchmark links input modalities, event triggers, intervention types, and user needs, covering event‑timed tool invocation, query‑free assistance, notifications, warnings, state tracking, and continuous responses. Experimental results on this benchmark reveal three critical limitations of current Omni‑LLMs: unreliable timely tool invocation, poor coverage of query‑free reminders, and limited end‑to‑end guidance coverage.
whyItMatters":"The benchmark highlights specific performance gaps in Omni‑LLMs that are essential for developing streaming assistants capable of timely action, proactive initiative, and comprehensive response coverage."
By Yinsong Yan, Ling Wang, Yuxuan Wang, Yifan Yang, Ruiyang Xu, Junming Lin, Qi Chen, Meng Gao, Minghao Han, Yunfei Chu, Qize Yang, Xize Cheng, Jin Xu, Jibin Wu
The paper investigates how trajectory-based confidence estimation for large language models deteriorates after agentic reinforcement learning (Agentic RL) due to shifts in reasoning trajectories. It discovers that more stable trajectories consistently correlate with correct answers, a relationship that remains intact across both base and RL models. Leveraging this insight, the authors introduce StaR-TTA, a stability-guided test-time confidence adaptation framework that learns from labeled base trajectories and applies stability-derived reliability rankings to unlabeled RL trajectories, achieving improved AUROC scores across multiple QA benchmarks.
By Zhenzhen Ren, Xiaoqing Zhang, GuoBiao Li, Bowen Fang, Yanxi Wang, Xinpeng Zhang, Ke Han
ShutdownMod is a new method that transforms any agent trajectory into a shutdown evaluation by inserting a shutdown request into the agent’s context and recording its response. It equips LLM agents with a signature‑verification tool so they can differentiate between trusted interventions and unauthorized injections. The authors evaluated seven models on 50 tasks from SWE‑bench, Terminal‑Bench 2, and SWE‑chat across six shutdown scenarios, finding varying non‑compliance rates—up to 11.7% for Qwen 3.8 27B—yet no model was fully compliant in all settings.
By Param Biyani, Sebastien Luyasu, Cedric Potvliege, Divij Chawla, Elliott Thornley, Christos Ziakas
The paper introduces the Infinity Machine, a self‑improving AI agent designed to solve research problems whose solutions can be programmatically verified. The system operates autonomously: it generates candidate solutions, verifies them, and audits the results without human intervention, all within a sandboxed environment governed by a kernel‑enforced gate. In a six‑day experiment, the Infinity Machine achieved four new state‑of‑the‑art results in quantum information and improved qubit‑routing and hardware efficiency, while also documenting several failure modes.
By Harish Santhanalakshmi Ganesan
arXiv:2610. 35767v1 Announce Type: new Abstract: Root cause analysis (RCA) of microservice failures has attracted many LLM agents whose investigation is hand-crafted, through standard operating procedures, fixed stages and role agents.
By Tingzhu Bi, Xun Zhang, Jinglin Li, Ping Wang, Meng Ma
The paper introduces DECE, a unified arena that spans ten Eurogames with shared interfaces and evaluation protocols to assess large language model agents in competitive settings. It evaluates 12 model-harness configurations across 480 three-player matches, revealing consistent cross-game ranking agreement but also game-specific strengths and variations due to execution harnesses. The study highlights that competitive performance does not always align with token or cost efficiency, underscoring the importance of considering resource usage when comparing agent configurations.
By Fengrui Zhang, Xiang Li, Ruomeng Ding, Jiayi Yuan, Yuchen Huang, Zhaoyang Han, Xiaoliang Wang, Daoyuan Chen, Yaliang Li
The paper introduces the concept of Cyber Social Agents (CSAs), automated actors on social media defined by a taxonomy of archetypes based on operational tactics and rhetorical style. It integrates this taxonomy into a large language model (LLM)-driven simulation where agents interact within a network, allowing the study of how diverse agent types influence information diffusion and social influence. The approach bridges rule‑based agent models and LLM‑powered simulations by parameterizing generated agents with empirically grounded CSA personas.
By Lynnette Hui Xian Ng, Kathleen M. Carley
The paper investigates how coding agents handle large repositories when their context window is limited. It compares three strategies—prompt compression, code retrieval, and a simple file selector called CQ2-CARE—finding that allocating the token budget to selecting whole files verbatim enables the agent to fix more issues. CQ2-CARE outperforms both compressors and retrievers on a benchmark of 300 recent software‑engineering problems, achieving higher success rates with GPT‑5.6 and Claude Opus 5.
By Xiaocong Yang, Rishit Chatterjee, Himanshu Udupi, Vikram Ramavarapu, ChengXiang Zhai
RADAgent automates the entire RADseq workflow by dividing it into 16 modular agent modules that can be executed and verified independently. It uses an open-weight large language model to coordinate heterogeneous bioinformatics tools, manage state tracking, validate results, and recover from failures, all within a local computing environment. The system was evaluated on a real population‑genomics dataset of 476 samples from 19 populations, demonstrating reliable, end‑to‑end automation of RADseq analysis.
By Qiming Guo, Malaka Sudhakara Bava Sukruth Reddy, Honggang Zhao, Ye Wang, Chen Pan, Wenlu Wang
The paper introduces Recursive Semantic Divergence (RSD), an unsupervised, trajectory‑level consistency metric for large language model agents that use tools. RSD recursively measures expected cumulative semantic disagreement between two independent rollouts from the same state, using natural language inference to detect logical inconsistency. When applied as a reward signal during fine‑tuning, RSD significantly reduces variability in task progress across benchmarks and model scales while matching supervised fine‑tuning performance without needing ground‑truth labels.
By Harshavardhan Abichandani, Penny Chong, Atin Ghosh, Daniel Dahlmeier
The paper introduces StyleInstructCaps, a benchmark for controllable speaking‑style captioning (SSC) in audio‑language models. SSC generates natural language descriptions of speech delivery, capturing paralinguistic attributes such as vocal timbre, prosody, and expressivity, and requires dimension‑specific style captions rather than a single undifferentiated description. The benchmark provides a dataset and evaluation framework that assesses metadata groundedness, hallucination, instruction‑following ability, generalization to unseen instructions and audio datasets, and speaker‑style consistency, and shows that instruction‑diverse training improves groundedness, controllability, and speaker consistency across nine state‑of‑the‑art models.
By Shreeram Suresh Chandra, Ashwyn Sharma, Liang Lu, Xiutian Zhao, Leda Sari, Bj\"orn Schuller, Berrak Sisman
The paper introduces MARC, a Mechanism-Aware Reflective Correction system for generating articulated objects from images and text. MARC shares an executable kinematic graph across program construction, motion solving, and correction, enabling the system to diagnose and repair errors in closed‑chain and coupled mechanisms. On a 20‑object test set, MARC achieves 67.8% valid motion coverage and 60.0% overall success, outperforming six baselines and improving scores by 13.7 and 20.0 percentage points after correction.
By Xiong Yang, Junkun Hao, Zijian Xiao, Zipeng Ye, Yuchen Xie, Ran Yi
The paper introduces Zafin AIOS, an orchestration platform that enables AI-assisted software development in regulated finance. It presents an empirical comparison of AI-assisted versus human-led workloads and validates the framework through a Customer Zero case study on two production codebases—pricing execution and client deal negotiation. Results show that governed agentic orchestration boosts enterprise capacity, providing full observability, reducing effort by 38 % in pricing workflows, and cutting reporting work by 76 % in negotiation workflows while preserving data privacy, role-based access, and audit guardrails.
By Shahir Daya, Rob Dunlap, Alexandre Boulenger, Ahmed Abdelmoneim, Salvatore Vella
The As-If Machine (AIM) is a multi‑agent AI system that lets participants co‑write personalized counterfactual narratives about their own lives under different policy futures. In two large experiments—one on science funding and one on satellite cybersecurity—AIM significantly increased participants’ psychological closeness to the risks compared with expert vignettes and unrelated writing, with effects ranging from 0.36 to 0.90 standard deviations. Across 23,496 human‑AI interactions, AIM provided concrete detail while participants supplied first‑person perspective and emotion, demonstrating that co‑writing with AI can make distant policy risks feel immediate and personal.
By Joshua Ashkinaze, Yubo Zhou, Pengyu Chen, Avi Bhattacharya, Hyebin Park, Yanman Zhang, Stephanie Preston, Eric Gilbert, Ceren Budak
The paper introduces Enclawed, a deterministic security runtime that protects large language model agents running on high‑performance computing (HPC) systems. It provides hardware‑rooted supervision for every rank, preventing compromised hosts from causing catastrophic failures, and enforces launch policies, lease‑based heartbeats, and automatic eviction or killing of tampered jobs. Enclawed operates without administrator privileges, integrates via Secure Shell, and adds minimal startup overhead while successfully blocking a wide range of simulated attacks.
By Alfredo Metere
The paper introduces SecPRBench, a benchmark that evaluates large language model agents on real-world bug‑introducing commits from critical software systems such as Firefox, V8, and OpenSSL. It automates harvesting, building, and validating these commits, and uses an LLM‑as‑a‑judge to match findings with ground‑truth vulnerabilities. Across eight state‑of‑the‑art models, recall ranges from 32.5% to 78.1%, with Grok 4.6 and GPT‑6 sol performing best, while security‑focused harnesses like OpenAI’s Codex Security and Anthropic’s Defending Code influence recall differently.
By Yihan Yang, Chenyuan Yang, Zijie Zhao, Anh Nguyen Hoang, Ziqi Zhang, Weidong Wang, Zhiheng Lyu, Lingming Zhang
PatEx-Gym is a new framework for developing and evaluating agents that perform patent examination. It includes PatEx-Bench, a benchmark of 1,000 realistic tasks with large retrieval corpora, and PASS, an annotation‑free method for synthesizing search trajectories. A 4B‑parameter PatEx‑Agent trained on these synthetic trajectories provides a lightweight baseline that competes with much larger general‑purpose models, yet prior‑art discovery remains challenging.
By Qiyao Wang, Xinyi Chen, Hongbo Wang, Jiayuan Lu, Jianhang Li, Minghuan Tan, Yuan Lin, Min Yang
RSI Arena is a two‑stage experiment where ten large language models, each starting from NVIDIA Nemotron 3.5 Lightning 30B-A3B Base, autonomously train and refine themselves using human feedback. In the first six‑day stage, each agent receives 1,000 GPU‑hours and $300 in API credit to select data, design training procedures, and choose checkpoints, producing working assistants that close a median 76% of the base‑to‑Instruct gap on 12 benchmarks. The second stage gives agents access to comparison records and an additional three days of training, resulting in style‑adjusted relative rating improvements for eight of the ten models, though benchmark gains remain modest and some agents exhibit longer responses or increased harmful compliance.
By Yue Huang, Yanbo Wang, Lichi Li, Yuanyuan Chen, Andreas Haupt, Xiaonan Luo, Zheyuan Liu, Jihye Choi, Felix Juefei-Xu, Xiangliang Zhang, Radha Poovendran, Misha Sra, Ying Liu, Ben Burtenshaw, Thomas Wolf, Alex Pentland, Zichen Chen, Zhangchen Xu
The study compares a custom-built LLM agent harness with two general-purpose harnesses—Claude Code and OpenCode—used for testing Volvo Group’s Android-based in‑vehicle infotainment GUI. On Volvo’s stepwise IVI tests, the general-purpose harnesses achieve similar success rates but with higher inference costs, yet they reduce overall monthly costs by 3–66% due to lower engineering effort. In open‑ended public‑app tasks, the general-purpose harnesses perform better, recovering more effectively from incorrect actions.
By Arsham Gholamzadeh Khoee, Ruyu Fu, Jiaqi Wei, Ruikun Deng, Robert Feldt, Dhasarathy Parthasarathy, Yinan Yu
The paper investigates why an LSTM agent outperforms a memory‑free MLP on a partially observable XLand‑MiniGrid task. By separating training seed, map, and evaluation randomness across 60 runs, the authors find that much of the 25‑percentage‑point gap is due to noise, and that providing the time step to a memory‑free MLP reduces the gap to a few points. A linear probe shows the LSTM’s state largely encodes the time step, indicating that the apparent memory advantage is largely an advantage of knowing episode time.
"whyItMatters":"The study clarifies that the perceived memory benefit in this benchmark is largely attributable to time‑step awareness rather than true memory, guiding future baseline design and evaluation practices."
By Rodrigo Reveron-Fonseca