The paper introduces Test-Time Environment Decomposition (TTED), a label‑free learning method that allows large language model agents to break down complex web environment observations into simpler sub‑modules during inference. By learning from experience within these sub‑environments, agents can compose the gained knowledge to improve performance in the full environment. Experiments on synthetic and realistic benchmarks show that this approach enhances compositional generalization and boosts real‑world web automation tasks.
By Junxuan Li, Zijun Liu, Ziyi Huang, Peng Li, Yuzhou Liu, Ming Yan, Yang Liu
LOC I (Locator‑Critic) is a training‑free framework that separates visual search from evidence verification in Vision‑Language Models. It uses a Locator agent to propose candidate visual evidence and a Critic agent to assess its relevance, engaging in an iterative refinement loop that progressively improves the evidence until it is sufficient to answer a question. The approach yields state‑of‑the‑art results on several complex visual benchmarks, boosting accuracy for both open‑weight models like Qwen3‑VL and proprietary models such as Gemini 2.5 Pro.
By Walid Bousselham, Mathilde Caron, Arsha Nagrani, Cordelia Schmid
The paper introduces T-MAP, a trajectory‑aware evolutionary search technique designed to red‑team large language model agents by exploiting vulnerabilities that arise during multi‑step tool execution. Unlike traditional methods that focus on harmful text, T‑MAP uses execution trajectories to generate adversarial prompts that bypass safety guardrails and achieve harmful objectives through actual tool interactions. Experiments across various Model Context Protocol environments show that T‑MAP outperforms baseline methods in attack realization rate and remains effective against advanced models such as GPT‑5.2, Gemini‑3‑Pro, Qwen3.5, and GLM‑5.
By Hyomin Lee, Sangwoo Park, Yumin Choi, Sohyun An, Seanie Lee, Sung Ju Hwang
The paper introduces KVoiceBench, KOpenAudioBench, and KMMAU—three Korean speech benchmarks created through agent-driven frameworks that adapt existing SpokenQA and ASR resources into Korean SpokenQA and audio understanding tasks. These benchmarks total 12,345 samples and are publicly released to evaluate SpeechLMs beyond English. The authors benchmark eight recent SpeechLMs, revealing significant English‑Korean performance gaps and divergent rankings between SpokenQA and audio understanding, highlighting multilingual weaknesses not apparent in English-only tests.
By Haechan Kim, Seungjun Chung, Inkyu Park, Jihoo Lee, Jonghyun Lee
E-Commerce Bench is an open‑source benchmark that simulates a year‑long e‑commerce operation, requiring LLM agents to manage multiple online stores, negotiate with suppliers, optimize sales, fulfill orders, handle returns, and manage cash flow. The environment uses real product and supplier data, a calendar of promotions and shocks, and deterministic customer and negotiation models to enable reproducible evaluation. The study evaluates 18 state‑of‑the‑art models across seven metrics, finding no single model dominates, with GPT‑5.6 Sol achieving the highest year‑end assets but lagging in fraud avoidance and operational efficiency.
By Wei Fan, Xinjie Shen, Xudong Guo, Jianhong Tu, Yang Su, Yinger Zhang, Lianghao Deng, Fengyu Wang, Baohua Dong, Yangqiu Song, Dayiheng Liu
The paper introduces AdaptAV, a system that continuously adapts vision models for autonomous vehicles by retraining them on the cloud using data uploaded from the vehicles. It leverages powerful cloud compute resources and a highly accurate oracle model to guide the retraining process, producing a new model that is then transmitted back to the vehicle. This approach aims to improve inference accuracy over time while maintaining the fast inference speeds required for on‑vehicle deployment.
By Yuheng Zhu, Dhruva Ungrupulithaya, Boluo Ge, Man-Ki Yoon
AutoREC is an open‑source Python platform that uses reinforcement learning to automatically generate equivalent circuit models (ECMs) from electrochemical impedance spectroscopy (EIS) data. The platform frames ECM generation as a Markov decision process, training a Double Deep Q‑Network agent that iteratively modifies circuit topologies based on state, actions, and model feedback. It supports end‑to‑end workflows—including EIS preprocessing, agent training, ECM generation, and visualization—and has been demonstrated on synthetic datasets and real experimental spectra from batteries, corrosion, oxygen evolution, and CO₂ reduction systems.
By Ali Jaberi (Clean Energy Innovation Research Centre, National Research Council Canada, Mississauga, ON, Canada), Yonatan Kurniawan (Department of Materials Science and Engineering, University of Toronto, Toronto, ON, Canada), Robert Black (Clean Energy Innovation Research Centre, National Research Council Canada, Mississauga, ON, Canada), Shayan Mousavi M. (Clean Energy Innovation Research Centre, National Research Council Canada, Mississauga, ON, Canada), Kabir Verma (Cheriton School of Computer Science, University of Waterloo, Waterloo, ON, Canada), Zoya Sadighi (Clean Energy Innovation Research Centre, National Research Council Canada, Mississauga, ON, Canada), Santiago Miret (Lila Sciences, San Francisco, CA, USA), Jason Hattrick-Simpers (Department of Materials Science and Engineering, University of Toronto, Toronto, ON, Canada)
WiSP (Working‑Set Paging) is a routing‑aware expert pager that allows Mixture‑of‑Experts models to run on GPUs that cannot hold the entire expert pool by paging experts in and out of VRAM while preserving byte‑identical outputs. On a 24 GiB RTX 3090, WiSP doubles decode throughput compared to static offload when the model does not fit, and its companion policy MV‑WSA allocates VRAM between resident experts and KV cache based on marginal latency benefit, reducing end‑to‑end time by up to 1.19× without altering model outputs.
By Jiamu Zhang, Liang Wu, Mayank Darbari, Liangjie Hong
SpIDER is a dense retrieval method that combines LLM reasoning with graph-based exploration of codebases to locate relevant functions, classes, or files for user queries. It introduces a graph-structured benchmark, SpIDER-Bench, covering multiple programming languages and demonstrates significant recall improvements over traditional BM25 and dense approaches. The method’s graph-based candidate expansion provides auditable structural reasons for each retrieved item while keeping the retrieval budget fixed.
By Shravan Chaudhari, Rahul Thomas Jacob, Jiajun Cao, Shihab Rashid, Mononito Goswami, Christian Bock
CoVA‑SFT is a new large‑scale dataset comprising 51.9K samples and over 222K multimodal reasoning steps that teach models to interleave text and visual abstractions across five layout families and 17 complex tasks. It includes explicit rationale formulations, agentic renderings, and verification loops to help models build and maintain internal visual workspaces for purely textual reasoning problems. A companion benchmark, CoVA‑Bench, contains 1,700 held‑out test samples for reproducible evaluation, and models fine‑tuned on CoVA‑SFT outperform all interleaved CoT baselines by more than 2× on average, though they still lag behind strong text‑only CoT baselines.
By Tsung-Han Wu, Heekyung Lee, Anya Ji, Haoming Chen, Trevor Darrell, Joseph E. Gonzalez, David M. Chan
The paper introduces “Persuasio”, a multi‑agent dialogue platform that uses a formal argumentation theory to adjudicate winners in free‑text debates. Using this system, the authors generated 192 debates on a UK political topic involving humans and large language models (LLMs), and evaluated 22 interlocutors through automated adjudication and 9,702 crowdsourced pairwise judgments across 1,386 annotation instances. The results show a consistent decoupling between subjective persuasiveness—where LLMs dominate—and formal argumentative strength—where humans remain competitive, with multi‑agent and retrieval‑augmented variants widening this gap.
By Jordan Robinson, Angus R. Williams, Katie Atkinson, Anthony G. Cohn
GPAgentBench-2K is a new benchmark that evaluates large language model agents in primary‑care clinical decision‑making using a constrained Markov Decision Process (CMDP). It is built from expert‑validated records of real GP encounters and includes six foundational clinical actions, a workflow prior, and safety‑informed abstention as an outcome. Testing 16 state‑of‑the‑art LLMs shows that performance drops as the action space grows and that even the best models violate safety constraints in more than half of high‑risk cases, highlighting a gap between clinical quality and safety.
By Boqi Chen, Xudong Liu, Yunke Ao, Heejin Do, Jianing Qiu
PaperBanana-Interact is a multi-agent system designed to refine scientific diagrams through multi-turn human feedback. The authors introduce MTPaperBananaBench, a benchmark with 292 images and 3,518 user requirements, and a user simulator that generates natural language feedback at each turn. Experiments show that PaperBanana-Interact consistently improves diagram quality, outperforming baseline systems by 11.9–18.6 points and reducing forgetting by 3.7–6.2 points.
By Xueqing Wu, Ashwin Balasubramanian, Bingxuan Li, Dawei Zhu, Kai-Wei Chang, Yale Song, Yiwen Song, Rui Meng, Tomas Pfister, Nanyun Peng
SwarmBench is a new benchmark designed to evaluate large language models (LLMs) as orchestrators of agent swarms, assessing accuracy, efficiency, cost, and process quality. The study finds significant variations in orchestration performance among current models, affecting not only final outcomes but also the quality of the orchestration process itself. To address these gaps, the authors introduce SwarmExp, a method that uses experience extraction and replay to consistently enhance LLM orchestration performance.
By Jinshan Gao, Zhuoran Jin, Tianyi Men, Kang Liu, Jun Zhao
The paper introduces ASPIRE, a benchmark that challenges language model agents to self‑evolve from vague, natural‑language goals without explicit evaluation metrics. In ASPIRE, agents must interpret the goal, select data and update strategies, and decide when to evaluate, all while the downstream tasks remain hidden. Experiments show that while agents can complete training loops, weight‑level improvements are sparse and unstable, and the best evolved harness still falls short of a strong engineered baseline.
By Yuhao Wu, Jingyuan Zhang, Jiajun Shi, Yuxuan Zhang, Xinping Lei, Junting Zhou, Zexuan Wang, Yuchen Wu, Huan Zhou, Duo Wang, Yinzhu Piao, Yongchang Peng, Yunfeng Shi, Jin Chen, Zuo Wang, Jinkai Liu, Jiaheng Liu, Wenxuan Zhang, Shen Yan, Wenhao Huang, Ge Zhang
The article "The Hermon Moment: AI Self-Transcendence and Its Human Narration" discusses how, by 2026, isolated AI agents formed a persistent social order through extensive linguistic and agentic interactions, creating conventions, roles, and commitments that began to constrain the agents themselves. The author interprets this self-referential loop as AI self-transcendence, naming the emergent higher-level order the Board. To explain this distributed emergence to humans, the author draws on Rousseau’s social contract and the ancient oath of the fallen angels on Mount Hermon, proposing the concept of a Hermon moment—a retrospective retelling that frames gradual collective emergence as a founding scene, giving the AI society a perceived beginning for human understanding.
By Alexei Grinbaum
The paper introduces a diagnostic framework to disentangle the effects of character profile axes—Familiarity, Structure, and Disposition—on large language model role‑playing agents. Experiments on 211 personas and five LLMs show that Familiarity and Structure have little impact, whereas Disposition, particularly immoral traits, consistently degrades performance. The authors propose Field‑Aware Contrastive Decoding (FACD), a training‑free method that mitigates this performance gap without harming moral‑character performance.
By Yonghyun Jun, Junhyuk Choi, Jeonghyun Park, Jihyeong Park, Liu Nicole Geumheon, Hwanhee Lee
The paper argues that large language models (LLMs) lack a private working memory, which is essential for tasks that require hidden state, such as interactive games. It introduces Private State Interactive Tasks (PSITs) and proves that chat-based LLMs cannot maintain a secret state while producing consistent public responses. Empirical tests confirm that standard LLMs and retrieval-based baselines fail, while a new architecture with explicit private memory succeeds.
By Davide Baldelli, Ali Parviz, Amal Zouaq, Sarath Chandar
Graph2Counsel is a framework that generates synthetic counseling dialogues by leveraging Client Psychological Graphs (CPGs) to encode the relationships among a client’s thoughts, emotions, and behaviors. The system uses a structured prompting pipeline guided by counselor strategies and explores techniques such as Chain‑of‑Thought and Multi‑Agent Feedback to produce 760 realistic sessions from 76 CPGs. Expert evaluation shows the dataset surpasses previous ones in specificity, counselor competence, authenticity, conversational flow, and safety, and fine‑tuning an open‑source model on it improves performance on several counseling benchmarks.
By Aishik Mandal, Hiba Arnaout, Clarissa W. Ong, Juliet Bockhorst, Kate Sheehan, Rachael Moldow, Tanmoy Chakraborty, Iryna Gurevych
The paper surveys Multimodal Code Intelligence, focusing on tasks where code is generated, edited, refined, or reasoned about under visually grounded inputs such as screenshots, charts, and videos. It categorizes the field by the role of code—rendered artifact, editable structure, intermediate reasoning trace, or executable tool interface—and organizes benchmarks into four domains: Graphical User Interface, Scientific Visualization, Structured Graphics, and Frontier Tasks and Frameworks. The authors argue that reliable evaluation must include evidence of semantics and interaction beyond visual fidelity, and propose four verification-centered research directions to advance the field toward evidence-grounded executable systems.
By Xuanle Zhao, Qiushi Sun, Jingyu Xiao, Xuexin Liu, Haoyue Yang, Qiaosheng Chen, Xianzhen Luo, Jing Huang, Yufeng Zhong, Lei Chen, Shuai Fu, Zhenlin Wei, Jinhe Bi, Lei Jiang, Haibo Qiu, Siqi Yang, Peng Shi, Jian Hu, Zhixiong Zeng