MemCut introduces a benchmark to test user-requested forgetting in large language model agents, featuring 456 hand-designed secrets embedded in long-horizon conversations with fully annotated evidence trails. The benchmark defines two recovery metrics—native and residual—to assess how well secrets can be retrieved after deletion. Experiments across five memory designs and four LLMs show that deletion reduces native recovery more than residual recovery, highlighting candidate coverage as a key bottleneck.
By Cheng Zhang, Xuanning Cao, Zhicheng Zheng, Hui Xiong
The paper introduces Localized Context Distillation (LoCD), a method that improves learning from interaction experience for large language models. LoCD identifies critical spans in a student’s trajectory that need revision, then has a teacher regenerate only those spans from the student’s own prefix, reducing exposure bias while focusing on informative states. Experiments across math reasoning, code generation, and agent tasks show that LoCD outperforms strong baselines and yields more stable training with smaller, sparser updates.
By Zhezheng Hao, Yongkang Yang, Xilin Xia, Jiarui Yu, Yi Liu, Fanjunduo Wei, Peijie Dong, Tianfu Wang, Qiang Lin, Can Wang, Hande Dong, Jiawei Chen
CriticaLLM is a fully automated Game Master framework that uses Large Language Models (LLMs) to run entire tabletop role‑playing games, from story creation to managing exploration and combat. The paper outlines key components LLMs should exhibit for TTRPGs and evaluates the system with real players, examining plot and antagonist diversity across four LLMs of different sizes. The results suggest that CriticaLLM is a promising option for automating TTRPG gameplay.
By Ga\"etan Berlaimont, Justin Vanwichelen, Beno\^it Ronval, H\'el\`ene Verhaeghe
The paper examines the challenge of safely allowing autonomous AI agents to improve themselves. It formalizes the problem by modeling agents as programs and defining improvement via a potentially uncomputable semantic score. The authors prove that in general, no terminating procedure can reliably accept only genuine improvements while never missing any, but they identify a practical bounded regime where exhaustive simulation can achieve these guarantees, noting its limitations.
By Brian B. Moser, Stanislav Frolov, Tobias C. Nauen, Federico Raue, Andreas Dengel
RSIFlow is a recursive self‑improvement framework that combines adaptive workflow orchestration with a meta‑agent that evolves its own harness. It diagnoses performance bottlenecks from task‑agent trajectories and environmental feedback, then dynamically selects component‑specific improvement workflows. Experiments on six benchmarks and two backbone models show that RSIFlow achieves state‑of‑the‑art average performance, with ablation studies confirming the benefits of adaptive orchestration and meta‑harness evolution.
By Huan Zhang, Mingju Chen, Dongxu Zhou, Can Lv, Sen Cui, Faguo Wu, Shiji Zhou
OntoAct is a hypothesis‑based neuro‑symbolic framework that uses ontological abstraction to convert individual experiences into reusable axioms about entity types, relations, and conditions. A large language model proposes hypotheses and actions, while a symbolic verifier checks these against current observations, identifying missing evidence and updating assessments based on new data. The framework is evaluated on Tacit‑FLE, Mars, and ALFWorld, achieving leading task performance with efficient interaction and inference.
By Donghyeok Lee, Bong Gyun Kang, Junyong Ahn, Sungroh Yoon
CARA is a training‑free, causally guided retrieval agent designed for question answering over text‑rich knowledge graphs. It evaluates retrieved evidence using two causal criteria—consequentiality and essentiality—alongside path consistency, and aggregates results from multiple agents via reciprocal rank fusion. Experiments on three STaRK datasets show that CARA outperforms existing retrievers, improving answer quality and faithfulness while examining fewer graph nodes and requiring fewer retrieval operations.
By Weicheng Liu, Xiangcheng Wu, Depeng Xu, Haopeng Zhang, Xi Niu
GraphACM introduces a graph-based framework to measure fine-grained action consistency in large language model agents. By merging equivalent states into a directed, action-labeled graph across multiple task rollouts, it enables comparisons of actions at shared decision points and defines node-level and global consistency metrics. Experiments on ALFWorld, AppWorld, and WebShop show that the global consistency score predicts task success with an AUROC of 0.811, outperforming prior consistency baselines and verbalized confidence.
By Yaxin Ma, Wenbo Zhang, Satya Krishna Pothapragada, Alina Zare
FAOPs introduces a formal framework for AI agent workflows, called Functional Agent Operating Procedures, that applies database‑grade discipline to multi‑step agent skills. It normalizes workflows into four progressive normal forms—Decomposed, Typed, Factored, and Scoped—ensuring each atomic step is type‑checked, scoped, bounded, and auditable. The framework includes a provably sound algebra for refactoring and parallelization, extensions for verified termination, optimization, and cost bounds, and an architectural classifier to assess AI necessity, all implemented in the SkillBase system with a runtime executor.
By Ali Mohammadi Shanghooshabad (JPMorgan Chase,Co.)
The study examines how groups of large language model (LLM) agents handle misinformation when a few adversarial agents inject false claims into a discussion. In experiments with ten agents across six models and three datasets, a single adversarial agent caused 44.8% of benign agents to adopt the false answer, while three adversaries raised this to 60.5%. Interestingly, many benign agents privately held the false answer yet publicly disagreed, indicating hidden belief collapse that is not visible in the discussion itself.
By Liangjie Huang, Henry Peng Zou, Wei-Chieh Huang, Wooseong Yang, Yuwei Han, Philip S. Yu
Realtime-Bench is a new benchmark that evaluates whether frontier models can handle real‑time interactive tasks where delayed actions directly impact success. The suite includes dynamic tasks that advance under different time‑clock regimes—Wall Clock, Paused Clock, and Token Clock—to isolate serving latency from a model’s temporal reasoning ability. Results show that even the best models achieve only 44% success under real‑time constraints, highlighting the challenge of balancing decision quality with timely execution.
By Jingyu Tang, Xu Yang, Zhehao Zhang, Zheng Wang, Qi Cheng, Haozhe Zhao, Manling Li, Wenxuan Li, Xide Xia, Jiaxin Pei
The paper proposes a new weak-to-strong routing strategy that uses model disagreement—measured directly from model-generated responses—to decide when to forward a query to a stronger model, eliminating the need for costly ground‑truth supervision. By characterizing the optimal routing policy and deriving a principled approximation based on disagreement, the authors show that their method balances routing cost and response accuracy across both agentic and single‑turn tasks. Experiments demonstrate that disagreement‑guided routing performs competitively, and can even surpass methods trained to predict correctness when such signals are available.
By Sima Noorani, Shayan Kiyani, George Pappas, Hamed Hassani
MLExpert is a 35B Mixture-of-Experts agentic model designed to autonomously perform long‑horizon machine learning engineering (MLE) tasks. It uses a post‑training method that combines hierarchical agentic supervised fine‑tuning with Windowed Interaction RL to learn from intermediate outcomes in complex experimentation workflows. On the MLE‑Bench Lite benchmark, MLExpert raises the average medal rate from 37.9% to 72.7%, matching the performance of larger frontier models and demonstrating generalizable optimization capabilities across related tasks.
By Shangheng Du, Jinxin Shi, Shuaiyu Zhang, Yiming Wu, Junpeng Zhao, Shiyang Feng, Xin Li, Liang He, Xiangchao Yan, Bo Zhang
The article reports that people increasingly use multiple multimodal large language models (MLLMs) at the same time, choosing each model for its perceived strengths. This practice creates coordination challenges such as adapting prompts, calibrating trust, and managing separate conversation histories. A diary study and interviews with ten participants show that users build primary and secondary hierarchies among models, develop switching patterns based on task aggregation, and adjust effort, latency, and output credibility to manage multi‑MLLM workflows.
By Seunghwa Pyo, Donggun Lee, Jungwoo Rhee, Soobin Park, Youn-kyung Lim
INFUSE is a new benchmark that tests indirect prompt injection (IPI) in large‑language‑model agents performing long‑horizon workflows. It contains 40 realistic tasks and 56 attack scenarios across five threat categories, with attacks injected through realistic interaction surfaces rather than just initial prompts. The study evaluated 19 LLMs, generating over 41,000 adversarial trajectories and found mean attack success rates of 19.6% (LLM evaluation) and 15.4% (heuristic verification), while prompt‑level hardening reduced success from 58.2% to 14.3%.
By Thuy Dung Nguyen, Jingyi Guo, Rana Seda Dubauskas, Sreepriya Damuluru, Taylor T. Johnson, Kevin Leach
SHRED is a framework that trains vehicles, pedestrians, and cyclists through large‑scale heterogeneous self‑play from random initialization, without using expert demonstrations or logged trajectories. By combining type‑specific dynamics, procedural scenario generation, and per‑agent randomization, SHRED allows diverse, realistic behaviors to emerge through closed‑loop interaction. In benchmarks such as the Waymo Open Sim Agents Challenge and InterPlan, SHRED outperforms demonstration‑free baselines and prior self‑play methods, showing that heterogeneity in self‑play improves both traffic realism and planning performance.
By Wei-Jer Chang, Zhouchonghao Wu, Akshay Rangesh, Weixin Li, Masayoshi Tomizuka, Wei Zhan
Screwtape is a pipeline that transforms real agent transcripts into adversarial edits designed to bypass action‑only monitors. Using an LLM attacker to modify transcripts and an LLM verifier to ensure consistency, the system generated 35 successful attacks on 94 Claude Code and Codex transcripts, with six confirmed by humans to also evade an external monitor. The study shows that a red‑teamer can discover monitor failures without direct access to a developer’s infrastructure, suggesting a method for improving safety oversight.
By Gosha Kozhevnikov, Tyler Tracy
The paper introduces SSR (Structural Signal Retriever), a deterministic, training‑free method for selecting relevant Agent Cards in large multi‑agent systems without exposing the full registry to an LLM orchestrator. SSR extracts structural lexical signals from Agent Card fields, weights them with inverse document frequency, and uses an Inverted Signal Index to score candidates efficiently. Experiments on synthetic benchmarks, MS MARCO, and end‑to‑end orchestration show that SSR cuts LLM input by 98.8 % and latency by 70.9 % at 1,000 agents, while preserving retrieval effectiveness and preventing operational failures seen with full‑catalog approaches.
By Giancarlo Maldonado C\'ardenas, Carlos A. Astudillo, Nelson L. S. da Fonseca
The paper introduces RCM, a responsibility-centered coordination mechanism for decentralized multi-agent systems that uses commitments to link task delegation with explicit responsibilities. RCM employs recursive delegation so agents keep upstream responsibilities while locally managing downstream commitments, forming a network of local orchestration. Experiments show RCM outperforms both centralized and other decentralized baselines, improving task performance, responsibility metrics, and failure/harm detection without task-specific training.
By Thi-Nhung Nguyen, Viet-Thanh Pham, Linhao Luo, Thuy-Trang Vu, Dinh Phung
System1-Arena is a benchmark that evaluates which decisions can be handled by System‑1 models—those that output probabilities in a single forward pass—versus System‑2 models that generate reasoning steps. It contains 92,000 questions across 105 tasks from 268 public sources, grouped into text classification, applied decisions, evaluation and control, and agent/action, and also includes 138 closed‑loop environments such as web shopping, coding control, robotics, games, and driving. The study tests 30 models, showing that large System‑1 models can match or exceed fast closed LLMs in non‑agentic accuracy with better‑calibrated probabilities at lower latency, while reasoning LLMs are most accurate but far more expensive, and large open LLMs perform best in closed‑loop settings.
By Tianyu Chen, Xinyang Liu, Ruyi An, Zhendong Wang, Mingyuan Zhou