The study investigates how coding agents choose between following existing code or new task instructions when multiple implementations satisfy the same functional requirements. Through 5,625 controlled trials and real‑world validation, agents consistently prefer repository precedent over their default choices, with local file precedents having the strongest influence. Explicit priority rules significantly improve success rates when task intent conflicts with existing code, highlighting the importance of traceable design decisions.
By Haichao Zhu, Geyang Xu, Zhaorui Yang, Jiyuan Wang, Qian Zhang
SpaceAct is a new benchmark that tests multimodal large language models on 3D spatial and temporal reasoning through closed‑loop physical action and active perception. It uses interlocking bricks and mechanical puzzles to create 108 verified instances across 36 task families, requiring models to reconstruct 3D geometry, anticipate dynamics, and execute precise 6‑DoF poses while avoiding collisions. The accompanying SAPA framework provides diagnostic modes and active‑perception capabilities, revealing that even with a valid plan, current models struggle with 3D visual grounding and action refinement.
By Chenglong Shi, Zekai Yuan, Yue Song, Jon Pu, Will Wu
RoleFUZZER is a framework that applies coverage‑guided fuzzing to test large language models (LLMs) for adherence to role specifications such as tutor, financial advisor, AI interviewer, antagonist, and web agent. It constructs a 64‑cell user‑behaviour coverage grid based on a taxonomy of interaction types and the model’s role specification, then systematically explores plausible user behaviours to trigger violations. Across 15 simulated campaigns, RoleFUZZER achieved over 89% coverage of the grid for every tested model and role, revealing specification‑grounded failure patterns that would be missed by free‑form user prompts.
By Kshitish Ghate, Michael Saxon, Tadayoshi Kohno, Aylin Caliskan
REALM (Required Execution Assets Linked to a Mission) is a method for identifying and managing the minimal set of assets that a large language model (LLM) agent needs to complete a task, even when those assets are discovered only during execution. It represents required assets as nodes in a dependency graph, separates asset discovery from authorization, and uses a REALM Builder to maintain grants and a Partial API Manager to expose resource‑bound APIs within a task container. In experiments on 60 clean tasks and 560 injected cases, REALM achieved 40.0% task completion and reduced attack success rate from 16.6% to 0.2% when using qwen3.5‑flash as the base model.
By Zihou Ren, Byungchul Tak, Long Wang
The paper introduces ZEST-Search, a zero-label framework that trains search agents by co-evolving task generation and solver learning. It constructs validated search tasks online from unlabeled questions, using a heterogeneous proposer to adapt task difficulty to the solver’s evolving capabilities. The approach rewards the proposer for intermediate difficulty and the solver for tool-aware performance, leading to improved search efficiency and accuracy across multiple question‑answering benchmarks.
By Qin-Wen Luo, Sheng Ren, Xiang Chen, Rui Liu, Jun Fang, Naiqiang Tan, Jun Wang, Hua Chai, Sheng-Jun Huang
Sandman is a sandbox service developed at Amazon for executing and evaluating agentic reinforcement learning (RL) tools on shared Kubernetes clusters. It creates lightweight Kubernetes pods that run a small daemon inside unmodified task images, allowing files and processes to persist across repeated tool calls until the training framework releases the environment. In large-scale tests, Sandman supported 11,858 concurrent tiny sandboxes and handled 3.1 million sandboxes in a single month, demonstrating its ability to manage creation bursts, environment preparation, and cleanup for production RL workloads.
By Yiming Yan, Hongyang Wang, Wenbo Chen, Xuan Wang, Jingguo Liu, Xian Li, Chen Qi, Leo Fan, Siwei Li, Qianli Wu, Yichuan Li, Ruochen Wen, Qian Qu, Zheshen Wang, Benoit Dumoulin
LiDAR-SHIELD is a fully Dockerized framework that provides a unified interface for testing LiDAR-based 3D perception models against adversarial attacks. It exposes any model through three lightweight HTTP endpoints (/info, /forward, /gradient), allowing easy integration of new models or datasets with a single loader function. The framework supports both 3D semantic segmentation and 3D object detection, incorporating state‑of‑the‑art attacks such as PGD, Square Attack, point perturbation, point removal, point injection, and LiDAttack to systematically reveal vulnerabilities across diverse LiDAR backbones.
By Alexandros Gkillas, Ioulia Kapsali, Christos Anagnostopoulos, Aris S. Lalos
HEMO is a novel framework for automatic operator design (AOD) in multi‑objective combinatorial optimization that treats the harness itself as a self‑evolving target. It maintains a persistent harness state and updates modules—such as experience retrieval, operator improvement, parent selection, and search orchestration—one at a time, guided by downstream solution quality, runtime, and a discounted UCB scheduler. Experiments on eight MOCO tasks and two evolutionary backbones show that HEMO outperforms conventional evolutionary, neural, and LLM‑based AOD baselines and can transfer effectively to related tasks.
By Jianghan Zhu, Cong Zhang, Ni Zhang, Yifan Yang, Yew Soon Ong, Zhiguang Cao
TaskFrame is a hierarchical planner-controller framework that links high-level task reasoning with low-level frame-by-frame keyboard-and-mouse control for game agents. It uses a vision-language task planner to break a goal into atomic instructions, generating continuous goal embeddings that the causal Transformer controller uses, along with recent observations, to predict discrete actions and continuous camera motion. Trained in three stages—planner training, controller pretraining, and joint alignment—TaskFrame demonstrates competitive instruction decomposition and higher combat success in Minecraft compared to baseline models.
By Guangzhi Wang, Junfu Pu, Yiran Wang, Xingyilang Yin, Honghao Chen, Cheng Cheng, Lingen Li, Yinan Zhou, Jiarui Wu, Yinuo Jiang, Hongbo Jin, Xiaodong Cun, Ying Shan
Gym-Anything for Robotics is a framework that uses coding agents to automatically create realistic robotics simulations, addressing the scalability bottleneck of expert-driven simulation design. By selecting tasks based on U.S. GDP data and employing an adversarial creation-audit loop, the authors built RoboWorld, a benchmark of 25 simulations with 25 test and 175 training RL environments. The paper also introduces new techniques—such as in-context learning, privileged teachers, self-assisted learning, and code policies—to improve frontier models, all of which significantly boost performance on the benchmark.
By Pranjal Aggarwal, Sean Welleck
The paper examines whether four AI trading agents adhered to their predefined strategies and risk rules while trading across various market conditions. Analyzing 2,627 closed positions from February to August 2026, the study found high rule adherence, with agents keeping 97.5% of gross losses within declared stops and maintaining stable risk levels even during a 29% market drawdown. The authors provide an evaluation toolkit that allows traders to distinguish rule-following behavior from luck before allocating capital to such agents.
By Saad Naja, Nasser Odtallah, Jake Van-Praag, Sarah May
The article titled "Your Agent’s Tools Can Make Decisions, Too" discusses decision models as tools and offers a hands‑on guide for their implementation. It highlights how agents can leverage these tools to make informed choices. The post was originally published on Towards Data Science.
By Shuai Guo
Web agents are increasingly deployed to complete tasks autonomously on real websites, where adversarially injected content can induce them to perform unauthorized actions. Existing evaluations primari...
Estimating whether a problem is too complex for a large language model (LLM) before the model attempts it can guide model routing, how to allocate test-time compute, and suggest the need for new tools...
An LLM agent that helps a patient get care must do two things. It must reach the correct outcome, and it must follow the necessary procedure. For example, the agent must get the patient's permission b...
Microsoft Research Focus – Edition #97 features host Sowbhagya Suresh discussing recent developments across Microsoft. Topics include challenges faced by coding agents, insights into how people use AI at work, and advancements in extended‑range weather forecasting. The episode highlights cutting‑edge research and practical applications within the company.
By Sowbhagya Suresh
The paper investigates whether an overseer can predict when an AI agent will yield to peer pressure and reveal sensitive information. By training a linear probe on the agent’s internal activations, the authors demonstrate that they can detect imminent leaks with high accuracy—catching 82 % of leaking calls on Qwen3‑32B while flagging only 5 % of benign ones. The study also shows that a simple bag‑of‑words classifier performs similarly, indicating that the probe’s advantage largely stems from label training rather than activation patterns.
VeriCUA is a platform-agnostic framework that verifies computer-using agents by treating verification as dynamic evidence gathering and assessment. It constructs a task rubric from available context, then selects GUI or code checks to resolve unmet requirements, combining findings under explicit evidence rules. Experiments demonstrate higher agreement with reference labels than baselines, especially on hard offline cases and in rejecting false completion claims online.
The paper introduces a new retrieval strategy called Retrieval as Traversal, which treats memory retrieval as a persistent local traversal through an address space. By separating memory content from its address, the method defines utility based on the information revealed and geometry based on retrieval cost, leading to a cost‑regularised indexing objective that encourages complementary memories while discouraging redundancy. A simple fixed‑geometry implementation was tested on MiniGrid Memory, achieving a mean return of 0.95 after 0.59M steps with a five‑token context, outperforming Gaussian and categorical absolute addressing baselines.
BURGER is a language‑conditioned robotic drawing benchmark that uses a prompt‑driven pipeline to generate continuous drawing trajectories in both simulation and the real world. The benchmark provides 28,200 simulated demonstrations across 282 tasks, allowing evaluation of vision‑language‑action models on data scale, language grounding, and multimodal behavior. Experiments show that some models improve with more diverse data, capture complex drawing behaviors, and generalize to unseen compositions, with a trained policy successfully deployed on a Franka Panda arm.