The paper introduces a layered evaluation protocol for generative scenario models used in autonomous driving, focusing on physical consistency and plausibility. It examines internal representations through kinematic alignment, statistical baseline comparison, latent controllability, and activation analysis, and then tests outputs against vehicle dynamics constraints such as lateral jerk thresholds. The protocol is applied to a VAE-based scenario generator and other generative models, revealing deeper insights than standard output-level metrics.
By Manasa Mariam Mammen, Zafer Kayatas, Stefan Wagner
RealCompanion is a benchmark that evaluates an AI companion’s ability to understand a human over long, real-world conversations. It consists of ten real relationships with 27,218 messages spanning up to 120 days, along with derived files such as a profile, persona, chat ground truth, and a question set that cites the relevant messages. The study finds that past context is rarely needed, memory detection is challenging, and agent systems vary widely in cost while achieving similar persona reconstruction.
By Arman Behnam, Sunglyoung Kim, Liangwei Yang
The paper introduces a continuous evaluation framework that assesses both outcome-level and process-level aspects of evolving enterprise AI agent skills. It applies this framework to two variants of a Business Value Determination skill, running 240 trials across multiple models, harnesses, and specifications. The results show that while most trials pass final numerical checks, a large majority still exhibit process-level deviations, and dependency attribution reduces the number of failed checks per run. The framework also provides reusable regression tests and highlights specification sensitivity across configurations.
By Ngoc Phuoc An Vo, Aarya Doshi, Vadim Sheinin
The paper introduces Bayesian Dialectical Argumentation (BDA), a method for aggregating answers from multiple large language models (LLMs) in a council setting. BDA treats each LLM’s typed moves—proposals, challenges, and concessions—as evidence in a classical annotator model, estimating per-agent reliability even when some agents are persistently unreliable. By weighting evidence according to these inferred reliabilities, BDA produces calibrated posterior probabilities for candidate answers and can invert unreliable agents instead of merely outvoting them, achieving superior calibration and robustness on both binary and multi-class benchmarks without extra LLM calls.
By Ionel Eduard Stan, Paolo Napoletano
Mimir is a physics‑grounded large language model agent designed for long‑horizon irrigation control. It operates on two timescales: a fast scale that uses a structured physical interface and deterministic simulator to validate and refine LLM proposals before execution, and a slow scale that consolidates recurrent failure patterns into persistent contextual principles. Across multiple sites, crops, and years, Mimir achieves the lowest aggregate control cost and reduces irrigation usage by about 51% compared to historical schedules, while ablation studies confirm the importance of forward simulation, verified revision, and persistent context.
By Yimeng Liu, Mi Zhang, Younsuk Dong, Zhichao Cao
The paper introduces VISTA, a visual harness that equips a general-purpose multimodal model with long‑horizon vision and a lossless visual memory. VISTA enables the model to directly perceive and actively retrieve past observations, allowing it to reorganize visual input during reasoning. On the ARC‑AGI‑3 benchmark, VISTA boosts Claude Opus 5.0’s Relative Human Action Efficiency from 40.68 to a perfect 100.00, completing all 25 public games with 57.4% fewer actions than first‑time human participants, and it also outperforms baselines on three additional visual game and puzzle benchmarks.
By Qiushi Han, Keya Hu, Linlu Qiu, Cathy Wu, Kaiming He
BudgetSchemaBench is a diagnostic tool for evaluating how different schema‑context budgets affect text‑to‑SQL systems. It automatically derives relevance labels from gold SQL, tests four budgets across 80 databases, and compares three schema representations while keeping table rankings fixed. The study shows that increasing the budget improves execution accuracy, especially for lexical retrieval, and that dense retrieval already captures most needed tables at low budgets.
By Chen Shen
The article surveys safety concerns for self‑evolving agents that continually update their internal state, such as model parameters and memories, from new interactions. It introduces the SAVER framework, which tracks reusable influence, adaptation, violations, exposure, and response to assess whether safety properties persist as agents evolve. The survey finds that legitimate state can become unsafe when its persistence, authority, or scope expands beyond its original conditions, and highlights gaps in current research on descendant repair and longitudinal evaluation.
By Jiahao Chen, Zhou Feng, Oubo Ma, Yichen Yan, Ruixiao Lin, Hangtao Zhang, Linkang Du, Hengyu An, Yong Yang, Jun Liu, Junhao Li, Naen Xu, Chunyi Zhou, Yuan Su, Zehao Jin, Qianli Ma, Leyi Qi, Yiming Wang, Zhe Ma, Yuwen Pu, Mengyao Du, Yuanyi Song, Enhao Huang, Zhihui Fu, Jun Wang, Jinfeng Li, Yuefeng Chen, Hui Xue, Yiming Li, Tianyu Du, Shouling Ji
The paper introduces the Cognitive Continuity Test (CCT), a policy-relative contract designed to verify state transitions in persistent AI agents. CCT uses scoped authority, provenance, deterministic application, semantic predicates, and candidate-persistence receipts to classify transitions as verified, violated, or requiring further evidence. Experiments with the IdentityLineageBench dataset show that the CCT can accurately match canonical labels and identify invalid fixtures, while also providing performance metrics for valid-path execution.
By Jun He, Deying Yu
Proof‑Gated Signing (PGS) is a method for securing on‑chain AI agents that control wallets by simulating proposed transactions, extracting their effects, and using an SMT solver to verify a declarative value‑and‑permission policy across a range of oracle‑uncertainty prices. PGS then compiles on‑chain post‑conditions—such as wallet balance bounds, payee receipts, allowance caps, and ownership checks—ensuring that any execution satisfying these conditions also satisfies the policy, even when the chain state drifts due to front‑running, contract upgrades, or token‑parameter changes. In a testbed of 260 scenarios, PGS prevented 93.6 % of harmful cases and passed 97.5 % of benign ones, outperforming simulation‑only checking and static allowlists, while incurring about 41 k gas and 0.1–0.2 s per check.
By Bravish Ghosh
CtrlWAM introduces a controllable world action model that jointly predicts actions (intent) and visual futures (foresight). By executing perturbed actions in a simulator and pairing them with noised visual outcomes, it aligns action predictions with their visual consequences, using warped video–action noise schedules to maintain visual layout responsiveness. The model extends beyond ego‑only control to multiple agent streams, improving action forecasts, video–action agreement, and command following in driving and robotics experiments.
By Chensheng Peng, Wenhao Ding, Ran Tian, Zewei Zhou, Jef Packer, Maximilian Igl, Peter Karkus, Yan Wang, Masayoshi Tomizuka, Boris Ivanovic, Marco Pavone, Yuxiao Chen
ALFRED is an open‑source mobile manipulator designed for long‑term plant monitoring, built from commercial parts and featuring a six‑degree‑of‑freedom arm, LiDAR, RGB‑D cameras, RTK GNSS, and an IMU on an Ackermann‑steered base. The platform was iteratively refined over four builds to meet six requirements—durability, modularity, repairability, sensing reach, endurance, and reproducibility—resulting in a 66.1% usable arm reach and clear LiDAR views in the final build. Over a year of monthly forest surveys, ALFRED completed 528 traversals without missing a scheduled collection, demonstrating its reliability despite battery wear and rapid build transitions.
By Ciar\'an Miceal Johnson, Christopher Quail, Garry Ellard, Alistair McConnell, Steve Tonneau, Fernando Auat Cheein
The paper introduces FaV-A, a multimodal agent that generates positive and negative space compositions in a staged manner. It first creates a base object, analyzes its shape to identify negative‑space semantics, and then produces compositional instructions for the final image. Experiments show that this approach yields more visually coherent and semantically aligned compositions than direct zero‑shot multimodal large language model baselines.
By Shiwen Wang, Jian Yang, Xu Wang, Xincan Wang, Weiming Dong
The paper proposes a new architecture for large language model (LLM) agents that enhances spatial understanding by combining geometrical tools with an LLM orchestrator in grid‑world environments. It first gathers geodesic trajectories, vector‑quantizes them to create a representative subset, and then has the LLM label each trajectory with a natural language description, turning them into reusable tools. During operation, the LLM selects the appropriate tool based on the current state and goal, while low‑level control executes the chosen trajectory, enabling efficient decision‑making in a partially observable 2D grid setting.
By Gabriel Turinici
Incident‑Arena is a new benchmark for AI coding agents focused on production incident response, featuring 20 tasks derived from real‑world open‑source software. Each task deploys a production application on an ephemerally created Kubernetes cluster, injects faults at various layers, and applies a sustained load profile. The benchmark introduces functional verifiers that maintain system‑level metrics while ensuring safe repairs, and shows that current frontier models achieve below 64.3% across the tasks, highlighting challenges in diagnosis, repair, and regression safety.
By Andre Fu, Malik Drabla, Leon Liu, Meji Abidoye, Marek Suppa, Lata Mishra, Adnan El Assadi, Yiyuan Li
MemFit is a long‑term memory system designed for conversational agents that stores each dialogue turn verbatim in an append‑only store, enabling near‑instantaneous, LLM‑free insertion. It indexes turns using segment summaries and employs an LLM‑free, multi‑path retrieval strategy that blends lexical and semantic signals with cross‑encoder reranking over caption‑augmented episodes. Experiments on LoCoMo, MemGallery, and LongMemEval‑S demonstrate state‑of‑the‑art performance while drastically reducing memory construction time and cost.
By Mitchell Piehl, Muchao Ye
The paper introduces "authorization succession," a framework that preserves authority across self‑modifying AI agent populations that can replace, fork, or roll back. It defines a protocol binding each generation to a manifest, root, unique parent, lineage, and population sequence, and establishes invariants that control root‑lifetime consumption and population exposure. The authors prove properties such as population‑safe succession, fork conservation, and rollback non‑reminting, and validate the approach with an executable evaluation covering 32 decisions and external adapters for two mutation systems.
By Genliang Zhu, Chu Wang
The paper introduces authorization‑paired evaluation and the FlowReview framework to address safety issues in multi‑agent systems where shared evidence can enable prohibited uses. By jointly enforcing blocking of prohibited actions and completion of authorized ones, FlowReview connects object resolution, permission ranking, and deterministic enforcement. Experiments show that reviewing combined artifacts can eliminate denied‑commit rates without sacrificing authorized supply, highlighting the need to link object identity and permission to execution for safe collaboration.
By Yunbei Zhang, Saiyue Lyu, Janet Wang, Yingqiang Ge, Jiang Guo, Jihun Hamm, Chandan K Reddy
The paper investigates how well large language models specify the software environments needed to run AI-generated code. Using a new agent protocol and a three‑layer dependency framework, the authors evaluate three coding agents across four languages and fifty tasks, finding that dependency specifications are often inconsistent, redundant, or incomplete. Agreement on dependency sets is as low as 7% for identical tasks, and newer agents show no improvement, indicating that environment specification remains a significant, unaddressed challenge in code generation.
By Bhanu Prakash Vangala, Tanu Malik
The paper investigates whether multimodal large language model (LLM) agents form partner‑specific conceptual pacts during repeated reference games. By comparing real agent dyads with a novel pseudo‑dyad baseline that preserves task structure but removes shared partner history, the authors show that humans compress descriptions and align labels through entrainment, whereas agents produce verbose, fixed‑effort descriptions with near‑ceiling label overlap regardless of partner. Thus, LLMs succeed in coordination without forming compact, history‑dependent referring expressions typical of human dialogue.
By Po-Ya Angela Wang, Chinmaya Mishra, Asl{\i} \"Ozy\"urek, Paula Rubio-Fern\'andez, Esam Ghaleb