AndroidFlux is a benchmark designed to assess how well mobile-use multimodal agents recover from errors. It restores recorded trajectories to checkpoints before, during, and after recoverable errors, allowing agents to start from identical device states. A complementary reward‑modeling benchmark evaluates recovery‑action preferences based on downstream rollout outcomes, and an offline RL dataset is released to improve both error recovery and overall task success.
By Gyubeum Lim, Yemo Koo, Ghunwoo Jang, Zsolt Kira
The paper introduces ParetoRSI-Gym, a framework for evaluating recursive self‑improvement (RSI) in language models across multiple domains—math, code, finance, medicine, and science—by having agents iteratively search for, curate, and train on data while keeping the base model, toolset, and training pipeline fixed. Experiments with three agent systems and three base models show that the best system (Claude Code with Opus‑5) improves generalization scores by an average of 9.2% relative to the starting model, with domain‑specific gains up to 63%. The study highlights that both the harness and the agent’s data‑selection strategy significantly influence outcomes, underscoring the need for realistic multi‑domain evaluation of RSI systems.
By Haojin Wang, Yangzhen Wu, Zeyi Chen, Jian Xie, Jiaying Ye, Lili Meng, Shuyun Tang, Shangding Gu, Dawn Song
The paper introduces a dynamic multi‑step decoding approach for time‑series foundation models, arguing that the best decoding strategy varies by instance rather than being fixed. A lightweight selector network ranks candidate rollout schedules using the backbone’s hidden states, trained on historical windows and evaluated via dynamic programming to keep complexity quadratic. At deployment, multiple selectors vote to choose a schedule, defaulting to one‑shot decoding when they disagree, and the method shows statistically significant improvements in point‑forecast accuracy on the fev‑bench benchmark with frozen Chronos‑2 and Moirai‑1.1‑R‑small backbones.
By Birmjune Kim, Hyungju Lee, Hyun Oh Song
CyberDual is a competitive framework where AI agents act as both attackers and reviewers in real software maintenance tasks. Attackers embed exploitable bugs into open‑source projects, while reviewer agents attempt to detect and fix them; success is measured by re‑executing the attacker’s proof‑of‑concept on the reviewed code. Across 1,088 valid attacks, three reviewers caught 85‑91% of bugs, yet about one in six attacks still produced a working exploit after review, highlighting gaps in current AI code‑review tools.
By Chenyuan Yang, Hwiwon Lee, Jongseong Kim, Yihan Yang, Dongjun Kim, Yinfang Chen, Lingming Zhang
SentryBench is a cost‑sensitive benchmark designed to evaluate large language model agents that use tools in public‑safety contexts. It measures agents on a frontier that jointly considers attack success, over‑refusal (missing real emergencies), and over‑reaction (unwarranted actions) under adversarial tool use, using scenarios grounded in real emergency‑call transcripts and disaster messages. The study shows that naive caution leads to high harm, while a targeted trust‑gate wrapper reduces harm across models and defenses.
By Md Abu Yusuf, Oyshi Tabassum Aditi, Md Faysal Hosain Mazed
Entity Matching using Decision Models explores a new class of large language model (LLM) approaches that output probabilities for predefined answer options instead of generating text. The study evaluates six decision models in a zero‑shot setting across eight benchmarks, comparing them to the generative LLM GPT‑6 Luna and the PLM‑based matcher Ditto. Results show that the top four decision models achieve mean F1 scores comparable to or slightly better than GPT‑6 Luna, outperform Ditto on most benchmarks, run faster and are cheaper, and can be further improved by a cascade that delegates uncertain pairs to an ensemble of generative LLMs.
By Aaron Steiner, Ralph Peeters, Christian Bizer
Audio-R1 is a reinforcement learning post‑training framework that leverages audio self‑supervised learning tasks to provide verifiable rewards for large audio‑language models. By automatically determining answers during task construction, Audio‑R1 verifies model responses and computes rewards, leading to consistent performance gains across multiple models and audio benchmarks. Ablation studies show each self‑supervised task contributes to the improvement, and joint training yields additional benefits, especially on downstream tasks distinct from the training tasks.
By Jihoon Jeong, Mirco Ravanelli, Zhepei Wang, Cem Subakan
RIDE‑Splat is a new pipeline that removes windshield reflections from driving videos to produce clean 3D reconstructions using 3D Gaussian Splatting. It assigns world‑consistent transmission to the 3D scene and models the camera‑attached reflection as a shared signed RGB image, learning geometry and reflection in two passes before fitting a clean 3DGS model. The authors also release the RIDE‑Splat Dataset, the first benchmark for reflection removal in driving scenes, and demonstrate that RIDE‑Splat outperforms existing methods on nine aggregate metrics, improving PSNR by up to 3.59 dB and reducing LPIPS by up to 0.0876.
By Kevin David Hayes, Alex Hanson, Allen Tu, Pranav Asthana, Matthias Zwicker, Tom Goldstein
AgentProofBench is a benchmark for reusable verification of web agents, linking frozen acceptance rules to execution from identical saved openings. It covers 97 tasks across 79 families and four web services, and provides paired tests that measure how well programs retain correctness after reuse and rebuild. Experiments with DeepSeek Flash and Pro show high retention rates and identify specific families where deployment-state panel correctness is critical.
By Jingjie Ning, Xueqi Li, Yi Lu, Yibo Kong
The paper introduces Thinking‑in‑Bets, a training‑free technique where a language‑model agent records its key claims, supporting checks, and confidence before receiving user feedback. By consulting this private claim checklist when challenged, the agent preserves verified work and focuses on uncertain or newly contested parts, reducing sycophantic revisions. Experiments on five Gemini models across eight benchmarks show that pre‑reply claim records nearly eliminate false reversals in patch review, protect report drafts from mistaken pushback, and lower unnecessary edits in multi‑turn coding while improving accuracy and confidence in non‑interactive tasks.
By Nghi D. Q. Bui
DynEdit is a dynamic framework for editing long-sequence knowledge in large language models. It uses token-level entropy to guide non-uniform sliding windows, focusing on high-entropy, information-dense regions that are harder to edit. Experiments on three benchmarks and two backbone models show DynEdit outperforms baselines, achieving up to +14.0 BLEU points on challenging paraphrase queries and near-saturated semantic scores across diverse knowledge domains.
By Jinhu Fu, Yan Bai, Yihang Lou, Li Sun, Jiahong Wu, Xiangxiang Chu, Sen Su
ToDoPro introduces an early token‑reduction strategy for reference‑based image generation that uses a few denoising steps on 2× downsampled reference images to probe token importance with the existing Diffusion Transformer. Tokens are then hierarchically pruned: low‑importance tokens are discarded, mid‑importance tokens are replaced by their downsampled versions, and high‑importance tokens are retained. Experiments on three base models and two benchmarks show significant inference speedups, reduced peak GPU memory, and improved generation quality by filtering out noisy reference tokens.
By Caiyi Sun, Yujing Sun, Siu-Ming Yiu
The paper introduces CanDO, a test‑time canonicalization technique that restores performance of score‑only black‑box point cloud models when faced with pose shifts. CanDO searches for an input transformation that maximizes model confidence using derivative‑free optimization, requiring only forward queries. Experiments show that it recovers 76% of the oracle gap for azimuthal misorientation and 51% for arbitrary SO(3) rotations in object part segmentation tasks.
By Seonghwan Park, Min-Gyu Park
NumTrace is a benchmark that audits the provenance of numerical claims made by financial analyst agents, checking each claim’s source and financial semantics. It contains 540 questions drawn from 2,163 SEC EDGAR documents and evaluates claims on three verdicts: value, binding, and support. The benchmark identifies right-number‑wrong‑account (RNWA) errors, revealing that 67.5–81.8% of failing claims from agents like Claude Opus 5.5, GPT‑6 Astra, and DeepSeek‑V4.1‑Flash are RNWA errors, especially in 13F co‑filing questions where most claims pass the value check but fewer than half pass the binding check.
By Tianlei Zhu, Matt White, Xue Liu, Xiao-Yang Liu
GUIFlow is a framework that enables temporal redundancy‑aware incremental inference for GUI agents built on vision‑language models. By selectively updating computational states—such as position‑aligned KV states, vision encoder outputs, and language model drafts—across interaction steps, it allows cross‑step reuse of visual encoding, prefill, and decoding without extra training or model changes. Experiments show up to 3.39× speedup on offline benchmarks and 3.49× in online OSWorld interactions while preserving task performance.
By Ye Li, Haoran Ding, Huanan Liu, Yuan Meng, Yutian Shen, Yuansong Wang, Kangye Ji, Hanyun Cui, Hongbo Duan, Chen Tang, Jiawei Li, Shu-Tao Xia, Zhi Wang
The paper introduces CarbonRouter, an online framework that jointly selects the model, GPU type, and region for each query in multi‑turn LLM web services to minimize carbon emissions. It incorporates a cache‑aware carbon cost model that accounts for the history recomputation cost when a conversation is routed to a different deployment, and a lightweight gate that keeps a session on its cached deployment when a move back is likely. Experiments on two real‑world conversation datasets show that CarbonRouter achieves the lowest carbon footprint among evaluated policies, reducing CO₂ emissions by 16.1 % on average and up to 23.1 % for longer sessions, while also improving predicted answer quality and estimated latency.
By Xiaoyang Zhang, Yuejun Yan, Qingang Zhang, Yonggang Wen
MarkovSim is an LLM agent harness designed for long‑horizon computational simulation workflows, featuring a built‑in context management scheme that keeps the agent’s context bounded. The system records workflow steps in a memory system with a predefined schema and, at finite‑state machine state transitions, constructs a compact representation of solution progress to use as context. Evaluated on finite element analysis benchmarks, MarkovSim successfully completed 16 of 18 runs, self‑correcting faults in solver operations and model physics while requiring engineer input only for judgment calls.
By Yeui-Lung Lei, Wen-Bo Chen, Pan Zhang
The paper introduces an interpretable interactive evolutionary multi-objective optimization algorithm that learns a tree-based preference model through active learning from pairwise comparisons. It extends the survival operator of NSGA-II and NSGA-III to guide the search toward the decision-maker’s preferred region. Experiments show the method outperforms both black-box and other interpretable models in locating the most preferred solution, and a real-life engineering case study demonstrates how the learned model reveals the objectives influencing preferences.
By Thomas Quadt, Boris \v{C}ule, Roy Lindelauf, Herman Monsuur, Mark Voskuijl
The paper introduces Culture-MMT, a benchmark for evaluating how well machine translation systems can reconstruct meaning from distributed evidence in social‑media posts. It contains 498 curated, de‑identified user‑generated content instances across 20 domains, classified into Symbol, Visual, and Hybrid based on the type of evidence required. Experiments with 17 model configurations show that real images significantly improve translation for Visual and Hybrid cases, while random images do not, underscoring the importance of accurate visual grounding and cultural adaptation.
By Dingming Li, Yiwen Qiu, Linjuan Wu, Hongxing Li, Zixuan Wang, Xinze Lyu, Ye Guo, Daoxin Zhang, Zhe Xu, Weiming Lu, Yongliang Shen
The paper introduces RAINFALL, a neuro‑symbolic framework that generates ranking functions for proving loop termination by treating synthesis as a code‑completion task with placeholders and verifying the output with a symbolic checker. RAINFALL achieves a 90.22% success rate on 358 terminating C programs, outperforming existing baselines and boosting overall performance to 96.65% when combined with PROTON 2.1. The study also demonstrates the importance of placeholder contracts, few‑shot examples, symbolic verification, and diagnostic feedback across multiple LLMs.
By Rongkang Li, Jiaxin Hu, Taolue Chen, Yong Li, Fu Song, Zhilin Wu