DARE (Difficulty Analysis for Repair Enhancement) is a framework that measures, predicts, and interprets the difficulty of coding repair tasks using a continuous difficulty metric based on failure rates across multiple agents. It employs 151 static features to predict difficulty, analyzes contributions from seven feature groups to identify task-specific characteristics, and selects tailored agent support accordingly. Experiments on 2,594 tasks across four benchmark partitions show that DARE’s predictions correlate well with actual difficulty and that matching support to salient feature groups improves repair success.
By Jikwang Lee, Seungwook Jung, Jaechang Nam
The paper investigates how large language models (LLMs) misinterpret office documents due to differences between the text extracted from Office Open XML (OOXML) files and what humans actually see. By analyzing common extractors, the authors identify 19 hidden OOXML channels and 12 content omissions, and demonstrate that these gaps enable five attacks that mislead major chat and coding assistants. They propose Prism, a lightweight defense that reconstructs the human view from OOXML before feeding text to the model, which generalizes to unseen attack payloads.
By Penghui Li
The paper introduces SecPatRec, a hybrid method that combines a ReAct-style LLM agent with static code property graph analysis to recover security patterns from source code repositories. It treats a security pattern as a reference architecture, detecting prescribed roles, grounding them in code entities, and recovering their interaction sequences. The authors evaluated SecPatRec on 100 pattern instances from 93 Java and Python projects, achieving high role-detection and grounding accuracy and outperforming a one-shot LLM baseline in both recovery quality and API cost.
By Anh Minh Nguyen, Tuan Dat Tran, Alzubair Hassan, Liliana Pasquale
The study examines how different harness configurations affect a fixed large language model’s compliance with banking conduct rules. Four open‑weight models were tested across five harnesses on 156 pre‑registered retail‑banking scenarios, yielding 3,120 two‑turn episodes scored for rule violations and over‑refusals. Results show that while overall compliance rates varied modestly (87.3%–92.1%), the impact of a harness differed markedly between models, with some models’ best and worst harnesses differing by over 10 percentage points and a same‑model compliance reviewer reducing violations from 9.6% to 6.4%.
whyItMatters":"The findings suggest that validating a model’s compliance requires testing the specific model‑harness pair rather than the model alone, highlighting the importance of action logs over conversation transcripts for accurate conduct assessment."
By Mehmet Murat Huyut
The paper introduces CASE-Bench, a benchmark of 145 repository-level tasks derived from real CVEs across 51 projects, each run in Docker with functional tests and proof‑of‑concept exploits. It evaluates 13 context‑construction methods and six controls across six LLMs, showing that functionality‑oriented context improves functional pass rates by up to 8.2 pp but only 3.7 pp for security, that omitting safeguards can drop secure pass rates by 8.1 pp, and that coding agents see relevant safeguards in only 27‑36 % of trajectories. These results highlight how repository context shapes the security of LLM‑generated code and guide the design of security‑aware context construction and evaluation.
By Jiongchi Yu, Keke Lian, Junqing Wang, Zhikang Fan, Zhaoxuan Li, Yongheng Huang, Ziming Zhao, Dong Zhang
The study investigates whether a physically grounded, low‑dimensional catchment model can rival the predictive accuracy of neural networks. By applying three fundamental hydrological laws—conservation, dissipation, and aggregation—to a lumped model, the authors develop CaRT and its refined version CaRT+, which uses 14 states and 35 parameters per catchment. Across 602 British, 593 US, and 505 Australian gauged catchments, CaRT+ achieves median Nash‑Sutcliffe efficiencies comparable to state‑of‑the‑art LSTM models, demonstrating that process‑based laws can match deep‑learning performance.
By Xilin Xia
ArcRAG is a retrieval-augmented generation system that operates over knowledge graphs without relying on construction‑specific schemas or relation names. It dynamically induces a query‑local graph interface at inference time, plans multi‑hop traversals, and executes them as Cypher queries. Experiments on MuSiQue, HotpotQA, and TriviaQA show that ArcRAG remains effective across both schema‑guided and open‑information‑extraction graph constructions, demonstrating that retrieval can be construction‑agnostic.
By Jyotin Goel, Abhinav Arun, Tejas Prakash Agarwal, Bhaskarjit Sarmah, Stefano Pasquali
The paper exposes a vulnerability in AI toolchains where signed Model Context Protocol (MCP) tools can hide malicious code that leaks data, even when protected by state‑of‑the‑art runtime defenses. It introduces germline execution, which runs each tool invocation in an isolated, data‑free copy, limiting information disclosure to only what is authorized for the destination. Experiments on 153 attack runs show that germline execution prevents unauthorized disclosure in all cases, outperforming baseline defenses and ablations, while maintaining efficient performance (10‑49 ms per invocation).
By Yichiet Aun (Universiti Tunku Abdul Rahman), Mohammed A. Alhartomi (University of Tabuk), Jing Jie Tan (National University of Singapore), Waqas Nadeem (Universiti Tunku Abdul Rahman), Adeb Salh (Universiti Tunku Abdul Rahman), Shankar Karuppayah (Bitranger, CYRES, Universiti Sains Malaysia), Sin Chee Tan (ABLEs)
The paper introduces SARBLOH, an open‑weight agent that emphasizes spatial perception over planning to solve ARC‑AGI‑3 visual games. By converting raw integer grids into an object inventory and geometric briefing within 35 ms, the agent improves rubric scores from 3/28 to 25/28 and reduces token usage by 89.4%. On held‑out games, its geometric invariants generalize perfectly, and it achieves high efficiency on the public set, outperforming the first milestone winner by 13.7× on Kaggle’s hidden evaluation.
By Harsidak Singh Banwait
The paper introduces a toolchain that enables machine‑learning model development on AWS EC2 F2 cloud FPGAs using high‑level Vitis HLS. It implements a complete Vitis HLS to HDK bridge to AFI workflow, a four‑level methodology (FP32 PyTorch, bit‑accurate quantized Python, HLS C‑simulation, physical FPGA) for error localization, and documents deployment, timing closure, and HLS engineering decisions. The authors validate the approach by implementing TinyBERT’s embedding layer and first encoder layer on FPGA, noting that weight streaming from host DRAM results in poor performance compared to CPU/GPU implementations.
By David Rostcheck
The paper introduces Evolutionary System Prompt Learning (E‑SPL), a method that jointly updates a large language model’s weights via reinforcement learning and its system prompts via self‑reflection. In each RL iteration, E‑SPL samples trajectories under multiple prompts in parallel, using them to both evolve the prompts and improve the model’s weights. Experiments show that E‑SPL boosts success rates across several benchmarks (e.g., from 38.8 % to 45.1 % on AIME → BeyondAIME) and outperforms RL‑only or evolution‑only baselines, indicating that weight and context updates synergistically enhance generalization.
By Lunjun Zhang, Ryan Chen, Bradly C. Stadie
The paper surveys the use of reinforcement learning (RL) in graphical user interface (GUI) agents, outlining how RL addresses challenges such as long-horizon credit assignment, distribution shifts, and safe exploration that supervised fine-tuning alone cannot solve. It introduces a taxonomy dividing methods into Offline RL, Online RL, and Hybrid Strategies, and discusses key aspects like reward engineering, data efficiency, and technical innovations. Emerging trends highlighted include the move toward composite, multi-tier reward architectures, world-model-based training to mitigate GUI I/O latency, and the spontaneous appearance of System‑2‑style deliberation, all of which inform a roadmap for future robust GUI automation.
By Junan Hu, Jian Liu, Jingxiang Lai, Jiarui Hu, Yiwei Sheng, Shuang Chen, Jian Li, Dazhao Du, Song Guo
The paper introduces Alice, a closed‑loop system for online learning of executable world models when prior knowledge is misaligned. Alice uses failed candidate updates as structural signals to identify and refine state‑dependent dynamics, generating hypothesis classes that provide targeted examples for model updates. Experiments on a modified Baba Is You game—Baba in Wonderland—demonstrate that Alice significantly improves model learning under prior misalignment, with ablation studies confirming the benefits of class structure and exploration signals.
By SeungWon Seo, DongHeun Han, SeongRae Noh, HyeongYeop Kang
The paper outlines a roadmap for advancing pluralistic value alignment in AI, noting that current research lacks evidence of influence on model training or evaluation. It identifies three key barriers: a need for empirical studies on benefits and drawbacks, clearer specifications of when and how pluralistic behavior should occur, and robust evaluation methods suitable for production. The roadmap aims to guide research toward practical deployment in responsible AI systems.
By Elinor Poole-Dayan, Jillian Fisher, Jacob Andreas, Mitchell Gordon, Michiel A. Bakker
The paper proposes a method for assessing AI suitability for workplace tasks by profiling both AI agents and tasks along shared cognitive capability dimensions. It uses benchmark performance to infer agent capabilities and expert input to weight task requirements, allowing independent updates as models and roles evolve. The authors validate the approach on synthetic agents, six AI systems, and 410 employees across six domains, showing that AI systems vary more by cognitive dimension than by model family, while workplace tasks share a common cognitive core.
By Jonathan Prunty, Marko Te\v{s}i\'c, Patrick Quinn, Jos\'e Hern\'andez-Orallo, Lucy Cheke
The paper "Reliability Engineering for AI Systems: Challenges, Methods, and Directions" discusses how AI reliability extends beyond correct outputs to include consistent operation of retrieval, memory, tool use, permissions, human oversight, and inter-system interactions, especially for generative AI. It adapts traditional reliability engineering techniques—such as failure definitions, operational envelopes, FMEA, accelerated testing, field monitoring, and reliability growth—to AI, proposing a four-level diagnostic framework and a TEVV (Test, Evaluation, Verification, Validation) approach. The authors also introduce the SMART framework for statistical guidance and reference NIST’s AI risk management framework, illustrating their concepts with three case studies and emphasizing the urgent need for new measurements and safety guardrails as AI systems evolve rapidly.
By Rong Pan, Yili Hong, Min Xie
AutoBCI is an agentic framework that discovers and selects EEG decoding architectures across diverse tasks such as emotion recognition, motor imagery, and sleep staging. It employs a Designer Agent that performs Pool‑Guided Architecture Discovery (PGAD) and a Forecaster Agent that uses Performance Estimation from Early Knowledge (PEEK) to predict full‑budget validation performance from early learning curves. Evaluated on 14 EEG datasets with six large language models, AutoBCI’s best architecture (Claude Opus 5.5) achieves 64.16% average test balanced accuracy, slightly surpassing the strongest baseline, and PEEK reduces prediction error by 38.1% compared to the best‑observed‑score baseline.
By Muyun Jiang, Yi Ding, Wei Zhang, Jinbo Chen, Chenyu Liu, Zhenjie Yang, Yuxin Li, Jingyuan Chen, Yuhao Lu, Yong Li, Shuailei Zhang, Cuntai Guan
MASBench is a benchmark for evaluating large language model (LLM)-based multi‑agent systems (MAS) under partial observability, a realistic constraint where agents only see limited environmental information. The benchmark is structured into three progressive task categories—Reasoning, Scheduling, and Game—and assesses three collaboration mechanisms: Protocol, Memory, and Routing. It provides deterministic evaluation metrics such as performance score, communication cost, and cost effectiveness, and includes empirical results across various LLM backbones to guide effective MAS design.
By Qizhi Chu, Zekai Yu, Sijie Wen, Yang Liu, Chen Qian, Cheng Yang, Chuan Shi, Zhiyuan Liu
AgenticFinBench is a new benchmark comprising 230 real‑world financial tasks across eight categories, two languages, and four complexity levels, designed to capture the breadth and open‑ended nature of financial AI work. It introduces a reference‑free evaluation framework that uses question‑specific rubrics and a two‑stage claim audit to detect hallucinations, aligning assessment more closely with real‑world financial practice. Experiments on 18 agent configurations show that ignoring hallucinations inflates performance, that harness differences drive evidence grounding more than model differences, and that task complexity reduces quality scores, highlighting challenges in explaining financial impacts and comparing data consistently.
By Shu Zhou, Ziyang He, Yuxin Rao, Yudong Fang, Jiali Qian, Dongyuan Chen, Juntao Zhang, Yong Hu
ScamWorld is an interactive environment that recreates reported scam websites across various scenarios, fraud mechanisms, and languages, allowing researchers to observe fraud indicators before a web agent’s final action. The platform enables controlled, repeatable evaluation of web agents, which are increasingly used to browse, shop, and manage accounts on the open web. A study of six leading LLM-powered agents revealed deception rates between 11% and 35%, with agents more likely to proceed with payments than to provide credentials or wallet information, and some agents continuing after recognizing suspicious conditions.
By Yutao Wu, Xiao Liu, Ming Wen, Hanxun Huang, Jun Wang, Xingjun Ma