The paper introduces Dynamic Boundary Evaluation (DBE), a method that moves beyond static benchmarks by locating the per‑prompt pass probability near 0.5 for large language models. DBE provides a calibrated item bank with difficulty labels, a Skill‑Guided Boundary Search algorithm that works with API‑level access, and an adaptive evaluation protocol that places new models on a unified ability scale. The approach is applied to safety, capability, and truthfulness tasks, expanding coverage without saturation and remaining compatible with existing datasets.
By Haoxiang Wang, Da Yu, Huishuai Zhang
AgentTime is a benchmark that tests whether AI agents can manage their own runtime by working for a specified duration, predicting their wall‑clock time, and estimating elapsed time afterward. It includes 222 tasks from 18 sources across coding, computer use, agentic work, and automated research, with duration‑following experiments that append a single instruction specifying how long to work. Results show significant variability in accuracy, with some agents deviating from requested runtimes by up to 2.9×, and many agents stopping prematurely even after completing a task.
By Michael Ofengenden, Maksym Andriushchenko
RideBench is a new benchmark built on a large-scale ride‑hailing time‑series dataset called Ride‑Hailing, which spans four years of half‑hourly data across 200 spatial areas and includes three exogenous scenarios: weather disturbance, holiday effect, and large‑scale event impact. The benchmark evaluates over 30 forecasting methods for both week‑ahead and 8‑week‑ahead horizons, revealing that while future‑known exogenous variables improve week‑ahead accuracy, current models still struggle with disturbance‑induced pattern changes and long‑horizon forecasting. These results highlight a mismatch between existing models and real‑world ride‑hailing needs, underscoring the necessity for better exploitation of exogenous information, scalability, and long‑horizon planning capabilities.
By Shengsheng Lin, Jing Hu, Zhengyang Hu, Jiazheng Sun, Zichun Cao, Siwei Sun, Zhichao Zou, Enyun Yu, Dongdong Li, Xinyi Hu, Weiwei Lin
The paper introduces QUILT, a new sparse‑attention execution mechanism that jointly processes neighboring queries to reuse shared key‑value (KV) entries, thereby reducing redundant memory traffic and computation. QUILT employs Shift‑and‑Compare Set Decomposition (SCSD) to convert irregular set operations into regular data‑parallel primitives and pipelines this with attention computation to hide overhead. The approach includes cascaded sharing across multiple granularities and a tile‑aware strategy that balances sharing with hardware utilization, achieving up to 55.1% lower kernel latency and 36.8% faster time‑to‑first‑token on LongBench benchmarks with negligible accuracy loss.
By Zhenduo Zhao, Qihui Zhou, Mingcong Song, Zhiyi Chen, Chuangguan Ye, Fengfan Hou, Zequn Gong, Jing Li, Hongjie Si, Guoping Long
The paper introduces BEANS-Next, a comprehensive benchmark covering acoustic perception, biological category recognition, scene understanding, and in-context learning in bioacoustics, and shows that current audio‑language models perform poorly outside the narrow label‑centric tasks traditionally evaluated. To address this gap, the authors present ROOTS, a large‑scale training dataset enriched with curated real‑world data, behavioral and acoustic metadata, and synthetic generation, which improves performance across all BEANS‑Next task groups. Both the benchmark and dataset are released as open‑source resources to accelerate research in bioacoustics and ethology.
By Christos Plachouras, David Robinson, Marius Miron, Gagan Narula, Paul Laisn\'e, Anthony L. T. Fine, Benno Weck, Ellen Gilsenan-McMahon, Diane Kim, Laura Hay Mack, Maddie Cusimano, Sara Keen, Lukas Rauch, Benjamin Hoffman, Emmanuel Chemla, Emmanouil Benetos, Johan Pauwels, Milad Alizadeh, Matthieu Geist, Olivier Pietquin
The paper investigates whether machine learning can predict chaotic quantum many-body dynamics by treating it as a time‑series forecasting problem. Using a few‑qubit PXP chain, the authors compare a nonlinear Transformer model with a simple linear DLinear model across regimes from ergodic to scarred dynamics. They find that while the Transformer performs well in ergodic regimes, its accuracy drops near scarred states, whereas DLinear maintains high accuracy across all regimes, indicating that complex quantum dynamics can be forecasted accurately with a simple linear mapping.
By Francesco Perciavalle, Agostino Gallo, Francesco Plastina, Gianluigi Greco, Nicola Lo Gullo, Carlo Adornetto
SparseDecoding is a decoding‑aware pruning framework that improves large language model inference by aligning pruning with the activations seen during actual autoregressive generation. It constructs calibration matrices from dense‑model activations (excluding prefill) and introduces an optimized N:M sparse matrix‑vector kernel with bitmask indexing to accelerate decoding. Experiments on models such as Llama‑3.1‑8B, Llama‑3.3‑70B, and Qwen3‑14B/32B show consistent performance gains over standard fixed‑text calibration and up to 1.48× wall‑clock speedup on A100 GPUs.
By Qitong Wang, Xinwei Niu, Mingluo Su, Shanwei Zhao, Shiai Zhu, Huan Wang
The paper presents a transfer‑learning framework that combines homotopy in mission parameters with Markov chain Monte Carlo (MCMC) to generate training data more efficiently for indirect low‑thrust trajectory optimization. By reformulating the multiobjective problem as sampling from an unnormalized target distribution in costate space, the authors compare three MCMC algorithms on a planar multi‑revolution transfer in the circular restricted three‑body problem, finding that gradient‑based variants offer the best trade‑off between sample quality and computational cost. The resulting MCMC samples are used to fine‑tune a diffusion model conditioned on the mass parameter, enabling the model to learn a global representation of the solution distribution and produce new high‑quality solutions, ultimately yielding 40 % more feasible solutions and a superior Pareto front compared to a state‑of‑the‑art indirect approach.
By Jannik Graebner, Ryne Beeson
The paper introduces TRACE, a protocol designed to diagnose whether changes in verifier scores for large language model agents reflect actual changes in agent behavior or merely alterations in the evaluation process. TRACE works by applying targeted changes to evaluation components, running paired experiments, and re‑scoring unchanged trajectories to isolate the source of score variation. Experiments on synthetic tasks and public benchmark tasks demonstrate that seemingly significant score shifts can often be attributed to evaluation artifacts, while TRACE can also detect genuine behavioral changes.
By Radhika Gaonkar
The paper investigates whether visible chain-of-thought (CoT) reasoning universally improves analytics code generation across different target languages (SQL vs. Python pandas). Using a benchmark that varies persona phrasing, target language, CoT format, and model settings, the authors find no consistent advantage from visible CoT or from matching the reasoning representation to the target language. Instead, the impact of visible reasoning depends on the persona, target, model configuration, and internal-reasoning setting, suggesting that reasoning strategies must be tailored rather than applied as a universal default.
By Bhawani Shankar Leelar, Pawan Chorasiya, Davin Hill, Robert E. Tillman, Tamer Soliman
The paper introduces Transport REINFORCE, a model‑free policy‑gradient method for discrete‑time mean‑field control that captures both the dynamics and population‑distribution effects of a policy. It perturbs the population distribution—directly on the simplex for finite state spaces and via a Gaussian‑mixture transport map for continuous spaces—to estimate the missing mean‑field contribution. The authors prove consistency of the perturbed objective and gradient, provide bias and mean‑square error bounds, and demonstrate improved performance over standard REINFORCE on several benchmarks.
By Adonis Jamal, Samy Mekkaoui, Yadh Hafsi, Huy\^en Pham
The study investigates how different message‑passing (MP) layers affect graph‑level regression performance in Graph Neural Networks (GNNs). By replacing the MP component in four existing GNN regressors with ten varied configurations—including convolutional, isomorphism‑based, and attention‑based designs—the authors evaluate performance across eleven neural‑network graph datasets, measuring metrics such as rank correlation, prediction error, top‑k retrieval, latency, and memory usage. Results show that the choice of MP layer significantly influences outcomes, with the optimal layer depending on graph size, training‑set size, and regression objective; classical layers like GEN, k‑GNN, and PNA outperform attention‑based layers on small graphs, while GATv2 excels on datasets with few large graphs.
By Keith G. Mills, Aedan J. DeFrates, Joong Ho Kim
Delphos is a deep reinforcement learning framework designed to aid the specification of discrete choice models. It treats model specification as a sequential decision-making task, where an agent selects modeling actions—such as adding constants, choosing taste parameters, applying transformations, or including interactions—to generate candidate models. Each candidate is estimated and evaluated using a reward function that reflects statistical fit and behavioral expectations, and a Deep Q-Network learns which sequences of decisions lead to high-quality models, as demonstrated on simulated and empirical datasets.
By Gabriel Nova, Stephane Hess, Sander van Cranenburgh
The paper investigates whether small large language model agents can operate within explicit wall‑clock time budgets while using the allotted time productively. Experiments with Qwen3.6‑27B on MLE‑Bench Lite and Qwen3‑4B on Zork I show that agents fail to translate budget statements into controlled time use, largely due to lack of timing feedback and poor anticipation of action durations. Interventions such as harness‑based timing exposure, enforcement hooks, and reinforcement learning with budget‑aware rewards improve budget adherence but do not lead to better task performance, revealing a persistent gap between respecting time limits and effectively allocating extra time.
By Aaron Wang, Neelabh Madan, Vlad Sobal, Matthew Trager, Michael Kleinman, Elman Mansimov, Wei Xia, Stefano Soatto
The paper introduces GenUI‑Harness, a multi‑agent system that pairs a Tool Agent for data retrieval with a GUI Coder Agent that generates structured interfaces to aid human‑agent interaction. It tackles training challenges by using Dynamic UX for lightweight reward collection and Reward Auditor to monitor reward quality. The authors present UI‑TAU Bench, a benchmark of 1,300 database‑backed tasks, and show that GenUI‑Harness improves Pass@3 scores and reduces dialogue rounds compared to existing models.
By Xiaolong Li, Xiaohan Xu, Jinyang Li, Xinnuo Xu, Ge Qu, Nan Huo, Jack Williams, Reynold Cheng
ReTeach is a reflective self‑distillation framework that builds a self‑teacher through multiple rounds of reflection and retry, relying only on self‑generated attempts and outcome‑level verification. Starting from an unsuccessful student rollout, the teacher alternates explicit reflection with new attempts until success or a retry limit is reached, without needing reference answers, external feedback, or cross‑example memory. The method uses outcome‑aware weighting to distinguish correct, reflection‑corrected, and unresolved examples, and on‑policy distillation aligns the student’s token‑level predictions with the teacher’s context‑conditioned distributions, achieving a 1.39‑point accuracy gain over GRPO across six reasoning benchmarks.
By Yafeng Tang, Hao Li, Hongsheng Yu, Qiang Fu
The paper introduces a three‑dimensional framework for comparing AI methods that solve sequential decision‑making puzzles. It maps each approach to a Markov decision process, evaluates autonomy via human ranking, and measures skill against computational cost. Applying this to the Tower of Hanoi, the authors compare graph‑based, reinforcement learning, and large language model techniques, finding that LLMs shift complexity to inference time, incurring higher memory and runtime costs.
By Sadig Gojayev, Carolina Fortuna
Generative Adversarial Loops (GAL) is a framework that alternates between a discriminator generating adversarial data to expose weaknesses in current AI systems and a generator discovering algorithms to overcome those weaknesses. Applied to tasks such as KV compression, sparse video generation, sparse attention, and context extension, GAL’s discriminator identifies shortcomings in state‑of‑the‑art techniques, while the generator produces new algorithms that improve performance on both adversarial and established benchmarks. The study demonstrates significant gains, for example boosting CompactorPress on KV compression from 0.35 to 0.97 on the discriminator dataset and improving Dual Chunk Attention from 0.20 to 0.90.
By Kislay Aditya Oj, Nidhi Jain, Sri Surya Varma Datla, Priyanka Jayaswal, Kumar Krishna Agrawal, Aditya Desai
The paper introduces Missing Modality-Aware Local Temperature Scaling (MMA‑LTS), a post‑hoc voxel‑wise confidence calibration technique designed for multimodal brain tumor segmentation. MMA‑LTS adapts a temperature field based on which MRI modalities are present and a voxel‑wise difficulty score, addressing calibration errors that arise when specific modalities are missing. Experiments on BraTS 2020 and FeTS 2024 demonstrate that MMA‑LTS improves calibration while maintaining segmentation accuracy across various missing‑modality scenarios.
By Sol Lee, Hyunji Kim, Sungrae Hong, Donghee Han, Mun Yi
ProtoSemImage proposes representing class prototypes as image-valued archetypes rather than vectors, using a four‑channel HSV space that encodes linguistic factors. The model learns a deformable row alignment between a document image and its archetype bank, enabling spatial template matching that highlights where and how an input diverges from its prototype. Experiments show that this image representation outperforms vector prototypes on a ten‑class task, while diagnostic tests reveal that the matching component, not the color compression, limits performance.
By Mohammad Zare, Pirooz Shamsinejadbabaki