Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.
Most existing equivariant architectures for Hamiltonian prediction propagate features in direct-sum spaces of irreducible spherical tensors, whereas the target Hamiltonian is naturally an operator blo...
We introduce VINE: a non-parametric mutual information (MI) estimator based on simplified D-vine copulas. Approximating the joint density with a simplified D-vine allows the density ratio to simplify...
The paper introduces DRO-NPE, a distributionally robust neural posterior estimation method that replaces the standard NPE objective with a worst‑case loss over a Wasserstein ambiguity set. It presents KL‑based metrics for miscoverage and miscalibration, demonstrating that the DRO‑NPE objective mitigates overfitting and reduces posterior overconfidence. Experiments on benchmark simulation‑based inference tasks show that DRO‑NPE consistently improves coverage and calibration while narrowing the gap between empirical and population NPE loss, especially in low‑simulation regimes.
The paper introduces UINO-FSS, a unified framework for few-shot semantic segmentation that merges knowledge from DINOv2 and SAM into a single-encoder architecture. It employs coarse-to-fine multimodal distillation, a bottleneck adapter for embedding alignment, a meta-visual prompt generator, and a mask decoder, enhanced by Mamba-based 4D correlation mining. Experiments on PASCAL-5i and COCO-20i demonstrate state‑of‑the‑art performance in 1‑shot settings, achieving 80.6% mIoU on PASCAL-5i and 64.5% on COCO-20i.
The paper investigates how a reviewer in multi‑agent language‑model systems can be authorized to override an upstream agent’s decision. It compares unrestricted override authority with an evidence‑gated approach that requires a valid record of a binding‑constraint violation and a compliant repair before replacement is allowed. Experiments on two open‑weight models across 96 spatial‑policy tasks show that the evidence gate sharply reduces corruption of correct proposals while largely preserving legitimate repairs, though the effectiveness varies with reviewer transitions and upstream states.
The paper introduces ReScopeBench, a benchmark designed to evaluate how agents manage the lifecycle of experience in evolving environments. It differentiates between structural experience—knowledge about how an environment is organized—and rule experience—principles that guide decisions across tasks. Using an experience-first construction approach, the study tracks how agents acquire, reuse, and revise experience across task streams, revealing that while retained experience boosts performance, agents often fail to keep it current when environments change.
The paper proposes a new reporting standard for attributing agent failures to their history, insisting on intervention‑verified evidence rather than post‑hoc taxonomies. It outlines a method to freeze an executable mid‑trajectory state, alter only the agent’s perceived history, and continue execution to obtain a measured replay certificate. The authors review existing manipulation classes and highlight that while several methods perform interventions, none currently report a quantified replay certificate, and they provide two cautionary examples illustrating the pitfalls of lacking such verification.
PersonaCodeBench is a new benchmark that evaluates coding agents in multi‑turn, persona‑driven sessions, turning user characteristics into verifiable obligations. The benchmark’s persona pool covers roles, technical abilities, interaction preferences, and practical constraints, generating task‑grounded, evolving requirements for implementation, permissions, testing, explanation, and delivery. Evaluation uses a controller to manage user simulator responses, online requirement evaluation, and separate verifiers for native correctness and persona fulfillment, with a joint pass condition requiring adherence to budget and validity conditions.
The paper evaluates FlyHash, a locality‑sensitive hashing scheme inspired by the fruit fly’s olfactory circuit, for use in large language model (LLM) agent memory retrieval. It demonstrates that FlyHash maintains a quality advantage over SimHash when bits are matched, and that this advantage persists with real agent‑memory relevance labels. However, FlyHash’s larger, slower codes and the dominance of dense‑vector indexing (HNSW) limit its practical benefits in terms of memory and latency savings.
The paper introduces RECAP, a method that improves speculative decoding for vision‑language models by recycling a target model’s own predictions. RECAP stores predictions using one‑, two‑, and three‑token keys, allowing contextual memories that are request‑specific and token‑level memories that persist across requests. By fusing retrieved predictions based on entropy‑confidence weighting and using best‑first search to build a candidate tree, RECAP achieves significant speedups over autoregressive decoding without requiring training or a draft model.
The paper surveys how graphs are increasingly used in large language model systems to support reasoning, knowledge grounding, retrieval, memory, execution, and multi‑agent coordination. It introduces an operation‑level taxonomy that distinguishes the function of each graph operation, the component that computes it, and how its output is subsequently used. The authors compare graph observations with external computations, analyze memory updates, and examine execution dependencies, then propose an evaluation framework that organizes benchmarks, diagnostics, and system costs, concluding with open challenges in adaptive computation, memory maintenance, and dependency‑aware recovery.
The paper introduces BiSID-AVS, a method that uses binaural audio cues to disambiguate which visual instance is actually producing sound when multiple same‑class objects are present. It grounds category‑consistent candidate regions with audio prompting, projects binaural interaural differences into a spatial response map, and refines the true sounding instance through dual‑level spatial conditioning. Experiments on binaurally extended AVS‑Synthetic and Habitat‑AVS datasets show significant gains in instance‑level mIoU and reduced false activation on non‑sounding same‑class regions.
ViBench 2.0 is a new benchmark for long‑horizon software building, featuring 17 full‑stack web products that evolve through 241 feature requests. The benchmark evaluates models only when the entire product is finished, using 544 test plans per product to ensure a strict grading protocol. Initial results show a reliability gap: top models pass most test plans but rarely deliver fully working products, though recent releases have improved performance significantly.
The paper investigates how explicit reasoning in language models can lead to a "reasoning‑trace overflow," where the model spends most of its 32k‑token output budget on reasoning rather than producing code. Across seven models and six code benchmarks, overflow is rare on simple tasks but common on harder benchmarks, often due to repetitive reasoning loops. The study shows that disabling reasoning can sometimes be competitive, while prompt steering and budgeted transitions can mitigate overflow and improve token efficiency.
HarnessFlow is a framework that automates the optimization of executable harnesses for large language model agents using a Monte Carlo Tree Search policy to balance exploration and exploitation. It refines harness candidates through a coding agent that leverages archived code, execution feedback, and reusable mechanisms, operating within a limited evaluation budget. Across six benchmarks covering agentic tasks, coding, and web interaction, HarnessFlow achieves the highest mean test scores, improving average performance by 10.89 percentage points over initial harnesses and 3.95 over Meta-Harness, while also reducing per-task execution costs.
LegalEvolver is a query‑optimization framework for legal judgment prediction that generates legally informative queries to improve retrieval quality. It uses heuristic query seeding, summarizes rollout experiences into rubrics, and applies rubric‑guided reinforcement learning for iterative self‑evolution. Experiments on large benchmarks show that evolving rubrics outperform fixed ones and that rubric guidance boosts performance beyond direct supervised fine‑tuning plus RL.
ServerBench is a benchmark that lets large language model (LLM) coding agents design and test specialized LLM serving systems. The benchmark evaluates agents on a single objective—maximizing latency‑qualified token value per dollar of GPU cost—using stateful replay to preserve session dependencies and independent verification to confirm system correctness. In experiments with 11 coding‑agent models, Claude Sonnet 5.5 achieved 2.97 dollars per dollar on Gemma E4B, outperforming the vLLM reference by 4.87×, and the study highlights key design insights such as coordinating capacity with load selection, interpreting performance feedback through relevant experiments, and verifying execution paths for reliable delivery.
The paper introduces GBPNet, a Generalized Belief Propagation Network that maintains message‑passing symmetry while improving learning capability for decoding linear codes. GBPNet employs cascaded variable‑node updates and learned element‑wise message scaling, preserving the Tanner graph structure. Two training strategies—progressive training and high‑SNR training—are proposed, and experiments show GBPNet achieves lower bit error rates than conventional belief propagation and other state‑of‑the‑art neural decoders.
VIPBench is a new benchmark that measures how humans perceive voice identity, using 124,876 same-or-different judgments from 1,290 listeners on 9,800 recording pairs involving real voices, voice clones, and morphed voices of 100 public figures. It defines four tasks to evaluate how well speaker‑representation models align with human judgments, including predicting majority responses and assessing similarity across real and synthetic recordings. Eleven publicly available speech models were evaluated, showing that listeners often misidentify cloned voices and that no model reaches the noise ceiling, with the best explaining only 61% of the explainable variance.
OdorSpaces is a physics‑based simulation platform that extends Habitat‑Sim with 3D odor transport and dispersion modeled by Fire Dynamics Simulator, allowing agents to navigate photorealistic indoor environments using both visual and olfactory data. The benchmark presents two source‑seeking tasks—an invisible gas‑leak and an early‑stage fire—providing a realistic testbed for visual‑olfactory embodied navigation. An initial multimodal reinforcement learning policy that fuses visual observations with temporal olfactory measurements outperforms several baselines, establishing a strong reference for future research.