Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.
Large vision-language models (LVLMs) have shown strong potential for embodied vision-language navigation (VLN), yet their performance degrades on long-horizon trajectories due to context-length limits...
The paper introduces Autonomous Science Bench, a benchmark for AI agents that must choose experiments under a limited budget. It features six real-world design-build-test-learn campaigns in biology, chemistry, and materials science, where agents select design batches, receive lab measurements, and iteratively refine a predictive model. Success is measured by two calibrated gates—discovery and learning—highlighting distinct agent weaknesses that overall success rates alone would mask.
The paper introduces Randomized Block Curation (RBC), a method for selecting useful lessons from language‑model agents’ persistent textual memory. RBC evaluates the impact of grouped lessons through randomized experiments on practice tasks, retaining only those that demonstrably improve task success. Experiments across two memory systems, three models, and three benchmarks show RBC raises average success to 66.9, outperforming both full playbooks and agent‑chosen lessons.
The article compares three sampling methods—NumPyro’s NUTS, the affine‑invariant MCMC sampler Emcee, and the nested sampling algorithm dynesty—by benchmarking them against an exact ground truth. It evaluates how each sampler performs and documents their failures in practice. The comparison provides insights into the strengths and limitations of each approach for posterior inference.
The paper reproduces five factuality alignment methods for large language models—three on‑policy RL approaches (PPO, GRPO, Dr.GRPO) and two offline preference optimization methods (DPO, KTO)—using a common benchmark and evaluation metrics. None of the methods consistently outperforms the base model, and the rewards appear to favor format, length, or brevity rather than factuality or relevance. The study also highlights problems with current evaluation practices, noting that metrics and benchmarks often fail to measure prompt relevance and rely on fragile LLM judges and retrieval for verification.
We investigate whether the past of an auxiliary time series provides predictive information about a target beyond that contained in the target's own history. We compare matched out-of-sample target-on...
TextualRL is a gradient‑free reinforcement learning framework that updates the textual context guiding large language model agents without altering their parameters. It critiques groups of rollouts from the same task, contrasts successful and failed runs, and uses cross‑group evidence to propose and validate context edits. Across six benchmarks, TextualRL boosts test accuracy by 24.4 and 15.3 percentage points for GPT‑5.5 and Qwen3.8‑27B, respectively.
FlowCred is a two-level framework designed to jointly post-train text-to-video models by separating prompt-level credit from generator residuals and coordinating prompt optimization with generator updates. It uses hierarchical credit assignment with matched prompt-seed rollout grids and functional coordination that probes alternative prompts during actual generation trajectories. Across multiple benchmarks, FlowCred consistently outperforms joint-training baselines, improving performance on T2V-CompBench and StoryEval without increasing decoding or reward-model call budgets.
The paper introduces OSVerify, an algorithmic framework for one-sided verification that prioritizes rare valid solutions while filtering out invalid candidates to reduce human review workload. OSVerify separates error proposal—searching for errors across solution granularities via hierarchical decomposition—from error validation, which distinguishes major errors from minor or hallucinated ones. Experiments on competition- and research-level benchmarks show a 46% reduction in human reviews of invalid solutions compared to voting baselines, increasing to 57% after post-training the validator. The framework is applied to identify errors in published research papers and to resolve three Langlands open conjectures, confirming its utility in human‑AI research collaboration.
LADDER is a long‑horizon robot manipulation benchmark that scales from 1 to 10 subgoals within the same environment, covering 145 tasks across three suites and two embodiments. The benchmark evaluates low‑level policies and orchestration methods, showing that while low‑level policies perform well on single‑subgoal tasks, they fail on extended tasks, and orchestration improves performance but is limited by execution errors. Tool‑selection orchestration with GPT‑Policy achieves the highest success rate (~62%) but at significant time and cost.
Generalizable AI-generated image (AIGI) and deepfake detection remain challenging due to the asymmetry phenomenon, where adaptation to limited training distributions can degrade the transferable repre...
Diffusion language models (DLMs) enable flexible parallel generation by resolving tokens according to the model's evolving predictability rather than a fixed left-to-right order. This flexibility, how...
Recovering high-quality images from degraded underwater observations is crucial for underwater exploration. Polarization observations show promise as an effective source of complementary physical cues...
PaceReasoner is a label‑free, test‑time reinforcement learning framework that improves short‑mode accuracy by leveraging a model’s long responses. It shares a trainable policy between long and short modes, using progressive, confidence‑weighted supervision and a behavior reward that shifts from encouraging short exploration to rewarding agreement with long answers. Across three backbones and four benchmarks, it raises short‑mode accuracy by 7.4–13.5 percentage points while keeping responses under 25 tokens on average.
Large language model (LLM) agents are increasingly used for vulnerability analysis, yet existing benchmarks provide limited coverage of exploitation that depends on interactions among network protocol...
The paper introduces MHVF, a training‑free framework for GUI grounding that uses multi‑hypothesis exploration and local visual verification with a single pretrained vision‑language model. MHVF generates spatial hypotheses from full‑screen predictions, refines them through zoomed re‑localization, and verifies candidate agreement with the instruction to resolve ambiguity. Experiments on eight benchmarks with five models show significant accuracy gains, notably improving UI‑Venus‑Ground‑7B by 4.3 pp on Mind2Web and raising Qwen2.5‑VL‑7B’s ScreenSpot‑Pro accuracy from 23.6 % to 53.6 %.
Medical multimodal reasoning models are typically trained on teacher-generated rationales, discarding the valid reasoning in students' own failed trajectories, while general-domain correction methods...
LatentRiskBench is a new benchmark designed to evaluate state-dependent safety risks in enterprise agents, where the safety of an action depends on the workflow state rather than the action alone. It pairs risky trajectories with benign ones that share the same terminal action but differ in latent policy-relevant states, forcing models to reconstruct context instead of relying on action cues. The benchmark introduces residual risk metrics and demonstrates that state-level evaluation detects 96% of violations compared to only 9% for action-level methods, highlighting the inadequacy of existing guardrails.
OpenDepth repurposes Image-to-Image (I2I) editing priors to improve monocular depth estimation in open-world conditions. By aligning shallow features in a Diffusion Transformer and adding a joint RGB reconstruction task, the method decouples environmental style from geometry, enabling robust depth inference across varied lighting and weather. The approach achieves new state‑of‑the‑art performance on standard and extreme weather benchmarks, maintaining structural integrity in zero‑shot settings.
Diffusion language models accelerate generation by decoding multiple masked positions per forward pass. However, accuracy is known to drop when parallel decoding trajectories deviate from their sequen...