Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

28,111 stories · RSS feed

Hugging Face Trending Papers
4d ago

Image Guided Depth Upsampling using Anisotropic Total Generalized Variation

The paper introduces a new depth image upsampling technique that tackles the low lateral resolution of modern depth cameras by formulating a convex optimization problem with higher‑order regularization. An anisotropic diffusion tensor derived from a high‑resolution intensity image guides the upsampling, and a primal‑dual algorithm is implemented for efficient, multi‑frame‑per‑second performance. Experiments on the Middlebury 2007 datasets demonstrate that this method surpasses state‑of‑the‑art approaches in both speed and accuracy, and the authors also present new datasets with precise ground truth to benchmark depth upsampling using real sensor data.

Hugging Face Trending Papers
4d ago

MultiInpaintCOCO: Fine-Grained Visual Understanding in Multiple Languages

MultiInpaintCOCO extends the English InpaintCOCO dataset to 13 typologically diverse languages, creating a multilingual benchmark of 953 minimal-pair examples focused on fine-grained visual details. The study evaluates state-of-the-art multilingual multimodal retrieval models, revealing that performance drops significantly for lower-resource languages and that many models exhibit a systematic bias toward English captions. The benchmark and code are publicly available at https://github.com/ufal/multi-inpaint-coco.

Hugging Face Trending Papers
4d ago

Top-$k$ Retrieval Leaves Memory Value Undetermined

The paper examines how a memory‑augmented agent can determine the value of its memories using only retrieval logs. It shows that top‑k retrieval fails to assign any value to memories regardless of the number of episodes logged, and that a single question can determine at most one memory’s value. To address this, the authors propose LEDGER, which estimates determined Shapley values with confidence intervals, merges memories with identical histories, and strategically retrieves to increase determinability, achieving a substantial rise in determined memories on HotpotQA and LoCoMo.

Hugging Face Trending Papers
4d ago

CUA-Master: Scaling Computer-use Agents through Open-World Recursive Improvements

CUA-Master introduces a nested double-loop recursive improvement framework to scale computer-use agents. The outer flywheel continuously integrates open-world data and model weaknesses to guide data construction and optimization, while the inner flywheel mines instructional videos for seed tasks and propagates them into reusable primitives for diverse task synthesis. The system also employs a feedback-guided snapshot-rollback retry mechanism for reliable trajectory collection, achieving competitive performance on multiple benchmarks and demonstrating continuous gains across iterations.

Hugging Face Trending Papers
4d ago

VLM-as-a-Judge for Person Re-Identification

The paper demonstrates that a frozen vision‑language model (VLM) can serve as a zero‑shot judge to rerank candidate lists for cloth‑changing person re‑identification (CC‑ReID). Using the VLM to rerank two expert models on the CHIRLA benchmark improves rank‑1 performance by an average of 12.9 points over the best expert and 9.4 over a naive ensemble, without any fine‑tuning on target data. By gating the VLM’s use to cases where the experts disagree at rank‑1, the authors reduce compute by 40% with minimal impact on rank‑1 accuracy.

Hugging Face Trending Papers
4d ago

GraTED: Graph--Tensor EEG Decoding for Passenger Hazard Perception in Autonomous Vehicles

GraTED is a Graph–Tensor EEG Decoding Model that preserves the spatial and temporal structure of passenger electroencephalography (EEG) data by modeling electrode positions as an undirected graph and applying graph convolution and low‑rank tensor classification. It is designed to predict risk perception and identify danger in autonomous‑vehicle scenarios. On the PassengerEEG dataset, GraTED outperforms seven baseline methods in balanced accuracy while using the fewest learnable parameters.

Hugging Face Trending Papers
4d ago

RxnMiner: A Multimodal Agent for Structured Reaction Data Extraction from Chemistry Figures

RxnMiner is a multimodal agent that extracts structured reaction data from chemistry figures by combining expert-informed procedures with a chemistry toolbox. It reads molecular structures, resolves shared and entry-specific evidence, and reconstructs reaction records from templates. On the ChemRxn-Bench benchmark, RxnMiner improves stereochemistry-aware reaction macro‑F1 from 21.0% to 71.0%, outperforming existing systems and generating a large silver corpus of 88,202 reaction records from 10,107 figures.

OpenAI Blog
4d ago

LifeVal

LifeVal is a new benchmark introduced by OpenAI that evaluates how effectively AI assists users in completing everyday tasks. The benchmark covers a range of activities, from locating childcare options to handling finances and navigating government paperwork. By measuring AI performance across these practical scenarios, LifeVal aims to gauge real-world usefulness of AI systems.

Hugging Face Trending Papers
4d ago

Unlocking the Potential of Vision Foundation Models for Multi-Task Dense Prediction via Mixture of Task Adapters

The paper introduces D2MoE, a Mixture of Task Adapters designed for vision foundation models to handle multi-task dense prediction. It separates adaptation into a task‑generic low‑rank channel (TG‑LoRA) that preserves pretrained priors and a task‑adaptive sparse expert module (TA‑MoE) that resolves task‑specific conflicts. A Semantic Prior Injector further aligns multi‑level task features with frozen foundation knowledge, achieving robust multi‑task performance with a small trainable footprint.

Hugging Face Trending Papers
4d ago

EffComp: Efficient Prompt Compression via Hybrid Reinforcement & Supervised Learning

EffComp is a framework for efficient prompt compression in RAG-based question answering. It uses BERT-style document reranker and sentence selector models to perform fast, extractive compression at the sentence level, thereby avoiding hallucinations. The hybrid reinforcement and supervised learning training optimizes compression ratio while maintaining LLM accuracy, and experiments show it outperforms existing methods with minimal latency.

Hugging Face Trending Papers
4d ago

AffToM: Benchmarking Recursive Affective Theory of Mind in Dynamic Multi-Party Interactions

AffToM is a new benchmark for affective Theory of Mind (ToM) in dynamic multi‑party interactions, comprising 1,922 test pairs and 19,759 training pairs. It evaluates Individual Affective Reasoning (IAR) and Multi‑party Affective Reasoning (MPAR), the latter tracking nested beliefs and emotions as events unfold. The authors introduce an event‑driven multi‑agent generation pipeline and a two‑stage post‑training framework, AffToM‑R1, which boosts MPAR accuracy by 13.38 percentage points over baseline models.

Hugging Face Trending Papers
4d ago

Geometry-Constrained Dynamic Hypergraph Convolutional Network with Contrastive Score Refinement for Skeleton-Based Action Recognition

The paper introduces DyGCN, a Geometry-Constrained Dynamic Hypergraph Convolutional Network designed for skeleton-based action recognition. It features a Dynamic Hypergraph Construction Network with Contrastive Score Refinement (DyGCT) that includes a GeoHC module for action-specific hyperedge construction and a CSR module that refines pairwise compatibility scores via prototype-based contrastive learning during training. Experiments on NTU RGB+D, NTU RGB+D 120, and Northwestern-UCLA datasets demonstrate state‑of‑the‑art performance, and the source code is publicly available.

Hugging Face Trending Papers
4d ago

PriCBMs: Multi-granular Concept Bottleneck Vision-Language Models for Visual Privacy Recognition

PriCBMs introduces Privacy Concept Bottleneck Models that use intermediate privacy-related concepts to identify category-relevant privacy cues via visual-concept matching before predicting privacy categories. The approach incorporates a concept selection strategy based on relevance, discriminability, and diversity, and adds a global-region modeling module to capture localized privacy evidence while preserving global context. Experiments on VISPR and FineViP demonstrate that PriCBMs outperform competitors across all four multi-label metrics, with ablation studies confirming the effectiveness of the design.

Hugging Face Trending Papers
5d ago

Latent Diffusion-GAN: Adversarial Learning in the Autoencoded Latent Space

Latent Diffusion-GAN (LD‑GAN) integrates adversarial learning into diffusion models by using the pretrained VAE as an energy‑based discriminator, avoiding extra parameters and preserving the latent priors. It introduces a structural consistency energy to align encoder and decoder features, improving perceptual quality. Experiments show LD‑GAN enhances sample fidelity, sharpness, and diversity across generation tasks while maintaining efficient training dynamics.

Hugging Face Trending Papers
5d ago

BGD-SF PolySegNet: boundary-guided dynamic selective fusion network for robust polyp segmentation in colonoscopy images

The paper presents BGD‑SF PolySegNet, a boundary‑guided encoder‑decoder network designed to improve polyp segmentation in colonoscopy images. It introduces three novel modules—BGD‑CMSCA, BG‑SAGF, and MBGH—that explicitly use boundary information to guide dynamic context aggregation, selective feature fusion, and mask refinement. Evaluated on multiple datasets, the model achieves state‑of‑the‑art Dice and boundary metrics while running at 35 FPS with 22.51 M parameters.

arXiv AI
5d ago

LLM Detoxification via Transition-Aware Reward-Guided Representation Sampling

The paper introduces Transition-Aware Reward-Guided Representation Sampling (TARGRS), a test-time detoxification framework for large language models that learns reward guidance over toxicity transitions in latent representation space. TARGRS constructs trajectories from non‑toxic to toxic states, uses toxicity scores along these paths to train a token‑level reward model, and then guides coarse‑to‑fine sampling in the opposite direction to produce detoxified outputs with minimal perturbation. Experiments on ten LLMs and three LVLMs show that TARGRS consistently outperforms existing baselines, reducing toxicity by up to 69.38% on RealToxicityPrompts and up to 64.23% on LVLMs while maintaining near‑original speed and core capabilities.

By Aishan Liu, Yisong Xiao, Xianglong Liu, Yongxin Huang, Siyuan Liang, Zonghao Ying, Yang Qu, Jian Yang, Yaodong Yang, Dacheng Tao
arXiv Machine Learning
5d ago

Before We Adapt: The Zero-Shot Generalization Bottleneck in Continual World Models

The paper investigates how model-based reinforcement learning can support continual learning by reusing prior knowledge. It introduces a contextual Markov decision process that separates behavioral and physical context, and studies zero‑shot reuse in FourRooms tasks using Dyna‑Q and TD‑MPC. The experiments show that zero‑shot reuse degrades under physical context changes and that this degradation is strongly tied to the observation representation, highlighting representation generalization as a key bottleneck.

By Amir Mesbah, Di Wu
arXiv Machine Learning
5d ago

CausalXRL: Explainable Reinforcement Learning through Causal Graph Reasoning

CausalXRL is a new framework that makes reinforcement learning agents explainable by using causal graph reasoning. It offers transparent, structured, multi‑level explanations of agent decisions. The authors validate the approach with quantitative tests on benchmark RL environments and a qualitative expert study on the real‑time strategy game MicroRTS, showing faithful explanations and efficient causal‑structure learning.

By Yanming Zhang, Eric Papenhausen, Klaus Mueller
arXiv Machine Learning
5d ago

StrictPRMBench: A Replayable Evaluation Benchmark for Auditing Fixed-Score PRM Aggregation

StrictPRMBench is a replayable evaluation benchmark designed to audit fixed‑score PRM aggregation rules. It standardizes mathematical reasoning pools, scorer evidence, and deletion interventions so that different reducers confront identical selection problems. Across seven checkpoints and two benchmarks, the benchmark reveals that product aggregation yields higher deletion‑induced selection instability than geometric mean, with statistically significant differences in most cells.

By Pengfei Zheng
arXiv AI
5d ago

From 1D Benchmarks to Complex 3D Geometries: Can Agents Reliably Specify Problems for Automated PDE Simulation?

The paper investigates whether AI agents can reliably generate complete, executable JSON specifications for automated PDE simulation, covering geometry creation, meshing, solving, and result export. Using a workflow that combines model extraction, deterministic scientific checks, clarification, and validated local patches, the authors evaluate specification correctness, execution completion, and numerical verification across 120 template-related requests. The approach achieves 98.3% correct first-turn specifications, outperforms schema-assisted and model-only baselines, and demonstrates successful automated execution in 30 complex 3D cases and 75% success in live user-to-results tests.

By Akshat Jani, Sakhinana Sagar Srinivas, Venkataramana Runkana