Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

28,111 stories · RSS feed

Hugging Face Trending Papers
Oct 27

GroundControl: Satellite-Conditioned Multi-View Diffusion for Reconstruction-Free Novel View Synthesis

GroundControl introduces a satellite‑conditioned multi‑view diffusion model for novel view synthesis that uses satellite images alongside street‑level inputs. The cross‑attention conditioning preserves pretrained diffusion priors while ensuring realistic spatial layouts, even when target views have little overlap with source views. Experiments on outdoor datasets demonstrate superior performance over both satellite‑only and street‑only baselines without relying on 3D reconstruction.

Hugging Face Trending Papers
Oct 27

M4Bench: Evaluating Procedural Specification for Clinical EHR Derivation Agents

M4Bench is a benchmark comprising 28 clinical derivation tasks across 15 families on MIMIC‑IV and eICU datasets, designed to evaluate whether AI coding agents can reproduce complex procedural specifications for electronic health record (EHR) variable derivation. The study shows that incorporating specification completeness into agent skills yields the largest performance gains, with modest additional benefits from delivery format and schema familiarity. The benchmark, along with tasks, skills, evaluation code, and sanitized evidence, is released as open source for clinician‑reviewable use.

Hugging Face Trending Papers
Oct 27

Channel Selection for Mixture-of-Experts Decoding Needs No Calibration Data

The paper demonstrates that Mixture-of-Experts (MoE) decoding can avoid the need for calibration data by using a top‑k channel selector instead of a calibrated threshold. Top‑k selection consistently matches or outperforms threshold‑based methods across various read budgets and tasks, eliminating issues like repetition‑loop collapse. Additionally, the authors propose a reader that uses a ternary sketch and low‑rank mapping to recover skipped channels, achieving 98.1% of full‑model accuracy while reading only 17.8% of routed‑expert weights and speeding up MoE blocks by up to 2.69× compared to optimized vLLM.

Hugging Face Trending Papers
Oct 27

Euclidean Crowding-Driven Adaptive Euclidean-Hyperbolic Graph Learning

The paper investigates why hyperbolic space outperforms Euclidean space for hierarchical graph data. It quantifies Euclidean crowding by showing that fixed‑dimensional Euclidean embeddings of tree‑like graphs inevitably suffer layer‑wise crowding, leading to weaker generalization bounds. The authors then derive a generalization bound that explains hyperbolic geometry’s advantage and introduce EH‑Fusion, a framework that maps Euclidean GNN representations into hyperbolic space and adaptively fuses both geometries based on neighborhood expansion patterns, with experiments confirming its effectiveness.

Hugging Face Trending Papers
Oct 27

TopoField: Technical Report for ECCV 2026 SceneFun3D Challenge

TopoField is a deterministic decoder built on VOLT that addresses the field-to-instance gap in functional segmentation of indoor point clouds. It aggregates 13 test-time forward passes into a semantic consensus field, decodes instance candidates, and ranks them class‑aware, using a graph‑geodesic partition to separate high‑confidence cores from support regions. On the ECCV 2026 SceneFun3D Challenge, TopoField achieved 28.286% AP, 41.383% AP50, and 47.558% AP25, earning second place.

Hugging Face Trending Papers
Oct 27

Does Your Generalist Skill Optimizer Really Incentivize Agentic Tasks? Rethinking Skill Optimization via Bipartite Attribution

The paper introduces BiPA, a method to evaluate skill optimizers that rewrite natural‑language procedures for agents performing long‑horizon tasks. BiPA replays the optimizer’s acceptance rule on identical candidates to isolate selection effects and records each episode as a bipartite graph of turns and read text, revealing that learned skills only marginally outperform hand‑written seeds and that many gains vanish on test data. The study finds optimizers add extra input tokens and turns, and proposes four design principles for more reliable skill editing.

Hugging Face Trending Papers
Oct 27

Beyond Parameter Protection: Fisher-Shaped EWC for Replay-Free Continual Learning

The paper introduces Fisher-Shaped EWC (FS‑EWC), a method that extends Elastic Weight Consolidation by controlling Fisher sensitivity over a neighborhood of potential future updates. FS‑EWC minimizes the worst‑case empirical Fisher information trace during task learning, favoring solutions less sensitive to bounded future displacement, and uses trajectory‑based model merging to construct the retained solution. Experiments on multiple benchmarks show that FS‑EWC outperforms standard EWC, achieving a better stability–plasticity balance and greater tolerance to parameter drift.

Hugging Face Trending Papers
Oct 27

SplatScout: Query-Driven Part-Level Referring Segmentation in Gaussian Scenes

SplatScout is a query‑driven, closed‑loop approach for part‑level referring segmentation in Gaussian scenes, placing perception under vision‑language model control. It actively selects views, directs segmentation, and refines predictions by inspecting rendered 3D results. On the newly introduced PartRef‑ScanNet++ benchmark, SplatScout achieves 0.538 mIoU, outperforming existing methods and demonstrating the benefits of iterative refinement and effective tool‑feedback utilization.

Hugging Face Trending Papers
Oct 27

Guided-RLVR: Guided Exploration in Reinforcement Learning with Verifiable Rewards

Guided‑RLVR enhances reinforcement learning with verifiable rewards (RLVR) by adding a lightweight controller that injects semantically meaningful steering tokens into the sampling distribution. This strategy encourages large language models to explore multiple reasoning perspectives, expanding the set of reasoning trajectories. Experiments on mathematical reasoning and code generation tasks demonstrate that Guided‑RLVR consistently outperforms existing RLVR methods while producing more diverse reasoning paths.

Hugging Face Trending Papers
Oct 27

Zooming without Zooming v2: From Seeing Details to Connecting Evidence for Generalizable Multimodal Perception

The paper introduces EvidenceConnect, a data‑synthesis framework that generates multimodal questions by zooming into related image regions to strengthen the capability of connecting evidence—identifying visual facts, linking them to entities, and composing them into answers. By training vision‑language models on this synthesized data with RLVR, the authors produce Zooming without Zooming v2 (ZwZ2), which improves fine‑grained and spatial perception, embodied spatial reasoning, multimodal long‑context understanding, and general visual reasoning. The results demonstrate that targeting the shared evidence‑connection capability yields more generalizable multimodal perception without tailoring data to specific benchmarks.

Hugging Face Trending Papers
Oct 27

The State and Fate of Multilingual, Contextual Evaluation in the NLP World

The paper audits 51 multilingual evaluation benchmarks covering 242 datasets and 219 languages, focusing on coverage, representativeness, and rigor. It finds that while coverage is broad, many languages appear in only one benchmark and entire regions are underrepresented, with low-resource languages evaluated on far fewer tasks than high-resource ones. The study also highlights that most datasets rely on English translations, creating artifacts and limiting cultural relevance, and that benchmark contamination through translation leakage and corpus overlap undermines rigor. Recommendations call for natively constructed, culturally grounded, contamination-aware evaluation frameworks that truly serve the communities they aim to assess.

Hugging Face Trending Papers
Oct 27

LearnPrunerV2: Learnable Progressive Token Pruning for Vision-Language Models

LearnPrunerV2 introduces a fully learnable token pruning framework for Vision‑Language Models, embedding lightweight modules in every backbone layer to estimate token importance by combining visual saliency and query relevance via a shadow token. It identifies a consistent staircase redundancy pattern across layers and employs Budget‑Deterministic Pruning to set fixed per‑layer budgets, yielding a hardware‑friendly pruning schedule. Experiments show that LearnPrunerV2 outperforms prior methods, preserving 99.8% of performance while removing 77.8% of visual tokens on LLaVA‑v1.5, thus achieving a superior accuracy‑efficiency trade‑off.

Hugging Face Trending Papers
Oct 27

Mining the Nineteenth Century for the Thirteenth: Register-Scale Forced Alignment and HTR Bootstrapping via Legacy Regesta and Antiquarian Editions

The paper investigates how nineteenth‑century scholarship—specifically Pressutti’s regesta and partial editions—can aid handwritten text recognition (HTR) for the registers of Pope Honorius III, which lack modern expert transcriptions beyond regnal years 1‑2. By comparing legacy layers to audited transcriptions, the authors identify a three‑tier division of labor: at the orthographic tier, projecting legacy editions onto curial spelling with replay reduces word‑error rate by 1.5 points; at the lexical tier, a legacy lexicon covers 82.2 % of out‑of‑vocabulary words and improves NER performance; and at the topological tier, regesta data help locate entry starts with 73.9 % coverage and estimate a character‑error rate of 7.80 % beyond folio 203. The study shows that while antiquarian scholarship cannot replace modern transcription, it can effectively steer, verify, and segment HTR across untranscribed registers.

Hugging Face Trending Papers
Oct 27

Tail Cues, Principal Corrections: Plug-and-Play Rectification for Open-Set Test-Time Adaptation

Tail Cues, Principal Corrections: Plug-and-Play Rectification for Open-Set Test-Time Adaptation introduces TALOR, a lightweight module that rectifies features during non‑stationary target streams by decomposing them into high‑gain principal and low‑gain tail coordinates using the source linear head as a geometric anchor. TALOR applies a soft‑routed affine correction in the principal subspace, subtracts a drift estimate via weighted ridge regression, and reconstructs the feature for the source head. On ImageNet‑C with six csOOD datasets, TALOR surpasses the next‑best UniEnt by 2.2% H‑score, runs 2.5× faster, and uses only 12% of its GPU memory, while also boosting COME to 68.4% H‑score with minimal memory overhead.

Hugging Face Trending Papers
Oct 27

When All Rollouts Fail: Policy-Compatible Joint Decoding for Hard-Problem Reasoning

The paper introduces Policy-Compatible Reinforcement Learning (PICO), a method that combines distribution-constrained joint decoding with on-policy rollouts to address the learning cliff in reinforcement learning with verifiable rewards (RLVR). PICO guides token-level decoding using expert data while preserving the model’s relative token probabilities, and employs a dynamic top‑k mechanism to keep sampling within a finite divergence budget. This approach yields verified, policy‑compatible trajectories that are jointly optimized with on‑policy rollouts, leading to consistent performance gains over standard RLVR and prior off‑policy baselines across diverse reasoning datasets and agentic benchmarks.

arXiv AI
Oct 27

Local Entropy Fluctuations for Black-Box LLM-Generated Text Detection

The paper introduces a new metric, Relative Peak Prominence Rate (RelPPR), to detect black-box LLM-generated text by measuring localized entropy fluctuations. It shows that existing dual-model and resampling-based detectors fail on instruction-conditioned text, and that RelPPR outperforms them across diverse models and datasets. The authors provide code for their detector on GitHub.

By Xinpeng Wang, Bruce Rushing, Zhizhen Chen, Zhe Zeng
arXiv AI
Oct 27

SPARTA: A Sparse Ternary LLM Accelerator with Adaptive Attention and Unified Datapath for Long-Context Edge Inference

SPARTA is a sparse ternary LLM accelerator designed for long‑context inference on edge devices. It combines a phase‑aware sparse dataflow, a unified compute fabric that maps ternary SpMM/SpMV and bit‑serial attention onto shared processing elements, and an adaptive two‑stage attention engine that reduces KV‑cache traffic. The design achieves significant reductions in KV traffic (4.1‑4.8× over full attention) and outperforms NVIDIA Jetson AGX Orin and AMD Ryzen 7 9700X in generation throughput and energy efficiency, while also surpassing other accelerators in throughput, energy, and area efficiency.

By Yanzhen Zhou, Zhengyu Han, Qunyou Liu, Zheyu Yan, Weihong Xu, Cheng Zhuo