Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

28,111 stories · RSS feed

Hugging Face Trending Papers
Oct 29

Spectral Invariant Fusion and Refinement for Domain Generalized Semantic Segmentation

Spectral Invariant Fusion and Refinement for Domain Generalized Semantic Segmentation (SIFR) introduces a frequency-aware dual-stream framework that addresses domain shift by performing spectral-guided invariant fusion of frozen CLIP and DINOv2 features and a spectral decoder refinement module. The fusion stage filters out domain-sensitive responses while preserving complementary semantic and structural information, and the decoder stage stabilizes multi-scale features through low-frequency statistics-based normalization and adaptive frequency selection. Experiments on synthetic-to-real, real-to-real, and adverse-weather benchmarks show consistent improvements over previous state-of-the-art methods.

Hugging Face Trending Papers
Oct 29

Open-set multi-view seahorse re-identification from underwater photograph sequences

The paper introduces novel techniques for taxonomic classification, sex determination, and open‑set re‑identification of seahorses using sequences of underwater photographs. By training on a large dataset of 35,914 images from 474 individual seahorses, the authors show that aggregating information across multiple photos achieves state‑of‑the‑art performance. This approach enables more accurate, minimally invasive Capture‑Mark‑Recapture studies at scale.

Hugging Face Trending Papers
Oct 29

Grammar Control for Language Learning with Sequential Monte Carlo and Small Language Models

The paper presents a method that uses sequential Monte Carlo (SMC) to guide small language models toward generating specific target grammar structures for second language learners. By applying a minimal set of hand‑coded potential functions, SMC achieves near‑perfect coverage of the desired grammar, allowing these lightweight models to outperform state‑of‑the‑art models that are 140 times larger. The authors also demonstrate that code synthesis with frontier models can automatically generate new potentials, enabling the approach to scale to hundreds of grammar skills without additional training or supervised data.

Hugging Face Trending Papers
Oct 29

WorldProbe: When Video World Models Fall Apart under Long-Horizon Spatial Exploration

WorldProbe is a new diagnostic benchmark that tests video world models by letting a goal‑conditioned multimodal agent adapt camera actions online during generation, creating 164 test cases across four probing families. The benchmark evaluates 10 state‑of‑the‑art models over more than 1,600 rollouts, totaling roughly 800 minutes and horizons up to 180 seconds, revealing that while local visual and content quality can be maintained for longer, long‑range memory and evolution remain weak. The study highlights a gap between locally stable generation and persistent world consistency that existing short‑interaction benchmarks miss.

Hugging Face Trending Papers
Oct 29

Practical Assessment of Agentic Systems in Acoustics

The paper evaluates how large language models, when integrated into a harness that supplies information and executes software, perform as agents in acoustic engineering tasks. In one experiment the agent accurately detects and locates faults in vehicle interior audio recordings, showing that a weak model limits performance regardless of harness design. In a second case study the agent reliably designs metamaterial unit cells to meet specified insertion loss criteria, illustrating how different harness components influence agent behavior.

Hugging Face Trending Papers
Oct 29

EditAgent: Rethinking Agentic Image Editing

EditAgent rethinks agentic image editing by expanding both the action and observation spaces. It routes each editing step to general editors and specialized methods, attaching verification records that allow off‑the‑shelf VLM judges to outperform finetuned reward models. The approach yields parallel candidate generation and a single listwise judgment, which experts prefer over sequential refinement in 78.5% of cases and improves two open‑weight editors across three benchmarks.

Hugging Face Trending Papers
Oct 29

From Intent to Improvement: Agentic Dataset Search for Better Model Training

The paper introduces AgenticDS, a contract-governed agentic framework that transforms a training data query into a typed search plan and executes it through deterministic operators, ensuring that retrieved datasets meet specified constraints. By verifying provenance‑grounded evidence before ranking, AgenticDS outperforms existing hybrid retrieval and commercial LLM agents, improving NDCG@10 by 25.7% on the HuDS‑Bench benchmark. A controlled search‑to‑train study demonstrates that the datasets found by AgenticDS lead to superior downstream accuracy in mathematics, code, and medicine tasks.

Hugging Face Trending Papers
Oct 29

Compile Away Uncertainty: Self-Evolving Agent Skills via Programmatic Compilation

The paper introduces SkillCompiler, a framework that transforms self‑evolving agent skills into executable programs to reduce uncertainty from repeated LLM reasoning. It decomposes skills into fine‑grained operations, selectively compiles deterministic parts while retaining LLM reasoning for context‑dependent steps, and reuses validated programs across evolving skills. Experiments on five benchmarks and six model families show SkillCompiler outperforms state‑of‑the‑art methods, improving task performance by up to 32.33%.

Hugging Face Trending Papers
Oct 29

CBE: A Complexity-adaptive Boundary-aware Ensemble Method for Imbalanced Data

The paper introduces CBE, a complexity‑adaptive boundary‑aware ensemble method designed to handle class imbalance in classification tasks. It links sampling weights to data complexity, dynamically adjusting boundary preference during under‑sampling, and employs a multi‑level complexity measure to categorize datasets into Easy, Median, and Hard. Experiments on 24 UCI and KEEL datasets show that CBE improves F‑measure and G‑mean on Easy and Median cases without harming performance on Hard cases.

Hugging Face Trending Papers
Oct 29

Unlimited OCR Works 2

Unlimited OCR 2 is a document parsing model that processes only the text it transcribes, using Sparse Reference Sliding Window Attention to keep computational costs constant regardless of document length. It builds on Unlimited OCR by replacing all decoder attention modules, enabling efficient parsing of documents with hundreds of pages. The model achieves significant per‑step speedups over Full Attention and R‑SWA, and sets new state‑of‑the‑art results on the MPDocBench and MarathonDocBench benchmarks for long‑document OCR.

Hugging Face Trending Papers
Oct 29

Youtu-Parsing-Omni: One Encoder, One Schema, All Modalities

Youtu-Parsing-Omni is an omni‑modal parser that transforms documents, images, audio, and video into structured records grounded in space and time, enabling agents to read, locate, and cite evidence across modalities. It uses a single encoder—initialized from a pre‑trained language model—to encode all modalities with shared weights in a unified space‑time coordinate system, while a few fusion layers allow video frames to attend to concurrent audio for cross‑modal fusion and temporal alignment. The system unifies ten parsing tasks under one schema, employing a self‑evolving post‑training procedure called OmniSchema‑Aware On‑Policy Distillation (OSAD) that co‑evolves the parser and its supervision, achieving state‑of‑the‑art performance on OmniDocBench v1.6 and leading open‑weight models on OmniParsingBench, with strong results on audio, video, chart‑parsing, and chemical‑fingerprint agreement.

Hugging Face Trending Papers
Oct 29

PCM: A Proof-Carrying Message Protocol for Verifiable Multi-Agent Communication

PCM is a protocol that attaches verifiable evidence to every claim sent between AI agents, using immutable Merkle logs and signed attestations that the sender cannot tamper with. Receivers evaluate messages offline and classify each claim as proven, unproven, or failed, ensuring that self‑reported claims can never be proven and that authority is traceable through recorded human approvals. In a benchmark of 926 real messages, PCM flagged only 10 stale‑evidence lies (1.4%) while other protocols accepted many more, and verification runs in about 1 ms with perfect agreement between independent verifiers.

Hugging Face Trending Papers
Oct 28

FairSplit: Decomposing the Embedding Space for Fair Classification

FairSplit decomposes the embedding space into a Fair Space for prediction and a Sensitive Space that isolates sensitive information, using a learnable mask and the Hilbert‑Schmidt Independence Criterion (HSIC) to guide the separation. The method simultaneously trains for classification accuracy and independence from sensitive attributes, providing theoretical bounds on the demographic parity gap based on the Fair Space dimensionality and HSIC. It supports multiclass tasks and non‑binary sensitive attributes, achieving competitive accuracy‑fairness trade‑offs on established benchmarks.