Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

28,306 stories · RSS feed

arXiv AI
Sep 30

Mirror-Score: Calibrated, Inference-only Scoring Exposes the Limits of Sequence-compatibility Ranking in D-peptide Design

Mirror-Score is a calibrated, inference‑only scoring framework that evaluates heterochiral D‑peptide/L‑protein complexes, addressing the inadequacies of raw ProteinMPNN negative log‑likelihood (NLL) ranking. The authors benchmarked 31 crystal complexes across four target families, showing that NLL has poor affinity correlation (Spearman 0.19) while Boltz‑2 mirror‑space co‑folding confidence and pLDDT metrics can correctly rank peptides within a viral‑entry family. The study also proposes a prospective design protocol for antimicrobial‑resistance targets and releases all code, data, and structures publicly.

By Jiada Li
arXiv AI
Sep 30

PreviewDiff: Multimodal Critic-Guided Search over Diffusion Latents

PreviewDiff is a test‑time search method that uses multimodal critics to guide diffusion model sampling. By decoding partial previews at selected denoising checkpoints, scoring them with a multimodal judge, and branching over semantic prompt edits, it allows the generation process to be edited and rerouted before completion. The approach consistently outperforms budget‑matched Best‑of‑N sampling and scalar‑search baselines on image and video benchmarks, with early interventions and wider search yielding the biggest gains.

By Vighnesh Subramaniam, Boris Katz, Brian Cheung, Chun-Liang Li, Tomas Pfister, Yale Song