Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

28,127 stories · RSS feed

arXiv Computation and Language
Oct 21

FLINT: Fixed-budget Late Interaction

arXiv:2610.27273v1 Announce Type: cross Abstract: Multi-vector retrieval models such as ColBERT often outperform single-vector models of comparable scale by matching queries and documents at the toke...

By Himank Handa, Negar Arabzadeh, Matei Zaharia
arXiv Computer Vision
Oct 21

SuperClip Pyramid for Video Temporal Grounding

arXiv:2610.26925v1 Announce Type: new Abstract: With the rapid expansion of online video content, video temporal grounding has become essential for localizing moments based on user-specific queries,...

By Sanghyeok Lee, Juyeon Ko, Joonmyung Choi, Jongha Kim, Dohwan Ko, Ji Soo Lee, Hyunwoo J. Kim
arXiv AI
Oct 21

AutoRedClaw: Automated Red-Teaming for Claw-Style Agents

AutoRedClaw is an automated red‑teaming framework designed for claw‑style agents such as OpenClaw, which maintain persistent configuration, memory, and reusable skills. It generates end‑to‑end attacks and the necessary environments, using a strict quality‑checking protocol and execution‑based validation to ensure realistic, effective samples. The framework produces a 330‑case benchmark, RedClaw‑Bench, achieving over 30% effective attack rate at less than $0.10 per successful sample and demonstrating higher attack success rates than existing benchmarks across multiple agent systems.

By Hao Li, Tongyan Hu, Weidi Luo, Shanghao Shi, Xiaogeng Liu, Aiden Suh, Yulong Cao, Ning Zhang, William Yeoh, Yevgeniy Vorobeychik, Chaowei Xiao