Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

28,322 stories · RSS feed

arXiv Machine Learning
Sep 16

Test-Time Unlearning via Sparse Autoencoder

arXiv:2609.16229v1 Announce Type: new Abstract: Machine unlearning aims to remove specific knowledge from a trained large language model (LLM) without retraining from scratch. Existing methods modify...

By Pingzhi Li, Jinhao Duan, Vaishnav Tadiparthi, Nakul Agarwal, Kwonjoon Lee, Ehsan Moradi Pari, Hossein Nourkhiz Mahjoub, Sijia Liu, Tianlong Chen
arXiv AI
Sep 16

Extracting ontology-compliant knowledge from scientific text describing irradiated materials using large language models

arXiv:2609.17291v1 Announce Type: new Abstract: The quest for new materials increasingly relies on predictive models and comprehensive simulations that span scales from atomic to macroscopic levels....

By Marco Luca Sbodio, Marcos Mart\'inez Galindo, Vanessa Lopez, Blanca Biel, Pablo Canca, Pedro Delgado, Jes\'us I. Mendieta-Moreno, Raphael Tack, Maria J. Caturla
arXiv AI
Sep 16

World Models for Embodied Intelligence: From Plausible to Controllable to Actionable

arXiv:2609.16697v1 Announce Type: cross Abstract: World models connect perception and decision-making in embodied intelligence by maintaining hidden state, anticipating consequences, comparing interv...

By Nanjie Yao, Hao Wang, Chong Cheng, Zhikang Chen, Wenzhe Li, Jiafei Lyu, Li Shen, Peilin Zhao, Zongqing Lu, Gao Huang, Steven Hoi, Dacheng Tao, Deheng Ye
arXiv AI
Sep 16

Same Answer, Different Representations: Hidden instability in VLMs

arXiv:2602.06652v2 Announce Type: replace Abstract: The robustness of Vision Language Models (VLMs) is commonly assessed through output-level invariance, implicitly assuming that stable predictions r...

By Farooq Ahmad Wani, Alessandro Suglia, Rohit Saxena, Aryo Pradipta Gema, Wai-Chung Kwan, Fazl Barez, Maria Sofia Bucarelli, Fabrizio Silvestri, Pasquale Minervini
arXiv Machine Learning
Sep 16

Sequence Recognition in Bharatnatyam dance

The paper presents a method for recognizing sequences of key postures and motions in Bharatanatyam Adavus using a CNN for key postures (99% accuracy) and an SVM for motions (84% accuracy). It then matches these sequences to ground truth with an Edit Distance algorithm, achieving 98% accuracy. The study claims novelty by considering both postures and motions, measuring prediction time, and testing scalability across all 15 Adavus and 58 variations.

By Himadri Bhuyan, Rohit Dhaipule, Partha Pratim Das
arXiv AI
Sep 16

Robustness as an Emergent Property of Task Performance

The paper argues that robustness—consistency across semantically equivalent inputs—naturally emerges once models achieve high task performance. Empirical analysis across various models and datasets shows a strong positive correlation between task difficulty and robustness, indicating that robustness is driven mainly by task-specific competence rather than inherent model properties. Consequently, as tasks mature and model performance saturates, robustness is expected to improve automatically.

By Shir Ashury-Tahan, Ariel Gera, Elron Bandel, Michal Shmueli-Scheuer, Leshem Choshen
arXiv Computer Vision
Sep 16

ViD: Vision-Dominant Gender Bias Mitigation for Large Vision-Language Models

The paper introduces ViD, a vision‑dominant gender bias mitigation framework for large vision‑language models. ViD uses causal analysis of attention patterns and dual mechanisms—backdoor adjustment and refined token selection—to suppress bias while preserving reasoning and generation quality. Experiments show a 14.7% reduction in gender bias on FACET and significant improvements on MS COCO image captioning, all without extra training overhead.

By Zhipeng Zhao, Zhaoqiang Wei, Peishun Liu, Youwei Zhao, Ruichun Tang