Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

16,906 stories · RSS feed

arXiv AI
Jul 10

Multimodal Unlearning Across Vision, Language, Video, and Audio: Survey of Methods, Datasets, and Benchmarks

arXiv:2607. 07907v1 Announce Type: cross Abstract: With the growing adoption of VLMs, DMs, LLMs, and AFMs, these multimodal foundation models can inadvertently encode sensitive, copyrighted, biased, or unsafe cross-modal associations that originate from their training data.

By Nobin Sarwar, Shubhashis Roy Dipta, Zheyuan Liu, Vaidehi Patil
arXiv Machine Learning
Jul 10

Frequency-Domain Multi-Modality Transportation Modeling

arXiv:2607. 08475v1 Announce Type: new Abstract: Multi-modality transportation refers to urban systems composed of multiple transportation modes, such as traffic flow and public transit, whose dynamics are coupled by shared temporal patterns.

By Jiewen Deng, Hangchen Liu, Junchen Li, Boyuan Zhang, Renhe Jiang
arXiv Machine Learning
Jul 10

Joint Discrete-Continuous Flow Matching for Open-Vocabulary Inverse Design of Multilayer Optical Coatings

arXiv:2607. 08392v1 Announce Type: cross Abstract: Amortized neural inverse design typically remains closed-world: component choices are fixed vocabulary tokens, coordinate grids are frozen at training time, and continuous variables are discretized into sequence tokens.

By Zhiyi Li, Yuheng Jin, Yidan Huang, Nan Chen, Hongyan Fu, Yikun Bu
arXiv AI
Jul 10

Persona Cartography: Charting Language Model Personality Traits in Weight Space

arXiv:2607. 07916v1 Announce Type: new Abstract: Large language models exhibit recurring behavioural patterns -- personas -- that shape generalisation and safety, but we lack reliable tools for decomposing, measuring, and controlling them.

By Luke Baines, Anton Gonzalvez Hawthorne, Mariia Koroliuk, Irakli Shalibashvili, Cl\'ement Dumas, Konstantinos Voudouris, David Demitri Africa
arXiv AI
Jul 10

Infinity-Parser2 Technical Report

arXiv:2607. 07836v1 Announce Type: new Abstract: We present Infinity-Parser2, a large multimodal model that couples a controllable data-synthesis pipeline with multi-task reinforcement learning for end-to-end document parsing, addressing the persistent scarcity of faithfully annotated parsing corpora.

By Zuming Huang, Jun Huang, Kexuan Ren, Baode Wang, Weizhen Li, Jianming Feng, Yu Wang, Yichen Yao, Shijun Lin, Yige Tang, Cheng Peng, Weidi Xu, Wei Chu, Yinghui Xu, Yuan Qi