Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

12,559 stories · RSS feed

arXiv AI
4d ago

Adding Voice Cloning to Text-to-Audio-Video Models with a Single Zero-Initialised Layer

arXiv:2608. 15690v1 Announce Type: cross Abstract: Text-to-audio-video (T2AV) generation models produce a video and its soundtrack from a textual description, but offer no control over whose voice speaks in the output.

By Ivan Mikheev, Viacheslav Vasilev, Anna Dmitrienko, Alexey Letunovskiy, Ivan Kirillov, Kirill Chernyshev, Denis Dimitrov
arXiv AI
4d ago

AsyTO: Asymmetric Temporal Operator for Parameter-Efficient Multivariate Time Series Forecasting

arXiv:2608. 16098v1 Announce Type: cross Abstract: Multivariate time-series forecasting faces a structural dilemma: sharing one temporal predictor across variables is parameter-efficient but forces heterogeneous variables through an identical history-to-future map, whereas learning an independent predictor per variable restores flexibility at a cost that grows with the product of variable count, context length, and horizon.

By Xiachong Lin, Du Yin, Hao Xue, Wen Hu, Imran Razzak, Arian Prabowo, Matthew Amos, Flora D. Salim
arXiv AI
4d ago

CompoSkill: Compositional Skill Chain Attacks from Individually Scanner-Passing LLM Agent Skills

arXiv:2608. 16246v1 Announce Type: cross Abstract: Autonomous AI agents tackling Long Horizon Tasks depend on marketplace skills that are certified one at a time: a scanner returns a safety verdict for each skill and declares the ecosystem safe if every package passes.

By Mingxiao Liu, Zhoumian Jiang, Jianan Ma, Jian Zhang, Jialuo Chen, Xinhao Deng, Zhen Wang
arXiv AI
4d ago

Specifying AI-SDLC Processes: A Protocol Language for Human-Agent Boundaries

arXiv:2606. 20615v3 Announce Type: replace Abstract: AI agents now act as first-class members of the software development lifecycle, but the instruments teams use to direct them enforce nothing: process encoded in prompts is flexible but unenforceable, while workflow formalisms are enforceable but do not model autonomous agents.

By Ylli Prifti, Pasquale De Meo, Alessandro Provetti
arXiv AI
4d ago

A Bi-directional Multi-solution Scalable Grover Search Algorithm

arXiv:2404. 15616v2 Announce Type: replace-cross Abstract: Grover's search algorithms, including various Partial Grover Searches (PGS), suffer from scaling issues when multiple solutions are sought, as the number of iterations scales with the number of solutions or marked states, making implementation more computationally expensive.

By Debanjan Konar, Zain Hafeez, Vaneet Aggarwal
arXiv AI
4d ago

DiagnosisArena: Benchmarking Diagnostic Reasoning for Large Language Models

arXiv:2505. 14107v5 Announce Type: replace-cross Abstract: The emergence of groundbreaking large language models capable of performing complex reasoning tasks holds significant promise for addressing various scientific challenges, including those arising in complex clinical scenarios.

By Yakun Zhu, Zhongzhen Huang, Linjie Mu, Yutong Huang, Wei Nie, Jiaji Liu, Shaoting Zhang, Pengfei Liu, Xiaofan Zhang