Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

14,048 stories · RSS feed

arXiv Machine Learning
Aug 6

From Non-Convex Self-Concordant Regularization to Scalable Quasi-Newton Training of PINNs

arXiv:2608. 04206v1 Announce Type: new Abstract: Physics-informed neural networks (PINNs) often require high-accuracy quasi-Newton refinement to obtain reliable partial differential equation solutions, but their residual objectives can exhibit indefinite, nearly singular, and poorly scaled local curvature.

By Chenhao Si, Kang An, Shiqian Ma, Ming Yan
arXiv AI
Aug 6

BrainBench: Benchmarking Large Language Models for Comprehensive EEG Understanding

arXiv:2608. 04156v1 Announce Type: new Abstract: Electroencephalography (EEG) analysis extends beyond assigning predefined labels to recordings; it requires workflows connecting natural-language instructions, signal processing, quantitative evidence, and scientific interpretation.

By Yangxuan Zhou, Sha Zhao, Yuning Chen, Chen Wu, Jiquan Wang, Shijian Li, Gang Pan
arXiv Machine Learning
Aug 6

The Price of Isolation: Estimating the Ecosystem Cost of Symmetric Two-Sided A/B Testing

arXiv:2608. 04432v1 Announce Type: cross Abstract: On two-sided content platforms, symmetric two-sided isolation (assigning matched fractions of creators and viewers to isolated treatment and control submarkets) is widely used for creator-side and cold-start experiments because it removes cross-arm marketplace interference.

By Yuanyuan Shen, Yiren Yan, Wenjie Li, Chunhui Zhu
arXiv Machine Learning
Aug 6

Toward Skill-Native LLMs: Skill Entropy for Benchmarking and Training Long-Horizon Reasoning

arXiv:2608. 05139v1 Announce Type: cross Abstract: Long-horizon reasoning in recent LLMs demands that the model switch between distinct skills inside a reasoning chain, such as first doing a math derivation, then using the result to plan a schedule.

By Yinghui He, Ling Yang, Jiarui Liu, Yongjin Yang, Lechen Zhang, Yingcheng Wu, Zhenfei Yin, Mengdi Wang, Sanjeev Arora
arXiv AI
Aug 6

FinRpt: Dataset, Evaluation System and LLM-based Multi-agent Framework for Equity Research Report Generation

arXiv:2511. 07322v3 Announce Type: replace-cross Abstract: While LLMs have shown great success in financial tasks like stock prediction and question answering, their application in fully automating Equity Research Report generation remains uncharted territory.

By Song Jin, Shuqi Li, Shukun Zhang, Rui Yan
arXiv AI
Aug 6

Adversarially Robust Abductive Fusion of Pre-trained Transformer-based Perception Models

arXiv:2608. 04190v1 Announce Type: new Abstract: Deploying pre-trained perception models in novel environments degrades their accuracy under distributional shift, and assembling them alone does not recover it: combiners such as majority voting trade recall for precision and are brittle to coordinated failures.

By Mario Leiva, Yue Ma, Qinru Qiu, Gerardo Simari, Paulo Shakarian