arXiv AI June 10, 2026 By Ruinan Wang, Ian Nabney, Mohammad Golbabaee

Importance-Aware Scheduling for High-Dimensional Hyperparameter Optimization

arXiv:2606. 10068v1 Announce Type: cross Abstract: Hyperparameter Optimization (HPO) is essential for building high-performing ML/DL models, yet conventional optimizers often struggle in high-dimensional spaces where evaluations are costly and progress is diluted across many low-impact variables.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv AI.

benchmarks

arXiv Machine Learning

Jun 24

The Sparse Frontier: Sparse Attention Trade-offs in Transformer LLMs

arXiv:2504. 17768v3 Announce Type: replace-cross Abstract: Sparse attention offers a promising strategy to extend long-context capabilities in Transformer LLMs, yet its efficiency-accuracy trade-offs remain unclear due to the lack of comprehensive evaluation.

By Piotr Nawrot, Robert Li, Renjie Huang, Sebastian Ruder, Kelly Marchisio, Edoardo M. Ponti

llms efficiency

Hugging Face Trending Papers

Jun 1

CRePE: Convolution-aware Relative Importance in Post-training Pruning with Efficient Search

Deploying Large Language Models (LLMs) in practice incurs substantial memory and computational costs. Post-training pruning (PTP) is an effective approach to reducing these costs by removing weights without additional training.

llms efficiency benchmarks

arXiv Machine Learning

Jun 2

CRePE: Convolution-aware Relative Importance in Post-training Pruning with Efficient Search

arXiv:2606. 01544v1 Announce Type: new Abstract: Deploying Large Language Models (LLMs) in practice incurs substantial memory and computational costs.

By Cheonjun Park

llms efficiency benchmarks

arXiv AI

Aug 10

Fast LapSum: Exact Differentiable Top-k at Million Scale

arXiv:2608. 06912v1 Announce Type: new Abstract: The top-$k$ operation is a fundamental building block of modern sparse computation, enabling token routing, expert activation, memory selection, and attention pruning.

By {\L}ukasz Struski, Joanna Wojciechowicz, Jakub Antczak, Marcin Mazur, Kamil Ksi\k{a}\.zek, Jacek Tabor

efficiency benchmarks safety

arXiv AI

Jun 29

When Is an LLM Worth It for Hyperparameter Optimization? A Budget-Matched Study on Tabular Data Finds the Warm-Start Is a Default Configuration, Not the Model

arXiv:2606. 21641v2 Announce Type: replace-cross Abstract: Large language models (LLMs) have been proposed as hyperparameter-optimization (HPO) advisors that "warm-start" search from prior knowledge, proposing strong configurations in very few evaluations.

By Carson Rodrigues, Oysturn Vas, Isaiah Abner DCosta, Nithish Kumar Prabhakaran

llms benchmarks

arXiv AI

Jul 14

Coresets Before Score Sets: Evaluation-Unsupervised Prompt Subset Selection for LLM Benchmarks

arXiv:2607. 09739v1 Announce Type: new Abstract: We study LLM benchmark coreset selection: selecting a small subset of prompts over multiple benchmarks whose induced model scores and rankings approximate those obtained from the full benchmark suite.

By Jihan Yao, Gantavya Bhatt, Arnav Das, Peter Jin, Ke Bao, Qiaolin Yu, Khushi Bhardwaj, Chang Su, Jialei Wang, Yikai Zhu, Sugam Devare, Damon Mosk-Aoyama, Zhen Dong, Venkat Krishna Srinivasan, Yineng Zhang, Oleksii Kuchaiev, Jiantao Jiao, Banghua Zhu, Jeff Bilmes

llms rag benchmarks

Related stories

The Sparse Frontier: Sparse Attention Trade-offs in Transformer LLMs

CRePE: Convolution-aware Relative Importance in Post-training Pruning with Efficient Search

CRePE: Convolution-aware Relative Importance in Post-training Pruning with Efficient Search

Fast LapSum: Exact Differentiable Top-k at Million Scale

When Is an LLM Worth It for Hyperparameter Optimization? A Budget-Matched Study on Tabular Data Finds the Warm-Start Is a Default Configuration, Not the Model

Coresets Before Score Sets: Evaluation-Unsupervised Prompt Subset Selection for LLM Benchmarks