arXiv AI

Scaling Decision-Focused Learning to Large Problems with Lagrangian Decomposition

arXiv:2606. 08797v1 Announce Type: cross Abstract: Decision-focused learning has shown great promise for addressing predict-then-optimize problems, particularly in the presence of under-specified models.

arXiv Machine Learning

Jun 19

A Solver-Free Training Method for Predict-then-Optimize

arXiv:2606. 19587v1 Announce Type: cross Abstract: We propose a scalable method for training prediction (machine learning) models in the predict-then-optimize paradigm, where model outputs serve as coefficients for a subsequent linear optimization task.

By Beichen Wan, Mo Liu

benchmarks

arXiv Machine Learning

1d ago

Decision-Driven Regularization: A Blended Model for Learning and Optimization

arXiv:2608. 15124v1 Announce Type: new Abstract: In contextual optimization, the decision-maker seeks optimal decisions to minimize a cost function, that varies based on observed features.

By Gar Goei Loke, Qinshen Tang, Yangge Xiao, Xun Zhang

benchmarks

arXiv AI

Jun 2

Score Function Gradient Estimation to Widen the Applicability of Decision-Focused Learning

arXiv:2307. 05213v3 Announce Type: replace-cross Abstract: Many real-world optimization problems contain parameters that are unknown before deployment time, either due to stochasticity or to lack of information (e.

By Mattia Silvestri, Senne Berden, Jayanta Mandi, Ali \.Irfan Mahmuto\u{g}ullar{\i}, Brandon Amos, Tias Guns, Michele Lombardi

benchmarks

arXiv Machine Learning

Jun 18

BLADE: Scalable Bi-level Adaptive Data Selection for LLM Training

arXiv:2606. 18650v1 Announce Type: new Abstract: As Large Language Model (LLM) datasets scale to trillions of tokens, data selection has emerged as a critical frontier to filter out uninformative noise and construct adaptive learning trajectories.

By Jiaxing Wang, Deping Xiang, Jin Xu, Zirui Liu, Zicheng Zhang, Guoqiang Gong, Jun Fang, Chao Liu, Pengzhang Liu, Tongxuan Liu, Ke Zhang, Qixia Jiang

llms benchmarks

arXiv AI

Jul 7

OmniOpt: Taxonomy, Geometry, and Benchmarking of Modern Optimizers

arXiv:2607. 04033v1 Announce Type: cross Abstract: Optimizer selection for large-scale model training has become a system-level design decision constrained jointly by compute, memory, tuning budget, and task diversity, yet the landscape of over one hundred methods remains fragmented.

By Siyuan Li, Jiabao Pan, Yumou Liu, Zhuoli Ouyang, Xin Jin, Xinglong Xu, Jingxuan Wei, Shengye Pang, Jintao Che, Xuanhe Zhou, Conghui He, Cheng Tan

llms computer-vision benchmarks

arXiv Machine Learning

Jun 2

Decision-Focused On-Policy Learning for Contextual Linear Optimization with Partial Feedback

arXiv:2606. 01081v1 Announce Type: new Abstract: Decision-focused learning (DFL) trains predictive models by optimizing downstream decision quality rather than standalone prediction accuracy.

By Wyame Benslimane, Tinghan Ye, Pascal Van Hentenryck, Paul Grigas

reinforcement-learning benchmarks

arXiv Machine Learning

Jun 16

Diversity-Driven Offline Multi-Objective Optimization via Nested Pareto Set Learning

arXiv:2606. 15115v1 Announce Type: new Abstract: Multi-objective optimization (MOO) has emerged as a powerful approach to solving complex optimization problems involving multiple objectives.

By Yiyi Zhu, Yaolin Wen, Xiang Xia, Xin An, Hanyi Si, Xiang Shu, Yangde Fu, Liang Dou, Hong Qian

benchmarks safety

arXiv Machine Learning

Jun 19

Spectral DPPs via NEPv: A Scalable Continuous Relaxation of Determinantal MAP for Diversity-Aware Data Selection

arXiv:2606. 19411v1 Announce Type: new Abstract: Selecting a small, diverse, high-quality subset from a massive pool of candidates is a recurring primitive in modern machine learning -- data curation and coreset selection for training and fine-tuning large models, active-learning batch acquisition, prompt and exemplar selection for in-context learning, retrieval diversification, and experimental design.

By Richard Yi Da Xu

llms rag fine-tuning benchmarks

arXiv AI

Jul 24

SOAP, Muon, and Beyond: Pushing LLM Pretraining Scales

arXiv:2607. 20548v1 Announce Type: cross Abstract: Higher-order optimizers such as Muon and SOAP offer faster convergence than AdamW, but their computational cost and numerical stability challenges have limited adoption at scale.

By Mikail Khona, Aditya Vavre, Boxiang Wang, Deyu Fu, Hao Wu, Mike Chrzanowski, Bryan Catanzaro, Dheevatsa Mudigere, Jeff Pool, Michael Lightstone, Mohammad Shoeybi, Mostofa Patwary, Nima Tajbakhsh, Tijmen Blankevoort

llms

arXiv Machine Learning

Jul 16

Parallel gradient boosting for flexible estimation of conditional distributions

arXiv:2607. 13550v1 Announce Type: cross Abstract: Boosting is one of the most successful learning techniques for standard classification and regression tasks.

By R\'emy Chapelle (CESP, CB, EVDG), Nicolas Vayatis (CB), Bruno Falissard (CESP), Mohammed Sedki (CESP)

benchmarks

arXiv Machine Learning

Jul 7

Contaminated Multi-task Learning with Heterogeneity: Fundamental Limits and Optimal Algorithms

arXiv:2607. 02681v1 Announce Type: cross Abstract: Integrating information across related tasks can improve estimation and prediction in transfer, multi-task, and federated learning, but contamination and heterogeneity make robust borrowing challenging.

By Ye Tian, Mengchu Li, Marco Avella Medina

benchmarks

arXiv Machine Learning

Jul 28

WeCon: An Efficient Weight-Conditioned Neural Solver for Multi-Objective Combinatorial Optimization Problems

arXiv:2605. 22876v2 Announce Type: replace Abstract: Existing neural solvers for Multi-Objective Combinatorial Optimization Problems (MOCOPs) commonly adopt decomposition-based strategies that scalarize a MOCOP into multiple subproblems associated with distinct weight vectors.

By Xuan Wu, Jinbiao Chen, Yang Li, Lijie Wen, Chunguo Wu, Yuanshu Li, Yubin Xiao, Chunyan Miao, You Zhou, Di Wang

benchmarks

Related stories