arXiv AI
Jun 18
Surrogate Benchmarks for Model Merging Optimization
arXiv:2509. 02555v2 Announce Type: replace-cross Abstract: Model merging techniques aim to integrate the abilities of multiple models into a single model.
Multi-Objective Bayesian Optimization for Model Merging
arXiv:2608. 14264v1 Announce Type: cross Abstract: Model merging combines trained models directly in weight space, offering a compute-efficient alternative to additional fine-tuning.
Related stories
arXiv AI
Jul 13
LLM-Driven Evolutionary Generation of Multi-Objective Bayesian Optimization Algorithms
arXiv:2607. 08791v1 Announce Type: cross Abstract: Designing effective multi-objective Bayesian optimization (MOBO) algorithms requires balancing many interdependent design choices whose optimal configuration is problem-dependent and typically demands deep expertise.
arXiv Machine Learning
Jun 10
mlr3mbo: Bayesian Optimization in R
arXiv:2603. 29730v2 Announce Type: replace-cross Abstract: We present mlr3mbo, a modular toolbox for Bayesian optimization in R.
arXiv Machine Learning
Aug 11
A Hybrid Nested Harness for Decoupling Structure and Parameters in LLM-Driven Optimization
arXiv:2608. 08156v1 Announce Type: new Abstract: In evolutionary algorithms powered by language models, the LLM acts as a single operator that simultaneously updates structural components (like control flow) and continuous parameters.
arXiv AI
6d ago
High-dimensional Multi-objective Bayesian Optimization with Learned Variable Interactions
arXiv:2608. 11713v1 Announce Type: cross Abstract: Multi-objective Bayesian optimization (MOBO) is effective in identifying the Pareto fronts for expensive black-box problems.
arXiv AI
Jun 30
Model Merging to Evolution: Parameter Space Exploration for Expert Models
arXiv:2606. 28373v1 Announce Type: cross Abstract: Model merging integrates the capabilities of multiple expert models to create strong models for multiple tasks without additional training, thereby reducing computational resource requirements.
arXiv AI
Jun 2
FrontierOR: Benchmarking LLMs' Capacity for Efficient Algorithm Design in Large-Scale Optimization
arXiv:2605. 25246v3 Announce Type: replace Abstract: Large language models (LLMs) are increasingly used for optimization modeling and solver-code generation, yet practical operations research and optimization problems often require a harder capability: designing scalable algorithms that exploit problem structure and outperform direct formulation-and-solve baselines.
arXiv AI
5d ago
CABS+: Efficient and Scalable Model Merging via Conflict-Aware Sparsification and Adaptive Weight Allocation
arXiv:2608. 12842v1 Announce Type: new Abstract: Model merging has recently attracted significant attention as a promising paradigm for constructing unified multi-task models without requiring additional retraining.
arXiv AI
Jun 3
When Model Merging Breaks Routing: Training-Free Calibration for MoE
arXiv:2606. 03391v1 Announce Type: cross Abstract: Model merging has emerged as a cost-effective approach for consolidating the capabilities of multiple LLMs without retraining.
arXiv Machine Learning
Aug 6
GRIMIP: A General Framework for Instance-Specific Configuration of MIP Solvers Using LLMs
arXiv:2606. 23299v2 Announce Type: replace Abstract: Configuring the hyperparameters of Mixed-integer programming (MIP) solvers is a high-dimensional, instance-dependent optimization problem where suboptimal settings can degrade solving time by orders of magnitude.
arXiv AI
Jun 24
Variational Model Merging for Pareto Front Estimation in Multitask Finetuning
arXiv:2412. 08147v2 Announce Type: replace-cross Abstract: Pareto fronts are useful to find good task-mixing strategies for multitask finetuning, but they are also costly to compute.
arXiv Machine Learning
Jun 16
Diversity-Driven Offline Multi-Objective Optimization via Nested Pareto Set Learning
arXiv:2606. 15115v1 Announce Type: new Abstract: Multi-objective optimization (MOO) has emerged as a powerful approach to solving complex optimization problems involving multiple objectives.