arXiv Machine Learning
Jul 9
Efficient Long-Horizon Learning for Learned Optimization
arXiv:2607. 06772v1 Announce Type: new Abstract: Learned optimization aims to improve upon hand-designed optimizers (e.
OmniOpt: Taxonomy, Geometry, and Benchmarking of Modern Optimizers
arXiv:2607. 04033v1 Announce Type: cross Abstract: Optimizer selection for large-scale model training has become a system-level design decision constrained jointly by compute, memory, tuning budget, and task diversity, yet the landscape of over one hundred methods remains fragmented.
Related stories
arXiv Machine Learning
Jun 16
Zero-order Parameter-free Optimization for LMO-based Methods: Novel Approach for Efficient Fine-tuning
arXiv:2606. 14970v1 Announce Type: new Abstract: Fine-tuning large language models (LLMs) has become a central application of modern optimization, enabling pretrained models to adapt to diverse downstream tasks and domain-specific data.
Hugging Face Trending Papers
Jul 7
Efficient Long-Horizon Learning for Learned Optimization
Learned optimization aims to improve upon hand-designed optimizers (e. g.
arXiv AI
Aug 6
ORACLE: A Multi-Objective Reinforcement Learning-Based Analog Circuit Design Optimizer with Large Language Models-Guided Exploration
arXiv:2608. 04999v1 Announce Type: cross Abstract: Analog circuit design automation using reinforcement learning (RL) has emerged as a promising approach for reducing manual effort.
arXiv Machine Learning
Jun 2
OP-LoRA: The Blessing of Dimensionality
arXiv:2412. 10362v2 Announce Type: replace Abstract: Low-rank adapters (LoRA) enable finetuning of large models with only a small number of parameters.
arXiv AI
Jul 28
DecoupleMix: Decoupled Ratio Search and Convex Allocation for Scalable VLM Data Recipes
arXiv:2607. 24516v1 Announce Type: cross Abstract: While data curation for Vision Language Models (VLMs) is increasingly active, public practice for constructing pretraining mixtures remains largely heuristic: practitioners stack datasets that pass quality filters, set cross-domain ratios by intuition, and lack a principled, attributable criterion for admitting new data, while frontier recipes remain undisclosed.
arXiv AI
Jun 15
Beyond LoRA: Is Sparsity-Induced Adaptation Better?
arXiv:2606. 13767v1 Announce Type: cross Abstract: Low-rank adaptation (LoRA) and its variants provide a memory- and compute-efficient alternative to full fine-tuning of pre-trained models.
arXiv Machine Learning
1d ago
UniFed-VLM: Federated Instruction Tuning for Vision-Language Models with Multiple Heterogeneity
arXiv:2608. 15516v1 Announce Type: new Abstract: Vision-Language Models (VLMs) have demonstrated strong performance in multimodal understanding and generation.
arXiv Machine Learning
Jun 30
HiAP: A Multi-Granular Stochastic Auto-Pruning Framework for Vision Transformers
arXiv:2603. 12222v2 Announce Type: replace-cross Abstract: Vision Transformers require significant computational resources and memory bandwidth, severely limiting their deployment on resource-constraint hardware.
arXiv AI
Jun 9
Scaling Decision-Focused Learning to Large Problems with Lagrangian Decomposition
arXiv:2606. 08797v1 Announce Type: cross Abstract: Decision-focused learning has shown great promise for addressing predict-then-optimize problems, particularly in the presence of under-specified models.
arXiv AI
5d ago
CABS+: Efficient and Scalable Model Merging via Conflict-Aware Sparsification and Adaptive Weight Allocation
arXiv:2608. 12842v1 Announce Type: new Abstract: Model merging has recently attracted significant attention as a promising paradigm for constructing unified multi-task models without requiring additional retraining.
arXiv AI
Jul 24
OPTScientist: Multi-Agent Discovery of Typed Optimizer Programs for Transformer Pretraining
arXiv:2607. 20486v1 Announce Type: new Abstract: Designing optimizers for modern deep learning remains a challenging scientific problem, requiring the joint consideration of optimization geometry, state dynamics, numerical stability, implementation constraints, and empirical generalization.