arXiv Machine Learning June 2, 2026 By Minsik Choi, Geewook Kim

Decentralized Instruction Tuning: Conflict-Aware Splitting and Weight Merging

Read the original on arXiv Machine Learning →

arXiv:2606. 01717v1 Announce Type: new Abstract: Instruction tuning aligns large language models, including multimodal ones, with diverse user intents, but scaling to heterogeneous mixtures is hindered by gradient interference and bandwidth-heavy synchronization.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

1d ago

UniFed-VLM: Federated Instruction Tuning for Vision-Language Models with Multiple Heterogeneity

arXiv:2608. 15516v1 Announce Type: new Abstract: Vision-Language Models (VLMs) have demonstrated strong performance in multimodal understanding and generation.

By Pengyu Wang, Baochen Xiong, Xiaoshan Yang, Yifan Xu, Zhang Qimeng, Haifeng Chen, Changsheng Xu

llms fine-tuning efficiency multimodal benchmarks safety

arXiv AI

Jul 22

Federated Lightweight Fine-Tuning

arXiv:2607. 18343v1 Announce Type: cross Abstract: Federated fine-tuning is bottlenecked by communication: FedAvg and pseudo-gradient schemes transmit a payload that scales with the model, and gradient compression shrinks it by only a constant factor.

By Radhakrishna Achanta, Will Reed

fine-tuning

arXiv Machine Learning

Jun 19

Predicting Mergeability of Parameter-Efficient Fine-Tuning Updates

arXiv:2606. 19549v1 Announce Type: new Abstract: Low-rank adaptation (LoRA) makes it cheap to train many domain- and task-specific language model adapters, but whether two adapters can be merged is usually discovered only after both have been fully trained and evaluated.

By Lin Tang, Wei Zhang, Jing Li, Hongyu Chen, Ming Zhao, Yuxuan Wang

llms fine-tuning benchmarks safety

arXiv Machine Learning

Jul 10

Less Data, Faster Convergence: Goal-Driven Data Optimization for Multimodal Instruction Tuning

arXiv:2603. 12478v2 Announce Type: replace-cross Abstract: Multimodal instruction tuning is often compute-inefficient because training budgets are spread across large mixed image-video pools whose utility is highly uneven.

By Rujie Wu, Haozhe Zhao, Hai Ci, Yizhou Wang

fine-tuning multimodal benchmarks

arXiv AI

Jul 28

DomainPilot: Domain-Level Loss-Guided Two-Stage Data Mixture Optimization for Efficient Language Model Fine-Tuning

arXiv:2607. 22769v1 Announce Type: cross Abstract: The training efficacy of large language models (LLMs) is fundamentally constrained by the quality and composition of training data.

By He Zhang

llms fine-tuning benchmarks data-engineering

arXiv Machine Learning

Aug 4

DataClaw0: Agentic Tailoring Multimodal Data from Raw Streams

arXiv:2606. 21337v2 Announce Type: replace Abstract: Raw multimodal streams are abundant but noisy, redundant, and unaligned with any particular training objective.

By Cong Wan, Zeyu Guo, Zijian Cai, Jiangyang Li, SongLin Dong, Lin Peng, Xiangyang Luo, Zhiheng Ma, Yihong Gong

llms agents diffusion multimodal

Related stories

UniFed-VLM: Federated Instruction Tuning for Vision-Language Models with Multiple Heterogeneity

Federated Lightweight Fine-Tuning

Predicting Mergeability of Parameter-Efficient Fine-Tuning Updates

Less Data, Faster Convergence: Goal-Driven Data Optimization for Multimodal Instruction Tuning

DomainPilot: Domain-Level Loss-Guided Two-Stage Data Mixture Optimization for Efficient Language Model Fine-Tuning

DataClaw0: Agentic Tailoring Multimodal Data from Raw Streams