arXiv Machine Learning
Predicting and Repairing Merge Collapse in Large Language Models
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Machine Learning
Jun 19
Predicting Mergeability of Parameter-Efficient Fine-Tuning Updates
arXiv:2606. 19549v1 Announce Type: new Abstract: Low-rank adaptation (LoRA) makes it cheap to train many domain- and task-specific language model adapters, but whether two adapters can be merged is usually discovered only after both have been fully trained and evaluated.
arXiv AI
Jul 15
Are we Merging the Right Models? Impact of Expert Training Duration on Model Merging for LLMs
arXiv:2607. 11997v1 Announce Type: cross Abstract: Multi-task model merging combines separately trained expert models into a single model that handles all tasks without co-training.
arXiv AI
Jul 21
Rethinking Heterogeneous LLM Merging: A Weighted Model Averaging Perspective
arXiv:2607. 18026v1 Announce Type: new Abstract: Can large language models with substantially different parameter spaces be merged by direct weighted averaging, without training or semantic alignment?
arXiv AI
Jul 20
When Model Merging Rivals Joint Multi-Task Reinforcement Learning: A Task-Vector Geometry Analysis
arXiv:2607. 16062v1 Announce Type: cross Abstract: Model merging is promoted as a substitute for joint multi-task training, yet in the reinforcement-learning setting this substitution is essentially never tested against the baseline it claims to replace: methods merge independently released agents precisely because a joint model is unavailable.
arXiv AI
Jul 31
Mergeable Model-Side Aggregation States for Long-Context Language Models
arXiv:2607. 26448v1 Announce Type: cross Abstract: A known limitation of long-context language models is their increasingly unreliable performance in non-additive, set-based aggregation as context length grows.
arXiv AI
Jun 24
AdversaBench: Automated LLM Red-Teaming with Multi-Judge Confirmation and Cross-Model Transferability
arXiv:2606. 24589v1 Announce Type: new Abstract: Scaling adversarial evaluation of large language models requires both a method for generating hard inputs and a reliable way to confirm that resulting failures are real.