arXiv AI
Dynamic Model Merging Made Slim
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Machine Learning
Sep 22
Merge++: Universal Merge Refinement Through Data-Free Checkpoint Inversion
Merge++ is a post‑hoc refinement technique for model merging that synthesizes task‑representative images by inverting expert checkpoints and then distills expert knowledge into a merged model. It operates without any additional data beyond the checkpoints and can be applied universally across existing weight‑space merging algorithms. Experiments show consistent improvements, with average gains of +2 to +8 points and up to +25.9 on specific configurations.
arXiv AI
Aug 11
Signature-Guided Capacity Occupancy for Dense Expert Merging
arXiv:2608. 09201v1 Announce Type: new Abstract: Dense expert merging combines domain-specialized language models into one single checkpoint, typically by admitting task-vector support in weight space.
arXiv AI
Oct 9
FeatCal: Feature Calibration for Post-Merging Models
FeatCal is a post‑merging calibration method that reduces feature drift between a merged model and its task experts by adjusting weights layer by layer in forward order using a small calibration set. It employs a closed‑form update that avoids gradient descent or iterative optimization. On CLIP and GLUE benchmarks, FeatCal outperforms existing calibration baselines, achieving higher accuracy with fewer calibration examples and lower computational cost.
arXiv Machine Learning
Sep 22
Not All Task Vectors Need Equal Rank: Energy-Proportional Allocation for Model Merging
arXiv:2609.24517v1 Announce Type: cross Abstract: Model merging aims to combine multiple fine-tuned models derived from a common pretrained model into a single multi-task model without additional joi...
arXiv AI
Jul 21
Mediator: Memory-efficient LLM Merging with Less Parameter Conflicts and Uncertainty Based Routing
arXiv:2502. 04411v3 Announce Type: replace-cross Abstract: Model merging aggregates Large Language Models (LLMs) finetuned on different tasks into a stronger one.
arXiv Machine Learning
Jul 24
CT-Merging: Consensus Directions and Task-Level Scaling for LoRA Adapter Merging
arXiv:2607. 20561v1 Announce Type: new Abstract: LoRA adapters provide an efficient way to specialize a pretrained model for many downstream tasks, but deploying one adapter per task requires adapter storage and task selection at inference time.