arXiv AI

Rethinking Heterogeneous LLM Merging: A Weighted Model Averaging Perspective

arXiv:2607. 18026v1 Announce Type: new Abstract: Can large language models with substantially different parameter spaces be merged by direct weighted averaging, without training or semantic alignment?

arXiv AI
Jun 17

Know Thy Reasoner: Not All Language Models Explore Alike

arXiv:2604. 10827v2 Announce Type: replace Abstract: Compute scaling for LLM reasoning trades off exploring solution approaches (\emph{breadth}) against refining promising ones (\emph{depth}), yet why a given trade-off works, and why it often fails to transfer across models, remains unclear.

By Moulik Choraria, Argyrios Gerogiannis, Anirban Das, Supriyo Chakraborty, Sourya Basu, Sambit Sahu, Lav R. Varshney
arXiv AI
1d ago

SAPE: Sandwich Adapters for Parameter Efficiency in Large Language Model Fine-Tuning

arXiv:2608. 15360v1 Announce Type: cross Abstract: While Parameter-Efficient Fine-Tuning (PEFT) has substantially reduced the hardware cost of adapting Large Language Models (LLMs) by decreasing the number of trainable parameters, recent studies have sought to further improve PEFT through parameter sharing.

By Mohammad Aref Jafari-Raddani, Morteza Mohajjel Kafshdooz