Fine-tuning and adaptation

LoRA, PEFT, instruction tuning and domain adaptation — adapting a pretrained model without paying to train one.

8,205 stories · RSS feed

arXiv Computer Vision
Sep 15

Query-Conditioned Spherical Centroid Aggregation for Multimodal Retrieval

The paper introduces SCALAR, a query‑conditioned spherical centroid aggregator that assigns relevance‑based weights to each available modality before computing a spherical centroid. SCALAR supports arbitrary modality subsets, is trained with rank‑8 LoRA adapters on masked views, and achieves positive aggregation gains on four of five benchmarks, outperforming prior symmetric aggregators. With only 4.8 million trainable parameters, SCALAR attains the highest text‑to‑video R@1 on three benchmarks and surpasses the released GRAM checkpoint under test‑time modality dropout by 3.2 to 10.9 R@1.

By Ambuj Mehrish, Anindya Nag, Sebastiano Vascon
arXiv Computer Vision
Sep 15

Hyper-LLaVA: Hyperbolic Uncertainty-aware Modality-Balanced Routing for Multimodal Continual Instruction Tuning

Hyper-LLaVA introduces a hyperbolic uncertainty‑aware, modality‑balanced routing mechanism for multimodal continual instruction tuning. It improves intra‑modality task matching by evaluating sample‑to‑task distribution similarity in hyperbolic space and mitigates unreliable modality influence by quantifying task‑matching ambiguity to adaptively balance across modalities. The approach achieves significant performance gains over existing methods.

By Kunlun Xu, Yanqin Zhang, Wenwen Qiang, Jiahuan Zhou
arXiv AI
Sep 15

Donors and Recipients: On Asymmetric Transfer Across Tasks and Languages with Parameter-Efficient Fine-Tuning

The study investigates how fine‑tuning a large language model on a single task‑language pair influences performance on other task‑language pairs. Using LoRA fine‑tuning across multiple open‑weight LLM families, 11 languages, and four benchmarks, the authors decompose transfer into matched‑task, cross‑task, and cross‑task cross‑language regimes. They find that while single‑source fine‑tuning generally improves performance, the gains are highly asymmetric, with matched‑task cross‑language transfer being most effective and driven mainly by the target language rather than model architecture.

By Kajetan Dymkiewicz, Ivan Vulic, Helen Yannakoudakis, Eilam Shapira, Roi Reichart, Anna Korhonen
arXiv AI
Sep 15

MedSAM3: Delving into Segment Anything with Medical Concepts

MedSAM-3 is a text‑promptable medical segmentation model that builds on the Segment Anything Model (SAM) by fine‑tuning it with medical images and semantic concept labels. It enables precise anatomical segmentation through open‑vocabulary text descriptions, moving beyond purely geometric prompts. The accompanying MedSAM-3 Agent incorporates multimodal large language models to perform complex reasoning and iterative refinement, and experiments across X‑ray, MRI, ultrasound, CT, and video modalities show it outperforms existing specialist and foundation models.

By Anglin Liu, Xu R. Cao, Yifan Shen, Yi Lu, Xiang Li, Qianqian Chen, Jintai Chen