← Back to all news

Fine-tuning and adaptation

LoRA, PEFT, instruction tuning and domain adaptation — adapting a pretrained model without paying to train one.

8,205 stories · RSS feed

arXiv Computer Vision
Aug 25

How Architecture and Training Affect TPC Representations Across Experiments

arXiv:2608.21756v1 Announce Type: cross Abstract: Deep-learning efforts have increasingly shifted toward foundation model approaches. In experimental physics, this allows models and learned represent...

By Tyler Wheeler, Michelle P. Kuchera, Raghuram Ramanujan, William Sieland, Ryan Krupp, Daniel Bazin, Connor L. Cross, Hoi Yan Ian Heung, Andrew J. Jones, Ruchi Mahajan, Saiprasad Ravishankar, Pranjal Singh, Benjamin Votaw, Chris Wrede
ragfine-tuning
More like this →
arXiv AI
Aug 25

Teaching LLMs How ICU Physicians Approach Clinical Reasoning Through OMOP-Aligned Retrieval Improves Reasoning Across Clinical Domains

arXiv:2608.22622v1 Announce Type: cross Abstract: Clinical decision-making relies on identifying relevant patient information to guide diagnosis and treatment, a challenge that is especially difficul...

By Miguel Contreras, Scott Siegel, Subhash Nerella, Jessica Sena, Jiaqing Zhang, Heng Sun, Hruday Tej Akkaladevi, Peiyu Lu, Jordan Rosen, Sumit Kapoor, Sasank Desaraju, Grace R. Thompson, Jacob Purcell, Michael Petrauskis, Philip KW. Hong, Meghan Brennan, Sarah Chrabaszcz, Tierra Smith, Ronnie Ren, Michel S. Kabbash, Ceyhun Haziroglu, Rushi Patel, Gabriel Gomez, Charlotte Chaiklin, Randy Leung, Kenneth N. John, Whitman Wiggins, Philip Kayser, Vincent Bird, Maria Bruzzone, Tyler J. Loftus, Azra Bihorac, Parisa Rashidi
llmsfine-tuningbenchmarks
More like this →
arXiv AI
Aug 25

Length-Adaptive Decoding for Masked Diffusion Machine Translation

The paper introduces Entropy-Valley (EV), a training‑free method for selecting target length in masked diffusion machine translation. EV evaluates candidate canvases by mean predictive entropy from all‑mask forward passes, choosing the length the model is best prepared to fill. Compared to a baseline that uses training‑corpus length statistics, EV recovers a substantial portion of the COMET‑22 gain across En→Zh, Zh→En, and En→De, and expert evaluation confirms adequacy improvements, especially for Zh→En.

By Yan Zhan, Mengkai Hou, Wanting Zhang, Zhijun Gao
llmsdiffusionnlpfine-tuning
More like this →
arXiv Computation and Language
Aug 25

LakeHopper: Knowledge-Aware Adaptation of Column Type Annotators across Data Lakes

LakeHopper is a method for adapting column type annotators (CTA) from one data lake to another by treating cross‑lake adaptation as a knowledge‑management problem. It decomposes the source annotator’s knowledge into source‑specific, shared, and target‑specific parts, and then uses three mechanisms—label‑set realignment, LLM‑verified gap discovery, and cluster‑based propagation with rehearsal fine‑tuning—to adapt the annotator under a limited annotation budget. The approach achieves up to a 71.4% relative macro‑F1 improvement over three PLM backbones, reaches near‑full data quality with less than 6% of target labels, and trains 27–131 times faster than fine‑tuned table LLMs.

By Yushi Sun, Xujia Li, Nan Tang, Quanqing Xu, Chuanhui Yang, Lei Chen
llmsfine-tuningbenchmarksdata-engineering
More like this →
arXiv AI
Aug 25

From Isolation to Alignment: Unified LoRA for Efficient Multi-Task Learning

The paper introduces Align‑LoRA, a unified LoRA framework for multi‑task learning that replaces complex, isolated adapter designs with a single‑adapter model enhanced by a higher rank and an explicit alignment loss. It demonstrates that a router‑free, multi‑head model with high inter‑head redundancy can outperform more elaborate baselines, and that a unified LoRA can achieve competitive performance while enabling weight merging and zero inference latency. Extensive experiments and theoretical analysis confirm that Align‑LoRA surpasses prevailing approaches, offering a simpler, production‑friendly paradigm for parameter‑efficient fine‑tuning of large language models.

By Jinda Liu, Yi Chang, Yuan Wu
llmsdiffusionfine-tuningefficiencysafety
More like this →
arXiv AI
Aug 25

ProBel: Propaganda Detection with Techniques, Spans, and Explanations

ProBel is a bilingual Arabic and English resource for propaganda detection that aligns binary labels, multi-label annotations for 23 propaganda techniques grouped into six categories, technique-labeled spans, and reference explanations for news sentences. The dataset supports matched binary, coarse-grained, multi-label, and span-level tasks in both languages, and the authors evaluate zero‑shot prompting, task‑specific fine‑tuning, and joint training. A single bilingual multi‑task model achieves the best overall performance, with cross‑task analysis revealing that joint classification preserves binary performance while span‑only training can weaken sentence‑level prediction, and that joint bilingual training yields the most stable results.

By Mohamed Bayan Kmainasi, Ali Ezzat Shahroor, Elisa Sartori, Giovanni Da San Martino, Firoj Alam
llmsfine-tuning
More like this →
arXiv Computation and Language
Aug 25

A Multi-Domain and Multi-Task Generative Framework with Explicit Task and Domain Conditioning for Cross-Domain Event Extraction

The paper introduces a unified multi-domain and multi-task generative framework for event extraction that incorporates explicit domain conditioning signals and task-specific prompts. This design allows a single model to adapt dynamically to different event schemas without needing full event label sets during inference, supporting both pipeline and end-to-end extraction. Experiments on various benchmarks show competitive performance, strong cross-domain generalization, and practical scalability while maintaining domain-specific precision.

By Siting Liang, Omar Adjali, Daniel Sonntag
llmsfine-tuningbenchmarks
More like this →