Fine-tuning and adaptation

LoRA, PEFT, instruction tuning and domain adaptation — adapting a pretrained model without paying to train one.

8,208 stories · RSS feed

arXiv AI

Jul 17

Decoupled Alignment for Robust Plug-and-Play Adaptation

arXiv:2406. 01514v4 Announce Type: replace-cross Abstract: We introduce a training-free safety enhancement method for aligning large language models (LLMs) without the need for supervised fine-tuning or reinforcement learning from human feedback.

By Haozheng Luo, Jiahao Yu, Wenxin Zhang, Jialong Li, Chenghao Qiu, Yimin Wang, Eric Hanchen Jiang, Jerry Yao-Chieh Hu, Yan Chen, Binghui Wang, Xinyu Xing, Han Liu

llms reinforcement-learning fine-tuning efficiency safety

arXiv AI

Jul 17

Innocuous-Seeming Data, Latent Ideology: Ideological Generalisation in Finetuned LLMs

arXiv:2607. 14888v1 Announce Type: cross Abstract: Finetuning language models on small, curated datasets is standard practice for adapting them to specific policies or domains.

By Robert Graham, Edward Stevinson, Yariv Barsheshat

llms fine-tuning benchmarks safety

arXiv Machine Learning

Jul 17

A Continuous-Time Reinforcement Learning Framework for Fine-Tuning Discrete Diffusion Models

arXiv:2607. 14522v1 Announce Type: new Abstract: We formulate reinforcement learning (RL) in continuous time with discrete state spaces and possibly arbitrary action spaces via a stochastic control approach, where the state dynamics are modeled as a controlled continuous-time Markov chain (CTMC).

By Zikun Zhang, Jiayuan Sheng, David D. Yao, Wenpin Tang

llms diffusion reinforcement-learning fine-tuning

arXiv AI

Jul 17

Long-Context Fine-Tuning with Limited VRAM

arXiv:2607. 15105v1 Announce Type: new Abstract: Parameter-efficient fine-tuning reduces model and optimizer memory, but dense attention still makes long training sequences expensive.

By Vladimir Fedosov, Aleksandr Sazhin, Artemiy Grinenko, Frank Woernle

fine-tuning efficiency

arXiv Machine Learning

Jul 17

Dysco: Dynamic Subspace Boosting to Mitigate LoRA Interference in Federated Learning

arXiv:2607. 14367v1 Announce Type: new Abstract: Federated fine-tuning of large pre-trained models increasingly relies on Low-Rank Adaptation (LoRA) to reduce communication and computation, but heterogeneous clients can make adapter aggregation unstable.

By Haobo Zhang, Jiankun Wang, Suraj Rajendran, Weishen Pan, Lam Tsoi, Yong Chen, Fei Wang, Jiayu Zhou

llms fine-tuning safety

arXiv AI

Jul 17

Mixtures of SubExperts for Large Language Continual Learning

arXiv:2511. 06237v2 Announce Type: replace-cross Abstract: Enabling lifelong learning in LLMs demands resolving the stability-plasticity dilemma (i.

By Haeyong Kang, Hee Suk Yoon, Dahua Feng, Chang D. Yoo

llms fine-tuning efficiency benchmarks

arXiv AI

Jul 17

Reachability-Aware Pretraining for Efficient Target-Oriented Path Exploration in Temporal Knowledge Graph Reasoning

arXiv:2607. 14886v1 Announce Type: new Abstract: Temporal Knowledge Graph (TKG) reasoning under the extrapolation setting focuses on forecasting future time-stamped events (facts) from historical data in a temporal knowledge graph.

By Chien-Liang Liu, Tsao-Lun Chen

agents reinforcement-learning fine-tuning safety

arXiv AI

Jul 17

CFM-Bench: A Unified Multi-Domain, Multi-Task Benchmark for Channel Foundation Models

arXiv:2607. 14975v1 Announce Type: new Abstract: Channel foundation models (CFMs) are developing rapidly, with recent studies reporting benefits from pretraining across downstream wireless tasks.

By Yuan Gao, Wenjun Yu, Jun Jiang, Yunfan Li, Xinyu Guo, Shugong Xu

fine-tuning multimodal benchmarks

arXiv Machine Learning

Jul 17

Full-data accuracy with fewer labels for training and fine-tuning machine-learning force fields

arXiv:2607. 14486v1 Announce Type: cross Abstract: Machine-learning force fields (MLFFs) are reliable only near their training distribution, making efficient construction of diverse training sets a major bottleneck for both train-from-scratch and foundation fine-tuning workflows.

By Sheng Bi, Yi-Ze Wang, Jun Cheng

fine-tuning

arXiv Machine Learning

Jul 17

XCT-SAM: Sequential Parameter-Efficient Domain Adaptation of SAM for Industrial XCT Defect Segmentation

arXiv:2607. 14287v1 Announce Type: cross Abstract: Defect segmentation in additive manufacturing (AM) X-ray computed tomography (XCT) images remains challenging due to severe class imbalance and large distribution shifts across scan conditions.

By Md Mahedi Hasan, Md Mushfiqur Rahaman, Alan Pachkovskiy, Imtiaz Ahmed, Jeremy Dawson, Srinjoy Das

computer-vision fine-tuning benchmarks safety

arXiv AI

Jul 17

Stop Thinking, Start Looking: Efficient Post-Training for Multimodal Document Question Answering via Reasoning-Free Alignment

arXiv:2607. 14682v1 Announce Type: new Abstract: Efficient multimodal document question answering with explicit visual grounding, locating the precise document region that supports each answer remains an open challenge.

By Harikrishnan P M, Goutham Vignesh, Ganesh Parab, Saisubramaniam Gopalakrishnan, Vishal Vaddina, Varun V, Rohit Agrawal

nlp reinforcement-learning fine-tuning multimodal benchmarks safety

arXiv AI

Jul 17

Interpretable Language Model for Closed-Loop Type 1 Diabetes Control

arXiv:2607. 14126v1 Announce Type: new Abstract: Type 1 Diabetes (T1D) is a chronic, life-threatening autoimmune condition characterized by the complete destruction of insulin-producing pancreatic beta cells.

By Maya Sarkar

llms reinforcement-learning fine-tuning safety

arXiv AI

Jul 17

Digital Pantheon: Simulating and Auditing Coalition Formation with LLM Agents

arXiv:2607. 15095v1 Announce Type: cross Abstract: The formation of political coalitions is a complex negotiation driven by both concrete policy objectives and deep-seated ideological convictions.

By Dylan Van Mulders, Matthias Bogaert, Dirk Van den Poel

llms rag agents reinforcement-learning fine-tuning benchmarks safety

arXiv AI

Jul 17

Step-Level Preference Learning for Generative Agents in Social Simulations

arXiv:2607. 14485v1 Announce Type: new Abstract: Large language model (LLM)-based generative agents simulate human behavior through long-horizon decision-making processes that comprise intermediate steps such as planning, memory retrieval, reflection, and action selection.

By Wenchang Gao, Pingyue Sheng, Lanlan Qiu, Yunfei Ma, Jian Zhao, Baicheng Chen, Kangda Wang, Yuyang Tian, Shunqiang Mao, Tianxing He

llms agents fine-tuning

arXiv AI

Jul 17

An offline approach to fNIRS-guided reinforcement learning for robot behavior

arXiv:2607. 14393v1 Announce Type: cross Abstract: Human-in-the-loop Reinforcement Learning has become a popular approach to training, finetuning, and aligning robot behavior with user preferences.

By Julia Santaniello, Madelaine Brower, Benson Jiang, Donatello Sassaroli, Robert Jacob, Jivko Sinapov

agents reinforcement-learning robotics fine-tuning

arXiv Machine Learning

Jul 17

Domain Adaptation of Mismatched Proximal Denoiser for Plug-and-Play Image Reconstruction

arXiv:2607. 14894v1 Announce Type: cross Abstract: Plug-and-play proximal gradient descent (PnP-PGD) enables flexible image reconstruction by using denoisers as implicit priors.

By Guixian Xu, Jinglai Li, Junqi Tang

fine-tuning

arXiv AI

Jul 17

Multi-LLM Collaborative MRI Report Generation for Visual Instruction Tuning in Brain Oncology

arXiv:2607. 14581v1 Announce Type: new Abstract: Recent advances in large language models (LLMs) and their extension to vision-language models (VLMs) have made it easier to combine text and images for tasks such as report generation.

By Sinyoung Ra, Jonghun Kim, Hyunjin Park

llms nlp fine-tuning multimodal

arXiv AI

Jul 17

Demographically-Conditioned Synthetic Medical Images for Bias Mitigation and Bias Detection in Disease Classifiers

arXiv:2607. 14984v1 Announce Type: new Abstract: Per-subgroup fairness audits of medical image classifiers face a sample-size problem: minority subgroups in held-out test sets have so few samples that the resulting confidence intervals on per-subgroup performance are wider than the bias the audit is meant to detect.

By Mahmoud Ibrahim, Bart Elen, Chang Sun, Gokhan Ertaylan, Michel Dumontier

diffusion fine-tuning safety

arXiv Machine Learning

Jul 17

A vision foundation model for single-cell biology via spatial gene cartography

arXiv:2607. 14163v1 Announce Type: cross Abstract: Most single-cell foundation models are adapted from language models, representing each cell as a sequence of gene tokens.

By Ridvan Yesiloglu, Sakib Mostafa, James Zou, Ash Alizadeh, Jiajun Wu, Lei Xing, Ehsan Adeli, Md Tauhidul Islam

llms computer-vision fine-tuning

arXiv AI

Jul 17

DialogueVPR: Towards Conversational Visual Place Recognition

arXiv:2607. 14115v1 Announce Type: new Abstract: Inspired by how humans communicate spatial information, language-guided geo-localization has gained significant traction for its intuitive and practical value.

By Yukun Song, Changwei Wang, Xingtian Pei, Shibiao Xu, Wenhao Xu, Shunpeng Chen, Yu Zhang, Ke Zhang, Rongtao Xu, Xuxiang Feng, Pengyang Wang

fine-tuning multimodal benchmarks

Get the daily briefing