Fine-tuning and adaptation

LoRA, PEFT, instruction tuning and domain adaptation — adapting a pretrained model without paying to train one.

8,248 stories · RSS feed

arXiv Machine Learning

Jun 9

In-Context Learning of Stochastic Differential Equations with Foundation Inference Models

arXiv:2502. 19049v3 Announce Type: replace Abstract: Stochastic differential equations (SDEs) describe dynamical systems where deterministic flows, governed by a drift function, are superimposed with random fluctuations, dictated by a diffusion function.

By Patrick Seifner, Kostadin Cvejoski, David Berghaus, Cesar Ojeda, Ramses J. Sanchez

llms diffusion fine-tuning

arXiv AI

Jun 9

Thinking-Based Non-Thinking: Solving the Reward Hacking Problem in Training Hybrid Reasoning Models via Reinforcement Learning

arXiv:2601. 04805v2 Announce Type: replace Abstract: Large reasoning models (LRMs) have attracted much attention due to their exceptional performance.

By Siyuan Gan, Jiaheng Liu, Boyan Wang, Tianpei Yang, Runqing Miao, Yuyao Zhang, Fanyu Meng, Junlan Feng, Linjian Meng, Jing Huo, Yang Gao

reinforcement-learning fine-tuning benchmarks

arXiv AI

Jun 9

Generative Reasoning Re-ranker

arXiv:2602. 07774v5 Announce Type: replace-cross Abstract: Recent studies increasingly explore Large Language Models (LLMs) as a new paradigm for recommendation systems due to their scalability and world knowledge.

By Mingfu Liang, Yufei Li, Jay Xu, Kavosh Asadi, Xi Liu, Shuo Gu, Kaushik Rangadurai, Frank Shyu, Shuaiwen Wang, Song Yang, Zhijing Li, Jiang Liu, Mengying Sun, Fei Tian, Xiaohan Wei, Chonglin Sun, Jacob Tao, Shike Mei, Wenlin Chen, Santanu Kolay, Sandeep Pandey, Hamed Firooz, Luke Simon

llms rag nlp reinforcement-learning fine-tuning benchmarks

arXiv AI

Jun 9

Principled Agent Debate: Adversarial Arbitration for Sycophancy Reduction in Large Language Models

arXiv:2606. 07532v1 Announce Type: cross Abstract: RLHF-trained models are systematically biased toward agreement over accuracy, a structural property of the training process.

By Sam Ryan

llms agents reinforcement-learning fine-tuning benchmarks safety

arXiv AI

Jun 9

Emergence of Context Characteristics Sensitivity in Large Language Models

arXiv:2606. 09525v1 Announce Type: cross Abstract: During instruction fine-tuning (IFT), large language models (LLMs) learn to follow instructions by using the provided context to answer a query.

By Nadya Yuki Wangsajaya, Haeun Yu, Isabelle Augenstein

llms reinforcement-learning fine-tuning

arXiv AI

Jun 9

Model Multiplicity for Adversarial Detection in Small Language Model Training on Edge Devices

arXiv:2606. 07857v1 Announce Type: cross Abstract: The rise of edge-based machine learning has enabled distributed adaptation of language models across mobile and IoT devices, offering privacy preservation and real-time responsiveness.

By Stefan Behfar, Richard Mortier

llms robotics fine-tuning safety

arXiv AI

Jun 9

Hiding in Plain Floats: Steganographic Carriers for Indirect Prompt and Content Injection

arXiv:2606. 08403v1 Announce Type: cross Abstract: Text-centered prompt-injection defenses assume that the malicious signal is visible in one of the inspected text views.

By Mudit Sinha, Sanika Chavan

llms fine-tuning multimodal benchmarks

arXiv Machine Learning

Jun 9

DICE: Entropy-Regularized Equilibrium Selection for Stable Multi-Agent LLM Coordination

arXiv:2606. 08068v1 Announce Type: new Abstract: Multi-agent large language model (LLM) systems often fail to reliably outperform a single strong model equipped with best-of-N sampling.

By Yi Xie, Zhanke Zhou, Chentao Cao, Bo Liu, Bo Han

llms agents fine-tuning benchmarks

arXiv AI

Jun 9

ReCoVLA: VLM-Guided Reward Compilation for Failure Recovery in Vision-Language-Action Policies

arXiv:2606. 09630v1 Announce Type: cross Abstract: Vision-language-action (VLA) policies provide strong priors for language-conditioned manipulation, but remain brittle in off-nominal states requiring targeted recovery.

By Haodi Hu, Chung-Ta Huang, Jing Liu, Ye Wang, Kei Suzuki, Matthew Brand, Toshiaki Koike-Akino

llms robotics fine-tuning multimodal

arXiv AI

Jun 9

A Systematic Study of Behavioral Cloning for Scientific Data Annotation

arXiv:2606. 07568v1 Announce Type: cross Abstract: Scientific data annotation, such as tracking animals in video or proofreading neural reconstructions, remains bottlenecked by the "last mile" problem: even with strong automation, verification and correction consume substantial human effort.

By Ishaan Singh Chandok, Core Francisco Park

fine-tuning benchmarks

arXiv AI

Jun 9

TLDR: Compressing Audio Tokens for Efficient Autoregressive Text-to-Speech

arXiv:2606. 09019v1 Announce Type: cross Abstract: Codec-based autoregressive (AR) speech language models have achieved strong text-to-speech (TTS) quality by modeling speech as sequences of discrete audio tokens with large pretrained backbones.

By Yejin Lee, Junwon Moon, Hyoeun Kim, Hyunjin Choi, Heeseung Kim, Kyuhong Shim

llms fine-tuning efficiency multimodal

arXiv AI

Jun 9

Defending Against Malicious Finetuning by Scaling Train-time Adversarial Attacks

arXiv:2606. 07970v1 Announce Type: cross Abstract: Current open-weight large language models (LLMs) are prone to malicious finetuning attacks, which could compromise the safety alignment of LLMs with only a few steps of supervised finetuning (SFT) on poisoned datasets.

By Haoming Wen, Shi Chen, Qingyu Shi, Siyuan Liu, Minrui Luo, Jingzhao Zhang, Tianxing He

llms fine-tuning safety

arXiv AI

Jun 9

ZIPP:Zero-shot Image Personalization from Personas

arXiv:2606. 08841v1 Announce Type: new Abstract: Text-to-image diffusion models are increasingly deployed in open-ended creative contexts, yet their outputs remain impersonal, optimized for aggregate aesthetics rather than individual taste.

By Harini SI, Somesh Singh, Yaman Kumar Singla, David Doermann, Rajiv Ratn Shah

llms diffusion fine-tuning benchmarks safety

arXiv AI

Jun 9

Evaluating Hallucinations in Domain-Adapted Large Language Models

arXiv:2606. 07521v1 Announce Type: cross Abstract: This study investigates the phenomenon of hallucinations in domain-adapted Large Language Models (LLMs), focusing on the fine-tuning of the Llama-2 model with the Lamini dataset.

By Sanchita Porwal, Sai Prasath S, Xingjian Bi, Madelyn Scandlen

llms fine-tuning safety

arXiv Machine Learning

Jun 9

BrainSurgery: Reproducible and Reliable Declarative Weight Manipulations for Model Editing and Upcycling

arXiv:2606. 09707v1 Announce Type: new Abstract: As deep learning models scale, managing, inspecting, and modifying large checkpoints has become increasingly challenging.

By Gianluca Barmina, Annemette Broch Pirchert, Andrea Blasi N\'u\~nez, Lukas Galke Poech, Peter Schneider-Kamp

fine-tuning

arXiv AI

Jun 9

Emergent alignment and the projectability of ethical personas

arXiv:2606. 09475v1 Announce Type: new Abstract: Work on `emergent misalignment' shows that finetuning LLMs on narrow tasks can induce broadly misaligned behavior.

By Guillermo Del Pinal, Youngchan Lee, Cameron McNamara, Alejandro Perez Carballo

llms fine-tuning safety

arXiv Machine Learning

Jun 9

Learning Transfers: Kan Extensions for Neural Invariants

arXiv:2606. 07627v1 Announce Type: new Abstract: Transfer learning presumes that a representation learned on source tasks carries structure that remains usable on related target tasks.

By Luciano Melodia

fine-tuning

arXiv Machine Learning

Jun 9

The Sample Complexity of Parameter-Free Stochastic Convex Optimization

arXiv:2506. 11336v2 Announce Type: replace Abstract: We study the sample complexity of stochastic convex optimization when problem parameters such as the distance to optimality and the Lipschitz constant are unknown.

By Jared Lawrence, Ari Kalinsky, Hannah Bradfield, Yair Carmon, Oliver Hinder

llms fine-tuning

arXiv AI

Jun 9

MedVision: Benchmarking Quantitative Medical Image Analysis

arXiv:2511. 18676v2 Announce Type: replace-cross Abstract: Current vision-language models (VLMs) in medicine are primarily designed for categorical question answering (e.

By Yongcheng Yao, Yongshuo Zong, Raman Dutt, Yongxin Yang, Sotirios A Tsaftaris, Timothy Hospedales

llms nlp fine-tuning multimodal benchmarks

arXiv AI

Jun 9

RetroReasoner: A Reasoning LLM for Strategic Retrosynthesis Prediction

arXiv:2603. 12666v2 Announce Type: replace-cross Abstract: Retrosynthesis prediction aims to identify reactants that can synthesize a given product molecule.

By Hanbum Ko, Chanhui Lee, Ye Rin Kim, Rodrigo Hormazabal, Sehui Han, Sungbin Lim, Sungwoong Kim

llms reinforcement-learning fine-tuning

Get the daily briefing