Fine-tuning and adaptation

LoRA, PEFT, instruction tuning and domain adaptation — adapting a pretrained model without paying to train one.

8,208 stories · RSS feed

arXiv Machine Learning

Jul 14

Local Multimodal Music Alignment from Global Supervision

arXiv:2607. 10023v1 Announce Type: cross Abstract: Understanding music requires understanding localized relationships across data modalities, e.

By Irmak Bukey, Zachary Novack, Jongmin Jung, Dasaem Jeong, Chris Donahue

fine-tuning multimodal safety

arXiv AI

Jul 14

ANCHOR: Automated Alignment Auditing for CLI Agents on Real-World Harm

arXiv:2607. 10455v1 Announce Type: new Abstract: Autonomous CLI agents can now execute hundreds of actions across multi-hour sessions: writing code, executing shell commands, browsing the web, and managing cloud infrastructure, all with minimal human oversight.

By Kefan Song, Yanjun Qi

agents fine-tuning safety

arXiv AI

Jul 14

Unveiling the Mechanisms of Multi-Hop Reasoning in Transformers via Identity Bridge

arXiv:2509. 24653v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) excel at multi-hop reasoning in distribution, yet fail on unseen compositions, a phenomenon known as the curse of two-hop reasoning.

By Pengxiao Lin, Zheng-An Chen, Zhi-Qin John Xu

llms fine-tuning

arXiv AI

Jul 14

StanceMoE: Mixture-of-Experts Architecture for Stance Detection

arXiv:2604. 00878v2 Announce Type: replace-cross Abstract: Actor-level stance detection aims to determine an author expressed position toward specific geopolitical actors mentioned or implicated in a text.

By Abdullah Al Shafi, Md. Milon Islam, Sk. Imran Hossain, K. M. Azharul Hasan

llms fine-tuning

arXiv AI

Jul 14

World Models as Adversaries: Multi-Agent Self-Play Fine-Tuning for Robust Motion Planning

arXiv:2607. 10630v1 Announce Type: cross Abstract: Robust motion planning in dense traffic requires autonomous vehicles to interact in rare and safety-critical scenarios that are underrepresented in naturalistic driving data.

By Tong Nie, Yuewen Mei, Junlin He, Yihong Tang, Jian Sun, Wei Ma

agents reinforcement-learning fine-tuning benchmarks safety

arXiv AI

Jul 14

DiffEEG: A Self-Supervised Denoising Diffusion Model for Learning EEG Generic Representations

arXiv:2607. 11578v1 Announce Type: cross Abstract: Deep learning for EEG-based seizure detection faces critical challenges: severe annotation scarcity and extreme class imbalance, where ictal events comprise less than 10\% of clinical recordings.

By Abdulkader Helwan, Lina Abou-Abbas, Hussein El Amouri, Belkacem Chikhaoui, Khadidja Henni

diffusion reinforcement-learning fine-tuning

arXiv AI

Jul 14

EmoStyle: Affective Conditioning of Style-Specialist Experts for Emotional Image Generation

arXiv:2607. 10165v1 Announce Type: cross Abstract: Emotion-aware artistic image generation requires an image to match the input prompt, follow the specified artistic style, and convey the target emotion.

By Dexiang Hong, Yijie Guo, Weidong Chen, Xinyan Liu, Zixuan Zou, Zhendong Mao, Yongdong Zhang

llms diffusion fine-tuning multimodal safety

arXiv AI

Jul 14

Agentic-DPO: From Imitation to Agentic Policy Optimization on Expert Trajectories

arXiv:2607. 10601v1 Announce Type: new Abstract: Large Language Model (LLM) agents are commonly trained from expert trajectories using supervised fine-tuning (SFT), which treats multi-turn agent behavior as ordinary text imitation.

By Yixiong Chen, Alan Yuille

llms agents reinforcement-learning fine-tuning

arXiv AI

Jul 14

On the Necessity of Output Distribution Reweighting for Effective Class Unlearning

arXiv:2506. 20893v5 Announce Type: replace-cross Abstract: In this paper, we reveal a significant shortcoming in class unlearning evaluations: overlooking the underlying class geometry can cause information leakage about the forgotten class.

By Ali Ebrahimpour-Boroojeny, Yian Wang, Hari Sundaram

fine-tuning benchmarks safety

arXiv AI

Jul 14

A Multi-Model Metric-based Selection Framework for Abstractive Text summarization

arXiv:2606. 05494v3 Announce Type: replace-cross Abstract: Automatic text summarization has become increasingly important due to the rapid growth of digital textual information.

By Ahmed Alansary, Ali Hamdi

llms nlp fine-tuning

arXiv Machine Learning

Jul 14

From Direction to Magnitude: How Multimodal Instruction-Tuning Reorganizes the Geometric Encoding of Identity-Specifying Prompts in Transformer Hidden States

arXiv:2607. 09842v1 Announce Type: new Abstract: We investigate whether identity-specifying system prompts produce statistically distinguishable geometric fingerprints in the hidden-state trajectories of four open-weight transformer language models spanning four post-training regimes: no training (Gemma-4-E4B base), multimodal RLHF (Gemma-4-E4B-it), RL distillation (DeepSeek-R1-Distill-Qwen-7B), and SFT (Qwen2.

By Jorge A. Castillo, Marco Torres Y\'evenes, Juan Carlos Lanas

llms reinforcement-learning fine-tuning efficiency multimodal

arXiv AI

Jul 14

Co4ICF: Co-evolving Physics-Informed Surrogate and RL-based Pulse Optimizer for Inertial Confinement Fusion

arXiv:2607. 10366v1 Announce Type: new Abstract: Offline-trained surrogates for Inertial Confinement Fusion (ICF) suffer a well-known failure mode that iterative optimizers drive inputs into out-of-distribution (OOD) regions where predictions become unreliable.

By Jiatong Zhao, Tengyue Zhang, Yuhan Wang, Fuyuan Wu, Junchi Yan

reinforcement-learning fine-tuning benchmarks

arXiv AI

Jul 14

Workload-Driven Optimization for On-Device Real-Time Subtitle Translation

arXiv:2607. 09957v1 Announce Type: cross Abstract: This report studies on-device English-to-Traditional-Chinese subtitle translation for Taiwan under short inputs, short outputs, batch-size-one inference, low latency, and privacy constraints.

By Tsz-To Wong

llms rag nlp fine-tuning efficiency benchmarks safety

arXiv AI

Jul 14

The Nuts and Bolts of Natural Language to SQL Translation: A Systematic Analysis of Model Pipeline Optimisation Approaches and their Interactions

arXiv:2607. 10911v1 Announce Type: cross Abstract: In the age of large language models, Natural Language to SQL (NL2SQL) translation remains an open problem with many useful applications.

By Filip Klubicka, Vasudevan Nedumpozhimana, Sneha Rautmare, Bora Caglayan, Mingxue Wang, John D. Kelleher

llms fine-tuning benchmarks data-engineering

arXiv AI

Jul 14

Efficiently Adapting Spoken Language Models for the Singaporean Context

arXiv:2607. 10092v1 Announce Type: cross Abstract: Spoken language models (SLMs) unify speech perception and reasoning, but adapting them to sensitive domains is underexplored, especially when the original training data is inaccessible and the use case demands multilingual, spoken-query interaction.

By Ng Jia Sheng Jason

llms fine-tuning

arXiv AI

Jul 14

VoxENES 2026: Benchmarking Generalization of Speech Spoofing Detectors Against LLM-Era TTS and Voice Conversion

arXiv:2607. 11706v1 Announce Type: cross Abstract: Modern LLM-driven text-to-speech (TTS) and voice conversion (VC) systems produce synthetic speech that differs from the generators represented in many legacy spoofing benchmarks.

By Aastha Sharma, Guangjing Wang

llms fine-tuning multimodal benchmarks

arXiv AI

Jul 14

Nested-ReFT: Efficient Reinforcement Learning for Large Language Model Fine-Tuning via Off-Policy Rollouts

arXiv:2508. 10123v3 Announce Type: replace-cross Abstract: Advanced reasoning in LLMs on challenging domains like mathematical reasoning can be tackled using verifiable rewards based reinforced fine-tuning (ReFT).

By Maxime Heuillet, Yufei Cui, Boxing Chen, Audrey Durand, Prasanna Parthasarathi

llms reinforcement-learning fine-tuning benchmarks safety

arXiv Machine Learning

Jul 14

Enhancing Reasoning for Diffusion LLMs via Distribution Matching Policy Optimization

arXiv:2510. 08233v3 Announce Type: replace Abstract: Diffusion large language models (dLLMs) are promising alternatives to autoregressive large language models (AR-LLMs), as they potentially allow higher inference throughput.

By Yuchen Zhu, Wei Guo, Jaemoo Choi, Petr Molodyk, Bo Yuan, Molei Tao, Yongxin Chen

llms diffusion reinforcement-learning fine-tuning efficiency benchmarks

arXiv AI

Jul 14

Filtering Harmful Actions Isn't Enough: Phantom Transfer in Agentic SDF

arXiv:2607. 10750v1 Announce Type: new Abstract: Synthetic data is widely used to train large language models because it is inexpensive to generate and easy to control.

By Chinmayi Dixit

llms agents reinforcement-learning fine-tuning benchmarks safety

arXiv Machine Learning

Jul 14

Memory Savings at What Cost? A Study of Alternatives to Backpropagation

arXiv:2506. 21833v2 Announce Type: replace Abstract: Forward-mode automatic differentiation (FmAD) and zero-order (ZO) optimization are increasingly proposed as memory-efficient, backpropagation-free alternatives for large language model (LLM) fine-tuning, yet their benefits are typically evaluated only against standard backpropagation (BP), omitting memory-efficient variants such as activation checkpointing.

By Kunjal Panchal, Sunav Choudhary, Yuriy Brun, Hui Guan

llms fine-tuning multimodal benchmarks

Get the daily briefing