Fine-tuning and adaptation

LoRA, PEFT, instruction tuning and domain adaptation — adapting a pretrained model without paying to train one.

3,783 stories · RSS feed

arXiv Machine Learning

Jun 26

Reasoning Quality Emerges Early: Data Curation for Reasoning Models

arXiv:2606. 26797v1 Announce Type: new Abstract: Supervised fine-tuning (SFT) on a small, high-quality set of long reasoning traces is an effective approach for eliciting strong reasoning capabilities in Large Language Models (LLMs).

By Hongyi Henry Jin, Wenhan Yang, Meysam Ghaffari, Carlos Morato, Baharan Mirzasoleiman

llms fine-tuning data-engineering

arXiv Machine Learning

Jun 26

Sample-efficient Transfer Reinforcement Learning via Adaptive Reward Shaping and Policy-Ratio Reweighting Strategy

arXiv:2606. 26527v1 Announce Type: new Abstract: Transfer learning improves policy learning efficiency by reusing knowledge from source tasks, providing a feasible paradigm for safe and efficient autonomous highway lane changing decision-making.

By Wenjie Huang, Yang Li, Jingjia Teng, Mingwei Jin, Kai Song, Yougang Bian, Yongfu Li, Qisong Yang, Helai Huang

agents reinforcement-learning fine-tuning safety

arXiv AI

Jun 26

From Celebrities to Anyone: Characterizing AI Nudification Content, Technology, and Community Dynamics on 4chan

arXiv:2606. 27234v1 Announce Type: cross Abstract: AI nudification uses generative models to create synthetic non-consensual sexually explicit imagery (SNEACI) of real individuals.

By Chi Cui, Yixin Wu, Yang Zhang

diffusion fine-tuning

arXiv Machine Learning

Jun 26

GRAG: Generic Response-Augmented Generation Framework for Personalized Conversational Systems

arXiv:2606. 21097v2 Announce Type: replace-cross Abstract: Deploying highly capable personalized conversational agents in resource-constrained or privacy-sensitive environments remains a significant challenge.

By Junfeng Liu, Christopher T. Symons, Ranga Raju Vatsavai

llms agents fine-tuning benchmarks safety

arXiv AI

Jun 26

NebulaExp-8B: An Empirical Post-Training Pipeline via Full-Scale Ablation Research

arXiv:2606. 26671v1 Announce Type: new Abstract: Post-training alignment determines the reasoning and human preference following capabilities of large language models, yet most existing works withhold detailed data construction, filtering rules and training recipes, which hinders community reproducibility and lightweight model optimization.

By Qiaobo Hao, Yangqian Wu, Shunyi Wang, Zhongjian Zhang, Ziqun Li, Yayin He, Muqing Li, Chen Zhong

llms reinforcement-learning fine-tuning efficiency benchmarks safety

arXiv AI

Jun 26

Mitigating Hallucinations via Inter-Layer Consistency Aggregation in Large Vision-Language Models

arXiv:2505. 12343v2 Announce Type: replace-cross Abstract: Despite the impressive capabilities of Large Vision-Language Models (LVLMs), they remain susceptible to hallucinations, where generated content is inconsistent with the input image.

By Kai Tang, Jinhao You, Yichen Guo, Yiding Sun, Dongxu Zhang, Wenya Wang, Hanze Li, Tao Luo, Renyuan Li, Xiande Huang

llms fine-tuning multimodal benchmarks safety

arXiv Machine Learning

Jun 26

Escaping Iterative Parameter-Space Noise: Differentially Private Learning with a Hypernetwork

arXiv:2606. 26772v1 Announce Type: new Abstract: Differentially private (DP) training of neural networks is often hindered by the large amount of noise required by gradient-based methods such as DP-SGD, which repeatedly inject high-dimensional noise in parameter space throughout training.

By Naoki Nishikawa, Shokichi Takakura, Satoshi Hasegawa

rag diffusion fine-tuning safety

arXiv AI

Jun 26

Metaphors are a Source of Cross-Domain Misalignment of Large Reasoning Models

arXiv:2601. 03388v3 Announce Type: replace-cross Abstract: Earlier research has shown that metaphors influence human decision-making, raising the question of whether metaphors also influence large language models (LLMs)' reasoning pathways, given that their training data contain a large number of metaphors.

By Zhibo Hu, Chen Wang, Yanfeng Shu, Hye-young Paik, Liming Zhu

llms fine-tuning safety

arXiv AI

Jun 26

SamaVaani: Auditing and Debiasing Multilingual Clinical ASR for Indian Languages

arXiv:2606. 26901v1 Announce Type: cross Abstract: Automatic Speech Recognition (ASR) is increasingly used to document clinical encounters, yet its reliability in multilingual and demographically diverse Indian healthcare context remains largely unknown.

By Subham Kumar, Prakrithi Shivaprakash, Abhishek Manoharan, Astut Kurariya, Diptadhi Mukherjee, Prabhat Chand, Pratima Murthy, Koustav Rudra, Lekhansh Shukla, Animesh Mukherjee

llms fine-tuning multimodal benchmarks safety

arXiv AI

Jun 26

SSM Adapters via Hankel Reduced-order Modeling: Injection Site Determines Task Suitability in Long-Context Fine-Tuning

arXiv:2606. 26290v1 Announce Type: cross Abstract: While parameter-efficient fine-tuning (PEFT) typically targets attention projectors, its efficacy for tasks requiring sequential state accumulation remains under-explored.

By Omanshu Thapliyal

llms fine-tuning

arXiv AI

Jun 26

Localizing RL-Induced Tool Use to a Single Crosscoder Feature

arXiv:2606. 26474v1 Announce Type: cross Abstract: Fine-tuning through RL reshapes the internal representations of language models to enable agentic behaviors such as tool use, yet the mechanistic basis of these changes remains poorly understood.

By Andrii Shportko, Shubham Bhokare, Ahmed Zeyad A Alzahrani, Bowen Cheng, Gustavo Mercier, Jessica Hullman

llms agents reinforcement-learning fine-tuning

arXiv AI

Jun 26

Tuning Language Models by Mixture-of-Depths Ensemble

arXiv:2410. 13077v2 Announce Type: replace-cross Abstract: Transformer-based Large Language Models (LLMs) traditionally rely on final-layer loss for finetuning and final-layer representations for predictions, potentially overlooking the predictive power embedded in late layers.

By Haoyan Luo, Lucia Specia

llms fine-tuning safety

arXiv AI

Jun 26

Assert, don't describe: Linguistic features that shift LLM reasoning about animal welfare

arXiv:2606. 26104v1 Announce Type: cross Abstract: Animal-welfare advocates produce a lot of writing, and increasingly that writing trains the language models that millions of people then ask about animal welfare.

By Jasmine Brazilek, Harper Dunn

llms fine-tuning benchmarks

arXiv AI

Jun 26

Helpfulness Hurts: Domain-Dependent Degradation of Mid-Trained Compassion Values Under Post-Training

arXiv:2606. 26102v1 Announce Type: cross Abstract: Standard post-training pipelines apply supervised fine-tuning (SFT) and reinforcement learning (RL) to make language models helpful, but these processes may inadvertently degrade values instilled during pre-training.

By Jasmine Brazilek, Juliana Seawell

llms reinforcement-learning fine-tuning benchmarks

arXiv Machine Learning

Jun 26

RSPC: A Benchmark for Modeling Stress and Psychiatric Conditions in Digitally Mediated Relationships using Psychiatrist Annotations

arXiv:2606. 27247v1 Announce Type: new Abstract: In NLP, mental health conditions are often modeled as isolated phenomena, without interpersonal context.

By Parmitha Vangapandu, Sai Ganesh Mokkapati, Sathwik Narkedimilli, MSVPJ Sathvik, Timothy Liu, Simon See, Johannes C. Eichstaedt

llms nlp fine-tuning benchmarks

arXiv AI

Jun 26

Inherited Circuits, Learned Semantics: How Fine-Tuning Creates Evasion Vulnerabilities Invisible to Standard Evaluation

arXiv:2606. 27091v1 Announce Type: cross Abstract: LLMs fine-tuned for security classification are usually evaluated on held-out examples from the same distribution as their training data.

By Ryan Fetterman

llms fine-tuning benchmarks

arXiv Machine Learning

Jun 26

A Multi-Fidelity Convolutional Autoencoder-Transfer Learning Framework for Guided-Wave-Based Damage Diagnosis Using Large Simulated and Limited Experimental Datasets

arXiv:2606. 27304v1 Announce Type: new Abstract: Guided wave-based structural health monitoring (GWSHM) with onboard transducers offers significant potential for the early diagnosis of damage in engineering structures.

By Santosh Kapuria, Abhishek

fine-tuning

arXiv AI

Jun 26

From Lexicon to AI: A Structured-Data Pipeline for Specialized Conversational Systems in Low-Resource Languages

arXiv:2606. 26112v1 Announce Type: cross Abstract: Low-resource languages face a critical challenge in AI development: creating specialized conversational systems without access to massive training corpora.

By Siddhant Hitesh Mantri, Dhara Gorasiya, Malhar Kulkarni, Pushpak Bhattacharya

llms fine-tuning efficiency data-engineering

arXiv AI

Jun 26

LLM-based Models for Detecting Emerging Topics in Service Feedback

arXiv:2606. 26595v1 Announce Type: new Abstract: Enhancing the analysis of service feedback is essential for public sector organizations, particularly tax administrations, where trust and compliance depend on fair and effective service delivery.

By Mahsa Tavakoli, Ruth Bankey, Cristi\'an Bravo

llms fine-tuning efficiency safety

arXiv AI

Jun 26

Safe Autoregressive Image Generation with Iterative Self-Improving Codebooks

arXiv:2606. 27147v1 Announce Type: cross Abstract: Unlike diffusion-based models that operate in continuous latent spaces, autoregressive unified multimodal models produce images by sequentially predicting discretized visual tokens.

By Yunqi Xue, Zhijiang Li, Philip Torr, Jindong Gu

rag diffusion fine-tuning efficiency multimodal safety

Get the daily briefing