Fine-tuning and adaptation

LoRA, PEFT, instruction tuning and domain adaptation — adapting a pretrained model without paying to train one.

3,943 stories · RSS feed

arXiv Machine Learning

Jun 3

Visual Instruction Tuning Aligns Modalities through Abstraction

arXiv:2606. 03871v1 Announce Type: cross Abstract: Visual instruction tuning effectively adapts a pre-trained Large Language Model (LLM) to process image information alongside text.

By Luis Palacios, Lorenzo Basile, Diego Doimo, Alberto Cazzaniga

llms rag fine-tuning multimodal benchmarks safety

arXiv AI

Jun 3

Diagnosing Knowledge Gaps in LLM Tool Use: An Agentic Benchmark for Novel API Acquisition

arXiv:2606. 03657v1 Announce Type: new Abstract: Large language models for code generation often need to use APIs that are absent from their pretraining data.

By Jinnuo Liu, Yue Peng, Jinhan Niu, Hongyi Wen

llms agents fine-tuning benchmarks

arXiv AI

Jun 3

What Makes Interaction Trajectories Effective for Training Terminal Agents?

arXiv:2606. 03461v1 Announce Type: new Abstract: Stronger code agents are commonly assumed to be superior teachers for post-training, yet this assumption remains poorly disentangled from task difficulty, harness design, and student capacity.

By Sidi Yang, Chaofan Tao, Jierun Chen, Tiezheng Yu, Ruoyu Wang, Yuxin Jiang, Yiming Du, Wendong Xu, Jing Xiong, Taiqiang Wu, Lifeng Shang, Xiaohui Li, Ngai Wong, Haoli Bai

llms agents fine-tuning benchmarks

arXiv AI

Jun 3

BaltiVoice: A Speech Corpus and Fine-tuned Whisper ASR System for the Balti Language

arXiv:2606. 03504v1 Announce Type: cross Abstract: We present BaltiVoice, a 16.

By Muhammad Ali

fine-tuning multimodal

arXiv Machine Learning

Jun 3

Skill-RM: Unifying Heterogeneous Evaluation Criteria via Agent Skill

arXiv:2606. 03980v1 Announce Type: new Abstract: Reward models (RMs) provide critical feedback signals for LLM post-training, notably in reinforced fine-tuning (RFT) and reinforcement learning (RL) pipelines.

By Tao Chen, Gangwei Jiang, Pengyu Cheng, Siyuan Huang, Yihao Liu, Jingwei Ni, Jiaqi Guo, Mengyu Zhou, Kai Tang, Junling Liu, Qinliang Su, Xiaoxi Jiang, Guanjun Jiang

llms agents reinforcement-learning fine-tuning benchmarks

arXiv Machine Learning

Jun 3

Spatial Transcriptomics-Guided Alignment Enhances Molecular Profiling in Pathology Foundation Model

arXiv:2606. 03644v1 Announce Type: new Abstract: Comprehensive molecular profiling is essential for modern precision oncology but remains hindered by prohibitive costs, specimen exhaustion, and protracted turnaround times.

By Fengtao Zhou, Yingxue Xu, Zhengyu Zhang, Yihui Wang, Zhengrui Guo, Ling Liang, Jiabo Ma, Cheng Jin, Ziyi Liu, Huajun Zhou, Hongyi Wang, Du Cai, Chenglong Zhao, Xi Wang, Can Yang, Yu Wang, Wenbin Li, Feng Gao, Zhe Wang, Zhenhui Li, Xiuming Zhang, Li Liang, Hao Chen

fine-tuning multimodal safety

arXiv AI

Jun 3

Finetuning-Free Diffusion Model with Adaptive Constraint Guidance for Inorganic Crystal Structure Generation

arXiv:2604. 13354v2 Announce Type: replace-cross Abstract: The discovery of inorganic crystal structures with targeted properties is a significant challenge in materials science.

By Auguste de Lambilly, Vladimir Baturin, David Portehault, Guillaume Lambard, Nataliya Sokolovska, Florence d'Alch\'e-Buc, Jean-Claude Crivello

diffusion fine-tuning benchmarks

arXiv Machine Learning

Jun 3

Synthetic Hallucinations, Real Gains: Hard Negatives from Frontier Models for FIM Hallucination Mitigation

arXiv:2606. 03130v1 Announce Type: new Abstract: Small open-source code models that power IDE autocomplete still emit hallucinated Fill-in-the-Middle (FIM) completions: syntactically natural calls to methods, parameters, variables, and imports that do not exist in the surrounding project.

By Mahdi Erfanian, Nelson Daniel Troncoso, Aashna Garg, Amabel Gale, Xiaoyu Liu, Pareesa Ameneh Golnari, Shengyu Fu

llms reinforcement-learning fine-tuning benchmarks safety

arXiv Machine Learning

Jun 3

CoughSense: Five-Class Respiratory Disease Classification via Whisper Encoder Fine-Tuning and Dual-Encoder Cross-Attention Fusion with Balanced Contrastive Learning

arXiv:2606. 02998v1 Announce Type: new Abstract: Automated cough analysis offers a path to low-cost respiratory screening, but most existing work stops at binary COVID-19 detection.

By Nikhil Vincent

fine-tuning multimodal benchmarks

arXiv AI

Jun 3

D-Judge: Disrupting Multi-Turn Jailbreaks using Semantics-Preserving Output Rewriting

arXiv:2606. 02640v1 Announce Type: cross Abstract: Multi-turn jailbreak attacks pose a growing threat to large language model (LLM) safety because they exploit feedback from auxiliary judge models to iteratively refine prompts toward harmful goals.

By Huanli Gong, Zhipeng Wei, Yu Fu, Haz Sameen Shahgir, Ananya Gupta, Yue Dong, N. Benjamin Erichson

llms fine-tuning benchmarks safety

arXiv Machine Learning

Jun 3

RESCAST-100K: A Comprehensive Dataset for Cross-Domain Residential Load and Indoor Temperature Forecasting

arXiv:2606. 02852v1 Announce Type: new Abstract: Accurate short-term forecasting of residential energy load and indoor temperature is essential for home energy management systems, grid-level demand response, and community energy efficiency efforts.

By Jainam Dhruva, Yousaf Raza, A. B. Siddique, Simone Silvestri

llms robotics fine-tuning benchmarks

arXiv Machine Learning

Jun 3

Pruning Deep Neural Networks via the Marchenko--Pastur Distribution

arXiv:2606. 02608v1 Announce Type: new Abstract: We study a Marchenko--Pastur (MP) random-matrix approach to pruning deep neural networks with very small post-pruning fine-tuning budgets.

By Leonid Berlyand, Theo Bourdais, Houman Owhad, Yitzchak Shmalo

fine-tuning efficiency

arXiv Machine Learning

Jun 3

FGRPO: Federated GRPO with Adaptive Aggregation on Non-IID Data

arXiv:2606. 03094v1 Announce Type: new Abstract: Recent advances in language models have established reinforcement learning as the primary paradigm for eliciting self-correction and long-chain reasoning.

By Pengyu Chen, Shaowei Li, Kai Wang, Yunsheng Yuan, Kai Han, Jun Luo, Feng Li

llms reinforcement-learning fine-tuning safety

arXiv AI

Jun 3

A Training-Free Mixture-of-Agents Framework for Multi-Document Summarization using LLMs and Knowledge Graphs

arXiv:2606. 03867v1 Announce Type: cross Abstract: Multi-Document Summarization (MDS) plays a critical role in distilling essential information from collections of textual data.

By Cuong Vuong Tuan, Trang Mai Xuan, Tien-Cuong Nguyen, Vu-Duc Ngo, Thien Van Luong

llms agents nlp fine-tuning benchmarks

arXiv AI

Jun 3

Narrow Secret Loyalty Dodges Black-Box Audits

arXiv:2605. 06846v3 Announce Type: replace-cross Abstract: Recent work identifies secret loyalties as a distinct threat from standard backdoors.

By Alfie Lamerton, Fabien Roger

fine-tuning

arXiv Machine Learning

Jun 3

PerchRL: Vision-Based Agile Perching on Inclined Platforms under Rapid and Irregular Motion

arXiv:2606. 03441v1 Announce Type: cross Abstract: Autonomous vision-based perching of quadrotors on moving inclined platforms is critical for air-ground collaboration but remains challenging due to the limited field of view (FOV).

By Zihong Lu, Zongzhuo Liu, Huaxu Li, Jinqiang Cui, Jie Mei, Youmin Gong, U Kei Cheang, Boyu Zhou

agents reinforcement-learning fine-tuning

arXiv AI

Jun 3

ProEval: Proactive Failure Discovery and Efficient Performance Estimation for Generative AI Evaluation

arXiv:2604. 23099v2 Announce Type: replace-cross Abstract: Evaluating generative AI models is increasingly resource-intensive due to slow inference, expensive raters, and a rapidly growing landscape of models and benchmarks.

By Yizheng Huang, Wenjun Zeng, Aditi Kumaresan, Zi Wang

fine-tuning benchmarks safety

arXiv AI

Jun 3

PHASE: Physiology-Aware Hyperspectral Reconstruction via Object-to-Human Domain Adaptation

arXiv:2511. 13020v2 Announce Type: replace-cross Abstract: Although hyperspectral imaging offers unparalleled non-invasive physiological insight, its bulky hardware, slow acquisition, and regulatory burden severely limit its clinical availability.

By Yufei Wen, Shuxing Zhong, Jingdan Kang, Yuting Zhang, Jintai Chen, Kaishun Wu

fine-tuning benchmarks safety

arXiv AI

Jun 3

Learning the Neighborhood: Contrast-Free Multimodal Self-Supervised Molecular Graph Pretraining

arXiv:2509. 22468v2 Announce Type: replace-cross Abstract: High-quality molecular representations are essential for property prediction and molecular design, yet large labeled datasets remain scarce.

By Boshra Ariguib, Mathias Niepert, Andrei Manolache

llms rag diffusion fine-tuning multimodal benchmarks

arXiv AI

Jun 3

CoMPAS3D: A Dataset and Benchmark for Interactive Motion

arXiv:2507. 19684v2 Announce Type: replace-cross Abstract: Socially interactive humanoid robots must engage with humans through their bodies, adapting in real time to a partner's movement, intent, and abilities.

By Bermet Burkanova, Yasaman Etesam, Payam Jome Yazdian, Trinity Evans, Chuxuan Zhang, Zoe Stanley, Paige Tutt\"os\'i, Angelica Lim

llms robotics fine-tuning multimodal benchmarks safety

Get the daily briefing