Fine-tuning and adaptation

LoRA, PEFT, instruction tuning and domain adaptation — adapting a pretrained model without paying to train one.

8,248 stories · RSS feed

arXiv Machine Learning

Jun 9

Your Self-Play Algorithm is Secretly an Adversarial Imitator: Understanding LLM Self-Play through the Lens of Imitation Learning

arXiv:2602. 01357v2 Announce Type: replace Abstract: Self-play post-training methods has emerged as an effective approach for finetuning large language models and turn the weak language model into strong language model without preference data.

By Shangzhe Li, Xuchao Zhang, Chetan Bansal, Weitong Zhang

llms reinforcement-learning fine-tuning safety

arXiv AI

Jun 9

Anchor-Conditioned Compositional Control for Landscape Image Generation

arXiv:2606. 07638v1 Announce Type: cross Abstract: Image generative models, though widely used as creative tools, offer limited support for the kind of compositional control that photographers and visual artists routinely exercise.

By Gadha Lekshmi P, Govind Arun, Rohith Syam, Ahmed Elgammal

diffusion fine-tuning benchmarks safety

arXiv Machine Learning

Jun 9

Discovering Functionally Selective Brain Regions with a Deep Topographic Multimodal Model

arXiv:2606. 09770v1 Announce Type: cross Abstract: Nearby neurons in cortex share similar response profiles, producing systematic spatial organization across sensory and cognitive systems.

By Badr AlKhamissi, Johannes Mehrer, Lara Marinov, Ahmed Abdelaal, Abdulkadir Gokce, Martin Schrimpf

fine-tuning multimodal

arXiv AI

Jun 9

Benchmarking Vision-Language-Action Models on SO-101: Failure and Recovery Analysis

arXiv:2606. 08881v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) models have demonstrated strong generalization in robotic manipulation, yet existing evaluations are primarily conducted in simulation or on expensive robotic platforms, leaving their robustness on affordable real-world robots largely unexplored.

By Yi Yu, Xinchuan Qiu

robotics fine-tuning multimodal benchmarks

arXiv Machine Learning

Jun 9

Now You (Still) See Me: Detecting Evasive Steganographic Payloads in LLMs

arXiv:2606. 09411v1 Announce Type: cross Abstract: Large language models can be fine-tuned to encode prompt-borne secrets into fluent, seemingly benign outputs.

By Charles Westphal, Timothy Douglas, Keivan Navaie, Tiago Pimentel, Fernando E. Rosas

llms fine-tuning benchmarks

arXiv AI

Jun 9

Trait-space Monitoring for Emergent Misalignment During Supervised Finetuning

arXiv:2606. 07631v1 Announce Type: cross Abstract: Emergent misalignment (EM) occurs when narrow finetuning causes a model to behave dangerously outside the finetuning task.

By Huy Nghiem, Sy-Tuyen Ho, Sarah Wiegreffe, Hal Daum\'e III

llms fine-tuning safety

arXiv Machine Learning

Jun 9

ReTabSyn: Realistic Tabular Data Synthesis via Reinforcement Learning

arXiv:2603. 10823v2 Announce Type: replace-cross Abstract: Deep generative models can help with data scarcity and privacy by producing synthetic training data, but they struggle in low-data, imbalanced tabular settings to fully learn the complex data distribution.

By Xiaofeng Lin, Seungbae Kim, Zhuoya Li, Zachary DeSoto, Charles Fleming, Guang Cheng

llms reinforcement-learning fine-tuning benchmarks safety

arXiv Machine Learning

Jun 9

BrainSurgery: Reproducible and Reliable Declarative Weight Manipulations for Model Editing and Upcycling

arXiv:2606. 09707v1 Announce Type: new Abstract: As deep learning models scale, managing, inspecting, and modifying large checkpoints has become increasingly challenging.

By Gianluca Barmina, Annemette Broch Pirchert, Andrea Blasi N\'u\~nez, Lukas Galke Poech, Peter Schneider-Kamp

fine-tuning

arXiv AI

Jun 9

CrossVLA: Cross-Paradigm Post-Training and Inference Optimization for Vision-Language-Action Models

arXiv:2605. 21854v2 Announce Type: replace-cross Abstract: Vision-Language-Action (VLA) models have rapidly converged on a small set of architectural patterns: discrete-token autoregression (e.

By Zhi Liu

llms reinforcement-learning fine-tuning multimodal benchmarks safety

arXiv AI

Jun 9

PACT: Learning Diverse Diagnostic Strategies via Privileged Synthesis and Branch Consensus

arXiv:2606. 08938v1 Announce Type: cross Abstract: Clinical diagnosis requires flexible use of multiple reasoning paradigms under incomplete patient information.

By Gen Li, Yuanze Hu, Zhichao Yang, Qingchen Yu, Jianwei Lv, Yue Guo, Yujing Liu, Faguo Wu, Hongwei Zheng, Xiandong Li, Bo Yuan, Yifan Sun, Zhaoxin Fan

llms agents fine-tuning benchmarks

arXiv AI

Jun 9

Single-Cell Cross-Modal Transfer by Adversarial Fine-Tuning of Foundation Models

arXiv:2606. 07676v1 Announce Type: cross Abstract: Spatial transcriptomics (ST) is a powerful tool for exploring biological properties dependent on structure, proximity, and interaction in tissue.

By Joseph Boyd, Matthew Lyon, Martino Mansoldo, Christian Hurry, Finnian Firth

fine-tuning multimodal safety

arXiv AI

Jun 9

Large Language Models for Imbalanced Classification: Diversity makes the difference

arXiv:2510. 09783v2 Announce Type: replace-cross Abstract: Oversampling is one of the most widely used approaches for addressing imbalanced classification.

By Dang Nguyen, Sunil Gupta, Kien Do, Thin Nguyen, Taylor Braund, Alexis Whitton, Svetha Venkatesh

llms fine-tuning benchmarks

arXiv AI

Jun 9

Decision-Aware Memory Cards: Counterfactual-Inspired Context Selection and Compression for Tool-Using LLM Agents

arXiv:2606. 08151v1 Announce Type: new Abstract: Tool-using LLM agents often fail not because relevant text is absent, but because decisive evidence is not selected, compressed, or surfaced at action time.

By Xinyu Guan, Qianyang Zhao, Yuming Deng

llms rag agents fine-tuning benchmarks

arXiv AI

Jun 9

Sparrow: Sparse Rollout for Stable and Efficient Long-context RL of Large Language Models

arXiv:2606. 08446v1 Announce Type: cross Abstract: Despite being powerful, reinforcement learning with verifiable rewards (RLVR) induces extremely long COT, making it computationally expensive.

By Yang Zhou, Ranajoy Sadhukhan, Zhaofeng Sun, Zhuoming Chen, Souvik Kundu, Saket Dingliwal, Sai Muralidhar Jayanthi, Aram Galstyan, Haizhong Zheng, Beidi Chen

llms reinforcement-learning fine-tuning efficiency

arXiv AI

Jun 9

Defending Against Malicious Finetuning by Scaling Train-time Adversarial Attacks

arXiv:2606. 07970v1 Announce Type: cross Abstract: Current open-weight large language models (LLMs) are prone to malicious finetuning attacks, which could compromise the safety alignment of LLMs with only a few steps of supervised finetuning (SFT) on poisoned datasets.

By Haoming Wen, Shi Chen, Qingyu Shi, Siyuan Liu, Minrui Luo, Jingzhao Zhang, Tianxing He

llms fine-tuning safety

arXiv AI

Jun 9

TempoBench: Evaluating Temporal Causal Reasoning in Large Language Models

arXiv:2510. 27544v2 Announce Type: replace Abstract: Temporal reasoning involves understanding how systems evolve over time through input-driven state transitions.

By Nikolaus Holzer, William Fishell, Baishakhi Ray, Mark Santolucito

llms fine-tuning benchmarks

arXiv AI

Jun 9

HARP: Efficient Data Selection for Finetuning Large Language Models

arXiv:2606. 07690v1 Announce Type: cross Abstract: Finetuning data selection requires balancing two competing goals: selecting examples that improve the downstream objective, and doing so without repeatedly finetuning models.

By Ning Wang, Zhengxin Zhang, Maosen Tang, Yitang Gao, Claire Cardie, Sainyam Galhotra

llms rag fine-tuning efficiency safety

arXiv AI

Jun 9

Muon Learns More Robust and Transferable Features than Adam

arXiv:2606. 09658v1 Announce Type: cross Abstract: Muon has recently emerged as a state-of-the-art optimizer for pretraining Large Language Models (LLMs) and vision classifiers.

By Tianyu Ruan, Fengzhuo Zhang, Shuche Wang, Shihua Zhang

llms fine-tuning benchmarks

arXiv AI

Jun 9

Self-Mined Hardness for Safety Fine-Tuning

arXiv:2605. 03226v2 Announce Type: replace-cross Abstract: Safety fine-tuning of language models typically requires a curated adversarial dataset.

By Prakhar Gupta, Garv Shah, Donghua Zhang

llms fine-tuning multimodal safety

arXiv Machine Learning

Jun 9

Breaking the Tokenizer Barrier: On-Policy Distillation across Model Families

arXiv:2606. 09456v1 Announce Type: new Abstract: On-Policy Distillation (OPD) has become a core technique in the post-training of Large Language Models (LLMs) for transferring knowledge from domain experts to student models.

By Yifan Niu, Han Xiao, Dongyi Liu, Zelong Wang, Dihong Gong, Yasheng Wang, Jia Li

llms nlp fine-tuning efficiency benchmarks

Get the daily briefing