Fine-tuning and adaptation

LoRA, PEFT, instruction tuning and domain adaptation — adapting a pretrained model without paying to train one.

8,208 stories · RSS feed

arXiv AI

Jun 30

Data-Efficient Multimodal Alignment for Histopathology-based Molecular Prediction

arXiv:2606. 29949v1 Announce Type: cross Abstract: H&E-stained whole-slide images offer cohort-scale availability and rich spatial context but lack molecular specificity, whereas bulk RNA-seq provides transcriptome-wide resolution at high cost with limited archival availability.

By Dominik Winter, Dominik Vonficht, Lo\"ic Le Bescond, Christian Gebbe, Marco Rosati, Richard J. Chen, Markus Schick, Ross Stewart, Nicolas Brieu

llms fine-tuning multimodal safety

arXiv Machine Learning

Jun 30

On the Vulnerability of Parameter-Level Defenses to Model Merging

arXiv:2606. 30360v1 Announce Type: new Abstract: The training-free integration of expert models via model merging has exposed significant security risks, enabling free-riders to combine specialized models without authorization.

By Kuangpu Guo, Qingyan Zheng, Jian Liang, Yongcan Yu, Zilei Wang, Ran He, Tieniu Tan

fine-tuning

arXiv AI

Jun 30

Animation2Code: Evaluating Temporal Visual Reasoning in Video-to-Code Generation

arXiv:2606. 28593v1 Announce Type: cross Abstract: While recent vision-language models (VLMs) have achieved significant improvements on static visual-to-code tasks such as generating code for webpages, charts, or SVGs, it remains unclear whether they can recover temporal dynamics when motion is present.

By Anya Ji, Abhijith Varma Mudunuri, David M. Chan, Alane Suhr

llms fine-tuning multimodal benchmarks safety

arXiv AI

Jun 30

Post-training for Efficient Communication via Convention Formation

arXiv:2508. 06482v2 Announce Type: replace-cross Abstract: Humans communicate with increasing efficiency in multi-turn interactions, by adapting their language and forming ad-hoc conventions.

By Yilun Hua, Evan Wang, Yoav Artzi

llms fine-tuning benchmarks

arXiv AI

Jun 30

Towards Harnessing the Collaborative Power of Large and Small Models for Domain Tasks

arXiv:2504. 17421v2 Announce Type: replace-cross Abstract: Large language models (LMs) offer broad generalization capabilities but require vast amounts of data and computational resources for domain-specific tasks; small models (SMs), in contrast, are more efficient and tailored to specific domains yet lack general-purpose coverage.

By Yang Liu, Kejia Zhang, Bingjie Yan, Tianyuan Zou, Jianqing Zhang, Zixuan Gu, Xiangsen Chen, Jianbing Ding, Xidong Wang, Jingyi Li, Xiaozhou Ye, Ye Ouyang, Qiang Yang, Ya-Qin Zhang

llms fine-tuning safety

arXiv AI

Jun 30

Mechanistic Personality Analysis of LLMs Steering Personality via Latent Feature Interventions

arXiv:2606. 28770v1 Announce Type: new Abstract: Large Language Models (LLMs) have demonstrated the ability to simulate human-like OCEAN personality traits in generated text.

By David Courtis, Ting Hu

llms fine-tuning benchmarks safety

arXiv Machine Learning

Jun 30

Prototype Latent World Model Replay for Class-Incremental Learning

arXiv:2606. 29465v1 Announce Type: new Abstract: Class-incremental learning requires a model to learn new classes while preserving decision regions for old ones.

By Weizhi Nie, Hui Wang, Weijie Wang, Yuting Su

fine-tuning benchmarks

arXiv Machine Learning

Jun 30

PCGD: Physics-Guided Conditional Graph Diffusion for TCAD Device Simulation

arXiv:2606. 29272v1 Announce Type: new Abstract: Technology computer-aided design (TCAD) semiconductor device simulation is fundamentally constrained by the high computational cost of iteratively solving coupled drift-diffusion equations.

By Yihan Zhang, Zhiteng Zhang, Kun Chen, Chen Wang

diffusion fine-tuning benchmarks

arXiv Machine Learning

Jun 30

GLACIER: Rethinking Mass Spectrum Prediction as an Object Detection Problem

arXiv:2606. 29161v1 Announce Type: new Abstract: Predicting tandem mass spectra (MS/MS) from molecular structures represents a central task in analytical chemistry with direct relevance to clinical metabolomics, systems biology, and adjacent disciplines.

By Rui-Xi Wang, Runzhong Wang, Connor W. Coley

llms computer-vision fine-tuning benchmarks

arXiv AI

Jun 30

Demonstration-Free Robotic Control via LLM Agents

arXiv:2601. 20334v2 Announce Type: replace-cross Abstract: Robotic manipulation has increasingly adopted vision-language-action (VLA) models, which achieve strong performance but typically require task-specific demonstrations and fine-tuning, and often generalize poorly under domain shift.

By Brian Y. Tsui, Alan Y. Fang, Tiffany J. Hwu

llms agents robotics fine-tuning multimodal benchmarks

arXiv Machine Learning

Jun 30

LoRAShield: Data-Free Editing Alignment for Secure Personalized LoRA Sharing

arXiv:2507. 07056v2 Announce Type: replace-cross Abstract: The proliferation of Low-Rank Adaptation (LoRA) models has democratized personalized text-to-image generation, enabling users to share lightweight models (e.

By Jiahao Chen, Junhao Li, Yiming Wang, Yong Yang, Yi Jiang, Chunyi Zhou, Qingming Li, Tianyu Du, Shouling Ji

llms diffusion fine-tuning safety

arXiv AI

Jun 30

Translating Natural Language to Strategic Temporal Specifications via LLMs

arXiv:2606. 30441v1 Announce Type: cross Abstract: A rigorous formalization of system requirements is a fundamental prerequisite for the verification of Multi-Agent Systems (MAS).

By Marco Aruta, Francesco Improta, Vadim Malvone, Aniello Murano, Vladana Perlic

llms agents fine-tuning

arXiv AI

Jun 30

Echoes of Human Malice in Agents: Benchmarking LLMs for Multi-Turn Online Harassment Attacks

arXiv:2510. 14207v3 Announce Type: replace Abstract: Large Language Model (LLM) agents are powering a growing share of interactive web applications, yet remain vulnerable to misuse and harm.

By Trilok Padhi, Pinxian Lu, Abdulkadir Erol, Tanmay Sutar, Gauri Sharma, Mina Sonmez, Munmun De Choudhury, Ugur Kursuncu

llms agents fine-tuning benchmarks safety

arXiv AI

Jun 30

Field Order Should Not Matter: Permutation-Invariant Embedding Model Fine-Tuning for Structured Metadata Retrieval

arXiv:2606. 30473v1 Announce Type: cross Abstract: We study retrieval over catalogs of structured metadata, where each record is a small schema whose fields answer different kinds of query.

By Aivin V. Solatorio, Olivier Dupriez, Rafael Macalaba

llms rag agents fine-tuning benchmarks

arXiv AI

Jun 30

Unlocking the Visual Record of Materials Science: A Large-Scale Multimodal Dataset from Scientific Literature

arXiv:2606. 29667v1 Announce Type: cross Abstract: The materials science literature encodes decades of experimental knowledge in figures, yet this visual record remains locked away and inaccessible to AI at scale.

By Subham Ghosh, Shubham Tiwari, Mohammad Ibrahim, Abhishek Tewari

llms fine-tuning multimodal safety

arXiv Machine Learning

Jun 30

BaRA: Bayesian Adaptive Rank Allocation for Parameter-Efficient Fine-Tuning

arXiv:2606. 29184v1 Announce Type: new Abstract: While Low-rank adaptation (LoRA) enables highly efficient fine-tuning by constraining task-specific updates to fixed low-rank subspaces, this rigid design limits representational flexibility and often results in overconfident predictions and miscalibrated uncertainty, especially in low-data regimes.

By Zhibin Duan, Yuhong Wang, Jiahong Fu, Zongsheng Yue, Bo Chen, Zongben Xu

fine-tuning benchmarks

arXiv AI

Jun 30

Domain Adaptation with Adaptive Imagination for Visual Reinforcement Learning under Limited Target Data

arXiv:2606. 30192v1 Announce Type: new Abstract: Sim-to-real transfer remains a major obstacle for reinforcement learning (RL), especially for vision-based control where image observations exacerbate the state-distribution shift between simulation and the real world.

By Hyunwoo Park, Sang-Hyun Lee

reinforcement-learning robotics fine-tuning

arXiv AI

Jun 30

Beyond Scaling Law: A Data-Efficient Distillation Framework for Reasoning

arXiv:2508. 09883v2 Announce Type: replace-cross Abstract: Large language models (LLMs) demonstrate remarkable reasoning capabilities in tasks such as algorithmic coding and mathematical problem-solving.

By Xiaojun Wu, Xiaoguang Jiang, Huiyang Li, Jucai Zhai, Dengfeng Liu, Qiaobo Hao, Huang Liu, Zhiguo Yang, Ji Xie, Ninglun Gu, Jin Yang, Kailai Zhang, Yelun Bao, Jun Wang

llms reinforcement-learning fine-tuning efficiency benchmarks

arXiv AI

Jun 30

SEATauBench: Adapting Tool-Agent-User Evaluation Into Low-Resource Southeast Asian Languages

arXiv:2606. 28715v1 Announce Type: cross Abstract: While AI development and evaluation for Southeast Asia (SEA) has grown rapidly, agent capabilities in regional languages are still poorly understood despite its importance to sovereign AI.

By My Chiffon Nguyen, Aulia Adila, Saksorn Ruangtanusak, Kittiphat Leesombatwathana, Vissuta Gunawan Lim, Patomporn Payoungkhamdee, Samuel Cahyawijaya

agents fine-tuning benchmarks

arXiv AI

Jun 30

PlantExpertVQA: A Visual Question Answering Dataset for Benchmarking Vision-Language Models in Plant Science

arXiv:2508. 17117v3 Announce Type: replace-cross Abstract: Existing plant-disease datasets target classification and detection, leaving vision-language models unable to support interactive, reasoning-based diagnosis.

By Syed Nazmus Sakib, Nafiul Haque, Mohammad Zabed Hossain, Shifat E. Arman

llms nlp fine-tuning multimodal benchmarks

Get the daily briefing