Fine-tuning and adaptation

LoRA, PEFT, instruction tuning and domain adaptation — adapting a pretrained model without paying to train one.

8,205 stories · RSS feed

arXiv AI
Aug 25

Aslema at NADI 2026: Data Augmentation for Intent Recognition and Slot Filling

The paper introduces Aslema, a system for the NADI 2026 Shared Task 5, which includes intent recognition and slot filling. The authors evaluate four omni LLMs in zero‑shot and fine‑tuned settings, finding that fine‑tuning consistently outperforms zero‑shot inference. They further augment data by generating culturally grounded Tunisian Derja utterances with an LLM and synthetic speech via voice cloning, which improves performance; the final system based on Qwen3‑Omni‑30B achieves 86.8% intent accuracy and 34.7 WER on devtest, ranking 1st in slot filling and 4th in intent recognition on the official test set.

By Tajwaar Shafiq, Hunzalah Hassan Bhatti, Firoj Alam, Shammur Absar Chowdhury
arXiv Computation and Language
Aug 25

MIRROR: A Multi-Agent Framework with Iterative Adaptive Revision and Hierarchical Retrieval for Optimization Modeling in Operations Research

arXiv:2602.03318v4 Announce Type: replace Abstract: Operations Research (OR) relies on expert-driven modeling--a slow and fragile process ill-suited to novel scenarios. While large language models (L...

By Yifan Shi, Jiayi Wang, Minyi Wu, Ye Fan, Jialong Shi, Jianyong Sun
arXiv AI
Aug 25

DiaRelay: Relaying Dialogue Context with a Constant-Size Memory for Emotion Recognition in Conversation

DiaRelay introduces a lightweight adapter that lets large language models maintain a constant‑size dialogue‑level memory for emotion recognition in conversation. It builds on LoRA by adding a Selective Relay Memory Transition that aggregates useful historical evidence into a bounded memory, and a Dual‑axis Relay Memory Read that uses this memory to modulate low‑rank feature transformations. Experiments show DiaRelay achieves state‑of‑the‑art weighted F1 and accuracy on MELD with only 7.1 M additional trainable parameters, while also performing competitively on IEMOCAP.

By Zihao Zhou, Bin Yang, Jinghui Qin, Kebing Jin
arXiv Computation and Language
Aug 25

Safety-Aligned Weights Are Not Enough: Refusal-Teacher-Guided Finetuning Enhances Safety and Downstream Performance under Harmful Finetuning Attacks

arXiv:2506.07356v3 Announce Type: replace Abstract: While Finetuning-as-a-Service (FaaS) enables customization of Large Language Models (LLMs) using user data, this service is vulnerable to safety de...

By Seokil Ham, Yubin Choi, Yujin Yang, Seungju Cho, Younghun Kim, Changick Kim
arXiv Computer Vision
Aug 25

LiST: Local-Simplex Test-Time LoRA Fusion

arXiv:2608.22370v1 Announce Type: new Abstract: Task-specific LoRA adapters offer a modular way to specialize large language and vision-language models. However, existing adapter composition methods...

By Yihua Shao, Jia Li, Siyu Chen, Xinyu Luo, Yang Liu, Kecheng Chen, Xinwei Long, Lingyu Zhu, Fanhu Zeng, Maolin Wang, Ziyang Yan, Jingcai Guo, Hao Tang, Nicu Sebe, Zhenyi Wang
arXiv Computer Vision
Aug 25

Sa2VA: Marrying SAM2 with MLLM for Dense Grounded Understanding of Images and Videos

arXiv:2501.04001v4 Announce Type: replace Abstract: This work presents Sa2VA, the first comprehensive, unified model for dense grounded understanding of both images and videos. Unlike existing multi-...

By Haobo Yuan, Xiangtai Li, Tao Zhang, Yueyi Sun, Zilong Huang, Shilin Xu, Shunping Ji, Yunhai Tong, Lu Qi, Jiashi Feng, Ming-Hsuan Yang