Fine-tuning and adaptation

LoRA, PEFT, instruction tuning and domain adaptation — adapting a pretrained model without paying to train one.

8,205 stories · RSS feed

arXiv AI
Sep 17

BadQubits: An LLM-Based Framework for Static Pre-Execution Detection of Structurally Harmful Quantum Circuits

BadQubits is an LLM-based framework that statically analyzes OpenQASM 2.0 circuits before execution to detect structurally harmful patterns. The system evaluates four large language model architectures on 1,500 circuits, achieving 92.67% classification accuracy and 96.1% recall for harmful circuits with a fine‑tuned Qwen Coder 2.5 7B model. Comparative experiments show that LLMs retain sequential token structure, outperforming a bag‑of‑gates CNN, and that model decisions correlate with threat‑defining features such as SWAP density and measurement timing.

By Justin Woodring, Lamine Noureddine, Aisha Ali-Gombe
arXiv AI
Sep 17

AuthorMix: Modular Authorship Style Transfer via Layer-wise Adapter Mixing

AuthorMix is a lightweight, modular framework for authorship style transfer that uses layer‑wise adapter mixing. It trains individual style‑specific LoRA adapters on a small set of high‑resource authors, enabling rapid adaptation to new target styles with only a few examples. The method achieves the highest combined style‑meaning score among baselines, including GPT‑5.1, and improves meaning preservation, as confirmed by human evaluation.

By Sarubi Thillainathan, Ji-Ung Lee, Michael Sullivan, Alexander Koller
arXiv AI
Sep 17

CapMap-MS-TTA: 3rd Place Solution for the MUMU Track of the 8th LSVOS Challenge at ECCV 2026

The paper presents CapMap-MS-TTA, a unified multimodal model that achieved third place in the MUMU track of the 8th Large-scale Video Object Segmentation Challenge. The solution tackles image tagging, open‑vocabulary object detection, and English captioning under strict resource limits, using an expanded keyword lexicon and lightweight expand‑hints for tagging, and Florence‑2 with multi‑scale and horizontal‑flip TTA plus label‑aware NMS for detection. The approach improves the reproduced Florence‑2 baseline from 15.16 to a best public score of 16.4815 without fine‑tuning.

By Chengfeng Qiu, Kaifeng Wei
arXiv Computation and Language
Sep 17

ScienceIDE: Turning World's Scientific Codebase into Agent Learnable Environments

arXiv:2609.19134v1 Announce Type: new Abstract: Scientific code repositories encode decades of human knowledge in executable models, methods, and tools. Yet fragmented toolchains, implicit domain con...

By Hejia Geng, Zesen Huang, Haoyang Li, Wenbin Li, Koutian Wu, Zihan Zhou, Yuanbo Pang, Weihao Liu, Zigong Xu, Zhiping Li, Zongzheng Zhang, Chuanfei Dong, Jiankai Sun, Tianzhe Zheng, Fengyu Xie, Yue Ma, Yueheng Shi, Tong Xie, Zonglin Di, Xianrong Liu, Qucheng Gao, Yimin Liu, Jiaming Pan, Sheng Huang, Xiao-Han Ma, Lanqing Yuan, Zhenlin Zhu, Ziang Liu, Ziyang Xu, Junkai Wang, Kangkai Liang, Jiayi Xian, Zehong Zhao, Liuwei Xu, Jingxu Xie, Peijin Zhang, Qiang Gao, Chengyi Xing, Zhe Zhao, Xi Wang, Yaopeng Xing, Xing Meng, Zhenfei Yin, Yingcheng Wu, Ling Yang
arXiv Computer Vision
Sep 17

AgenTeeth: A Model-Agnostic Framework for Suppressing Hallucination in Frozen Vision-Language Models on Dental X-Rays via Tool Evidence Injection

arXiv:2609.17800v1 Announce Type: new Abstract: Vision-language models (VLMs) remain largely unreliable on panoramic dental radiographs and can rely on learned anatomical priors rather than evidence...

By Ahmed Rafid, Fariya Ahmed, Rumman Adib, Mehedi Ahamed, Ajwad Abrar, Tareque Mohmud Chowdhury
arXiv Machine Learning
Sep 17

Instrument Classification of Solo Sheet Music Images

This paper investigates instrument classification using solo sheet music images rather than audio. It converts images into sequences of musical words via bootleg score representation and treats the task as text classification, training AWD‑LSTM, GPT‑2, and RoBERTa models on IMSLP data for eight instruments. Pretraining on unlabeled data and fine‑tuning improves RoBERTa’s accuracy from 34.5% to 42.9%, and two proposed data‑augmentation methods raise accuracy by an additional 15%.

By Kevin Ji, Daniel Yang, TJ Tsai
arXiv Computation and Language
Sep 17

Correlation-Guided Encoder Selection for Multi-Encoder Large Audio-Language Models

The paper introduces CUES, a lightweight heuristic for selecting encoder combinations in large audio‑language models by estimating complementarity through Pearson correlations of single‑encoder performance profiles. Using a frozen SmolLM2‑135M backbone, CUES consistently identifies optimal encoder sets for each track on the XARES‑LLM benchmark without requiring fusion training or test data. On broad audio tasks, CUES selects a diverse trio of encoders, improving performance by 4.3% over Whisper‑medium, while on text generation it opts for a focused speech‑only pair, outperforming mHuBERT‑147 by 6.3%. The results illustrate how correlation signals guide a diversity–interference trade‑off across different task families.

By Pei-Jun Liao, Hung-Shin Lee, Wenze Ren, Kuo-Hsuan Hung, Hung-yi Lee, Hsin-Min Wang
arXiv AI
Sep 17

Rethinking Domain Specialization for Open-Ended Scientific Reasoning in Astronomy Language Models

The paper investigates whether domain-specific fine‑tuning benefits open‑ended scientific reasoning in astronomy. Using a curated 300‑question QA benchmark from 2017–2026 Olympiad‑style materials, the authors compare open‑weight, API‑served general‑purpose, multimodal, and astronomy‑specialized language models. Results show that strong general‑purpose models set the highest correctness baseline, but variations in metric agreement, judge sensitivity, benchmark composition, and modality suggest that domain specialization is task‑ and deployment‑dependent and that domain‑specific evaluation is crucial for scientific workflows.

By Vanessa Lama, Sanjay Das, Emily Herron, Yuan-Sen Ting, Tijmen de Haan, Junqi Yin, Tirthankar Ghosal, Feiyi Wang
arXiv AI
Sep 17

FairCompressAgent: An Agentic Framework for Fairness-Aware Model Compression for FPGA Deployment

FairCompressAgent (FCA) is an agentic framework that unifies fairness-aware pruning, incremental quantization, and sparse low‑rank factorization for FPGA deployment. A language‑model planner selects compression configurations based on model profiles and measured outcomes, while an execution layer handles compression, fine‑tuning, evaluation, and constraint‑based selection. Experiments on Fitzpatrick‑17k with VGG‑11 show FCA can reduce inference tensor storage by 59.54% under accuracy constraints, improve validation average precision, and lower equalized opportunity, achieving similar results to one‑shot planning with fewer candidate evaluations.

By Yuanbo Guo, Yiyu Shi
arXiv AI
Sep 17

EDCT-Bench: Uncovering Faithfulness Gaps in VLMs via Explanation-Driven Counterfactual Testing

EDCT-Bench is a benchmark that evaluates the faithfulness of Vision‑Language Models (VLMs) by using Explanation‑Driven Counterfactual Testing (EDCT). EDCT extracts visual concepts from a model’s natural language explanation, applies minimal verified edits to those concepts, and checks whether the model’s answer and explanation remain consistent with the edited image. The benchmark covers three domains—knowledge‑intensive VQA, safety‑critical driving, and 3D spatial reasoning—and reveals significant faithfulness gaps in current VLMs, while also showing that EDCT‑generated counterfactuals can improve training.

By Sihao Ding, Santosh Vasa, Aditi Ramadwar, Thomas Monninger
arXiv AI
Sep 17

Knowledge-Graph Based Augmentation versus Retrieval Augmented Generation for Cultural-Related Question Answering

The paper compares Knowledge-Graph Based Augmentation (Graph-RAG) with Retrieval-Augmented Generation (RAG) for answering culturally specific questions. Using the LatamQA dataset, Graph-RAG, built automatically from Wikipedia via KGGen, matches RAG performance and reduces the base LLM’s error by 72% with a standard KG and 78% with a benchmark-aware variant. The approach also transfers zero‑shot to Portuguese, showing multilingual applicability.

By Pablo Poulenard, Yannis Karmim, Valentin Barri\`ere
arXiv Machine Learning
Sep 17

The Unbearable Weight: Scaling Models and Methods for UAV Audio Classification

The paper investigates how to balance model size and fine‑tuning strategy for UAV audio classification. Using a dataset of 3,100 clips across 31 drone classes, it compares transformer and convolutional backbones under full fine‑tuning, classifier‑only fine‑tuning, and four parameter‑efficient fine‑tuning methods. Results show that selective batch‑norm tuning of EfficientNet‑B7 yields the best accuracy (97.65%) while updating less than 0.5% of parameters, and that lightweight CNNs generally outperform transformers in both accuracy and efficiency.

By Andrew P. Berg, Qian Zhang, Mia Y. Wang
arXiv Computer Vision
Sep 17

GOLF: Global Observation with Local Focus for Calibration-Aware Stereo Interaction Field Estimation

GOLF is a method that won the SHOW3D Interaction Field Estimation Challenge at HANDS@ECCV 2026. It predicts a 3D vector from each of 21 hand joints to the nearest point on a manipulated object using synchronized egocentric stereo views. The approach fuses dense global context, locally sampled hand/object evidence, and Plücker‑ray geometry, and employs a DINOv3 ViT‑H+/16 backbone with LoRA and trainable LayerNorm, achieving top scores on the hidden test set.

By Minqiang Zou, Riqiang Jin, Zhi Lv, Dong Luo, Lianghai Tian, Zhenyu Zhao, Qi Xu, Tong Wu, Mochen Yu, Yao Tang