Fine-tuning and adaptation

LoRA, PEFT, instruction tuning and domain adaptation — adapting a pretrained model without paying to train one.

8,205 stories · RSS feed

arXiv AI
Aug 19

EXPO-FT: Sample-Efficient Reinforcement Learning Finetuning for Vision-Language-Action Models

EXPO-FT is a system that enables stable, sample‑efficient reinforcement learning fine‑tuning of pretrained Vision‑Language‑Action (VLA) policies. It achieves perfect success on a range of manipulation tasks—such as routing string lights, striking a pool ball, and inserting a flower into a wine bottle—using only about 19.1 minutes of online robot data. The approach outperforms both RL-from-scratch and existing VLA fine‑tuning methods, and the authors provide an open‑source codebase to support wider adoption.

By Perry Dong, Kuo-Han Hung, Tian Gao, Dorsa Sadigh, Chelsea Finn
arXiv AI
Aug 19

DiSCO: Defending text-to-image generation through distribution-guided contrastive prompt optimization

DiSCO is a zero‑shot, black‑box defense for text‑to‑image models that operates solely at the prompt level. It expands prompts with a distribution‑guided suffix using beam search and contrastive scoring against safe and unsafe image pools generated by the target model, iteratively refining until safe content is produced. The method improves safety on the I2P benchmark under various red‑teaming attacks, reducing attack success rates by 37.7% and 25.13% while preserving semantic fidelity and image coherence.

By Tong Zhang, Motasem Alfarra, Carlos Hinojosa, Christos Louizos, Bernard Ghanem
arXiv AI
Aug 19

Q-Learning With World Models

The paper introduces QWM, a framework that integrates world models with standard Q‑learning to perform test‑time search over imagined trajectories. By training the policy and value function solely on real transitions, QWM avoids compounding model bias while still benefiting from predictive search. Experiments on the Robomimic and LIBERO manipulation benchmarks show that QWM outperforms strong prior state‑of‑the‑art methods in both sample efficiency and performance.

By Perry Dong, Yueru Jia, Chelsea Finn, Dorsa Sadigh
arXiv Machine Learning
Aug 19

Thinking in a Low-Resource Language: What SFT Builds, What RL Fixes, What Accuracy Cannot See

The paper evaluates how three large mixture‑of‑experts models (Alibaba, OpenAI, NVIDIA) can be fine‑tuned to reason in a low‑resource language, specifically Greek. Accuracy metrics show little change, but the authors uncover significant qualitative improvements: after supervised fine‑tuning, models reason in Greek on ~98% of items, with better grammaticality and retained general ability. Reinforcement learning with pre‑registered rewards further eliminates reasoning‑channel leaks and format skips, while the Greek‑reasoning habit remains robust to an accuracy‑only gradient.

By Ayoub Kirouane, Christos Petrocheilos
arXiv AI
Aug 19

The 10th AI City Challenge

The 10th AI City Challenge, held alongside ECCV 2026, celebrates a decade of benchmarking for intelligent transportation, smart cities, and physical AI. Since its 2017 inception focused on vehicle detection, classification, and tracking, the challenge has expanded into a comprehensive benchmark suite covering multi‑camera perception, multimodal reasoning, synthetic‑to‑real learning, generative forecasting, and privacy‑preserving evaluation. The 2026 edition saw 325 registered teams from 26 countries, with six main tracks—spanning multi‑camera 3D perception, transportation safety captioning and VQA, traffic anomaly reasoning, text‑based person anomaly search, generative traffic video forecasting, and cross‑city object detection—plus two out‑of‑domain leaderboards for fisheye traffic‑violation understanding and pedestrian situated‑intent VQA.

By Zheng Tang, Shuo Wang, David C. Anastasiu, Ming-Ching Chang, Anuj Sharma, Quan Kong, Munkhjargal Gochoo, Jun-Wei Hsieh, Tomasz Kornuta, Zhedong Zheng, Renran Tian, Judah Goldfeder, Fulgencio Navarro, Yuxing Wang, Yizhou Wang, Sameer Satish Pusegaonkar, Anqi Li, Nalin Dadhich, Ridham Kachhadiya, Dhanishtha Patil, Haoquan Liang, Jiajun Li, Han Zhang, Yilin Zhao, Zaid Pervaiz Bhat, Shuyu Yang, Ashutosh Kumar, Rong Wang, Rafael Martin Nieto, Peter Christiansen, Ahmed Abduljawad, Mohanrasu Shanmugam, Nadeem Shaik, Sujit Biswas, Xunlei Wu, Vidya Murali, Rama Chellappa
arXiv AI
Aug 19

PTXBench: Benchmark and Adapt LLMs for GPU Kernel Optimization with Architecture-specific PTX

PTXBench is a benchmark designed to evaluate and adapt large language models (LLMs) for GPU kernel optimization using architecture‑specific PTX code. It assesses functional correctness, runtime execution of target instructions, and speedup over state‑of‑the‑art libraries on GEMM and attention workloads on H100 and B200 GPUs. The study finds uneven success rates, especially on complex attention backward tasks, and shows that executing target instructions does not guarantee competitive performance, with no model consistently outperforming frontier libraries. The authors also fine‑tune Qwen3.6‑27B, noting that repair‑conditioned training improves some tasks but generalization remains inconsistent, highlighting the importance of data coverage, balance, and teacher quality.

By Genghan Zhang, Yixin Dong, Chengze Fan, Zhichen Zeng, Yueming Yuan, Shaowei Zhu, Kunle Olukotun
arXiv Machine Learning
Aug 19

A Data-Efficient Analytical Prior Machine Learning Framework for Sound Reduction Frequency Prediction in Helmholtz Resonators

The paper presents a data‑efficient machine learning framework that incorporates a low‑cost analytical model to enhance predictions of sound‑reduction frequencies in Helmholtz resonators. Two strategies are explored: (1) using the analytical model as a baseline and learning only the discrepancy with limited high‑fidelity simulation data, and (2) distilling the analytical mapping into a learned prior and calibrating it with scarce simulation data. Experiments on rectangular side‑branch resonators show that both approaches significantly reduce prediction error compared to direct learning, achieving mean absolute errors as low as 0.371 Hz with full‑model fine‑tuning.

By Jiaming Li
arXiv AI
Aug 19

Structural Plan-to-Model Conversion with Deterministic Geometry and Guarded Agentic Vision-Language Refinement

The paper introduces a novel framework that converts structural framing plans from PDF drawings into editable finite‑element model drafts without requiring task‑specific detector training. It combines a deterministic geometry extraction stage—estimating scale, recognizing five entity classes, and assembling a drafting grammar—with an agentic vision‑language refinement stage that proposes corrections, performs admission tests, and ensures fail‑closed transactions. Evaluation on a 100‑plan benchmark shows high accuracy, with scale within 0.1% and recall/precision values above 0.86 for all component types.

By Mohammad Talebi-Kalaleh, Qipei Mei
arXiv Machine Learning
Aug 19

Quantifying Memorization and Privacy Risks in Genomic Language Models

The paper introduces a comprehensive privacy evaluation framework for genomic language models (GLMs) that quantifies memorization risks using perplexity-based detection, canary sequence extraction, and membership inference. By planting canary sequences at different repetition rates in synthetic and real datasets, the authors systematically assess how repetition, model capacity, and training dynamics affect memorization across various GLM architectures. The study demonstrates that GLMs do memorize training data to varying degrees and that no single attack method fully captures this risk, highlighting the necessity of multi-vector privacy auditing for genomic AI systems.

By Alexander Nemecek, Wenbiao Li, Xiaoqian Jiang, Jaideep Vaidya, Erman Ayday
arXiv Machine Learning
Aug 19

AppendiGrade: An XAI-Enhanced Deep Learning Framework for Grading Appendicitis in Ultrasound with Gaussian Blur and Grad-CAM

The paper presents AppendiGrade, an XAI‑enhanced deep learning framework that automatically detects complicated appendicitis from ultrasound images. Using a dataset of 4,679 images across five classes, the authors trained four pretrained models and achieved a best accuracy of 95.58% with InceptionV3 after applying preprocessing, hyperparameter tuning, and image sharpening. Grad‑CAM heatmaps were generated to explain the model’s predictions, facilitating easier expert cross‑checking.

By Fahad Ahammed, Omar Faruq Shikdar, Navid Zaman, Md Tahsin, Md. Nawab Yousuf Ali, Golam Sorwar
arXiv AI
Aug 19

Training with synthetic data for drone detection in thermal imagery

The paper explores a synthetic-first training approach for detecting drones in medium- and long-wave infrared imagery, combining synthetic scene generation with fine-tuning on real data. It demonstrates that synthetic data can establish initial object representations, but real infrared data is crucial to close domain gaps and improve reliability. The study finds that aligning datasets has a greater impact on performance than increasing model size, and that semantic alignment in feature space is the strongest predictor of success, with radiometric factors like entropy and dynamic range also contributing.

By Tanel Liiv, Sander Soodla, Nzamba Bignoumba, Alma M. Liezenga, Toomas Pruuden
arXiv Machine Learning
Aug 19

VLCP: Vision Language Control Policy Closed-Loop Code Replanning for Robot Manipulation

VLCP (Vision Language Control Policy) is a training‑free robot manipulation approach that keeps a vision‑language model (VLM) frozen and uses it to generate short Python control functions. Unlike traditional methods that retry a fixed policy, VLCP rewrites the control code every K steps based on multi‑view RGB, proprioceptive state, and state delta, allowing failures to be corrected within the same episode. In a 57‑task MuJoCo/RoboVerse benchmark, VLCP achieves 35.1% pooled success versus 3.5% for a single‑query baseline, with a 27.3% within‑episode recovery rate on failed grasps and efficient token usage.

By Dhia Naouali, Minghan Wu, Claudia Wong, Abhinav Puthran, Omar G. Younis
arXiv Machine Learning
Aug 19

Temporal Leakage in Financial News NLP: A Multi-Architecture Audit with a Regime-Specific M&A Signal

The paper audits the impact of temporal leakage on financial-news direction prediction across 49,799 articles and 16 feature-model combinations, including TF‑IDF, MiniLM, FinBERT, and fine‑tuned RoBERTa‑large / DeBERTa‑v3‑large, as well as zero/few‑shot and LoRA probes of Llama‑3 and Qwen2.5. Random train‑test splits inflate MCC scores by 1.1× to 6.5×, with larger models and richer features showing greater gains, while end‑to‑end FinBERT fine‑tuning actually increases the gap. Only the mergers and acquisitions (M&A) category shows a positive locked‑test signal under near‑temporal chronological evaluation, with the signal localized to 2024‑2025 European‑tilted M&A semantics and not transferring to a 2009‑2020 U.S. corpus.

By Chenhao Xue, Raslen Guesmi, Siwei Feng, Yucheng Gong, Jacob Xavier Sundram, Jordan Pang, Lan Wang, Julian Kaljuvee
arXiv AI
Aug 19

Can Large Language Models Explain Flight Safety Events? A Prior-Guided Semantic LLM-based Approach

The paper introduces FlightLLM, a prior-guided semantic approach that uses large language models to explain flight safety events. It tackles challenges such as modal inconsistency, limited classification ability, and scarce domain data by combining feature engineering, semantic discretization, a CatBoost statistical expert, contrastive few-shot learning, and structured prompts. Evaluated on 704 real‑world A320 flights, FlightLLM achieves competitive classification and produces clear, aviation‑specific explanations for hard landing events.

By Lu Xu, Xu Li, Linjiang Zheng, Fan Li, Riquan Zhang, Jiaxing Shang
arXiv Machine Learning
Aug 19

Agents unlock new capabilities through Switching LoRA Adapters as a Tool (SLAaaT)

The paper introduces Switching LoRA Adapters as a Tool (SLAaaT), a method that lets agents dynamically switch between specialized LoRA adapters during a trajectory. By applying this to two synthetic coding tasks, the authors show that agents can solve problems they previously failed, autonomously select strategies that outperform a human heuristic, and reduce the capability tax by up to 18× compared to using a single adapter. SLAaaT also outperforms spawning subagents in both task performance and token efficiency.

By Kenneth Ge
arXiv Machine Learning
Aug 19

Debate Training Reduces Reward Hacking in RLAIF

The paper shows that fine‑tuning a large language model (LLM) with a debate framework—where a generator and a critic compete and a weaker LLM judge adjudicates—reduces reward hacking compared to standard reinforcement learning from AI feedback (RLAIF). In experiments on mathematics tasks, the debate approach keeps the judge’s performance stable, achieving a 45% higher peak validation accuracy than the RLAIF baseline and mitigating the rapid exploitation of judge errors. Additional findings indicate that weakening the judge speeds hacking unless countered by extra debate rounds, that debate can override misalignment prompts, and that word‑limit constraints on critiques help balance the game and prevent judge hacking. whyItMatters":"The study demonstrates a practical method to curb reward hacking in RL‑based AI systems, addressing a key obstacle for safely scaling AI oversight."

By Zachary Kenton, Lili Janzer, Rory Greig, Tian Huey Teh, Kirill Tyshchuk, Jonah Brown-Cohen, Harri Edwards, Senthooran Rajamanoharan, Noah Y. Siegel, Natasha Jaques, Rohin Shah
arXiv AI
Aug 19

Comparative Study of Out-of-the-Box Technology for Automatic Target Detection and Recognition

The paper evaluates out‑of‑the‑box object detection models for automatic target detection and recognition (ATD/R) in military settings. Six YOLO variants and two DETR variants were benchmarked on a new military dataset featuring vehicles, occlusions, and small targets, with performance measured in mAP@0.5 and mAP@0.5:0.95 across air‑to‑ground and ground‑to‑ground perspectives. Findings show larger models and DETR-based approaches perform best, fine‑tuning on the VisDrone dataset improves air‑to‑ground and small‑object performance, yet all models still struggle with small targets in air‑to‑ground scenarios.

By Alma M. Liezenga, Lotte Nijskens, Henrik R. Baumann, Stefan Becker, Simon Bensberg, Niccol\`o Camarlinghi, H{\aa}vard R. Eiring, Alexander W. Johnsgaard, Tanel Liiv, Giuseppe Martino, Matteo Marturini, Matthias Rapp, Jan Erik van Woerden, Alexander Wolpert, Hugo J. Kuijf
arXiv AI
Aug 19

SignalReasoner: Assessing the Upper Bound of 3B Models for Signal Mathematical Reasoning

SignalReasoner evaluates reinforcement fine‑tuning methods for the Qwen2.5‑3B‑Base language model on graduate‑level signal processing math problems from the WirelessMATHBench‑XL benchmark. Two training paradigms are compared: direct reinforcement learning with verifiable rewards, and supervised chain‑of‑thought fine‑tuning followed by domain‑specific reinforcement learning. The study benchmarks three policy optimization algorithms—GRPO, GSPO, and GMPO—to determine if domain‑aware chain‑of‑thought initialization improves performance, achieving a best accuracy of 39.12% versus 12.37% for the untrained base model.

By Guozheng Sun
arXiv Machine Learning
Aug 19

Hierarchical Data Selection via Manifold Coverage and Sparse Feature Coverage in LLM Post-training

The paper introduces MASS, a hierarchical data selection method that first groups data using low‑dimensional manifold coordinates learned by a dense autoencoder, then applies a TopK sparse autoencoder for quality‑aware feature coverage within each group. This approach addresses the shortcomings of traditional diversity metrics that mix semantic, supervisory, and noise signals. Experiments on Vision Flan and LLaVA‑CoT demonstrate that MASS outperforms existing baselines across various budgets and can match or exceed full‑data training with only a small subset.

By Peng Sun, Yi Yang, Antong Zhang, Chunxiao Li, Yanbo Wang, Dianbo Liu, xin chen, Kai Yu, Lu Chen, Tianfan Fu