Fine-tuning and adaptation

LoRA, PEFT, instruction tuning and domain adaptation — adapting a pretrained model without paying to train one.

8,205 stories · RSS feed

arXiv Computation and Language
Sep 7

Uncertainty Signals for Network Intent Translation: Risk Ranking and Ambiguity Localization

The paper studies how to assess risk in intent-based networking by examining uncertainty in large language model (LLM) translations of high‑level intents into network configurations. It introduces two uncertainty signals: sampling‑based predictive uncertainty for ranking translations by risk, and token‑level entropy for locating ambiguous source tokens. Experiments on a Llama‑3.1‑8B‑Instruct model fine‑tuned for Juniper EX3300 show that predictive uncertainty can rank risky translations, while token entropy correlates with ambiguity from parameters and keywords.

By Ala' A. Alsamarneh, Omar Alhussein
arXiv Computation and Language
Sep 7

Trait-Aware Policy Optimization for Autoregressive Multi-Trait Essay Scoring

The paper introduces Trait-Aware Policy Optimization (TAPO), a post‑training framework for autoregressive models that score essays across multiple traits. TAPO decomposes rewards by sample and trait, integrating global consistency, trait accuracy, format validity, and inter‑trait dependencies, while enriching prompts with trait descriptions. Experiments on various backbone models show TAPO consistently outperforms supervised fine‑tuning and scalar‑reward baselines, proving its effectiveness and transferability for multi‑trait essay scoring.

By Zhengyang Wang, Sanwoo Lee, Jiaxin Wang, Chenxi Miao, Weikang Li, Yunfang Wu
arXiv Computer Vision
Sep 7

FailureSpot: Label-Efficient Timestamp-Level Failure Detection for Vision-Language-Action Models

The paper introduces FailureSpot, a label‑efficient method for detecting failures at the timestamp level in vision‑language‑action (VLA) policies. It first generates weak supervision from unlabeled VLA action chunks by identifying abnormal patterns, then employs active learning to annotate only the most uncertain trajectories. Experiments on multiple VLA policies demonstrate improved performance for both timestamp‑level and trajectory‑level failure detection.

By Jie Ma, Zongxi Liu, Yi Zhu
arXiv AI
Sep 7

An Empirical Study into Clustering of Unseen Datasets with Self-Supervised Encoders

The paper investigates whether pretrained image models can generalize to unseen datasets by clustering their embeddings. Using encoders trained only on ImageNet‑1k, both supervised and self‑supervised, the authors evaluate clustering performance on out‑of‑domain images. They find that supervised encoders perform better within the training domain, while self‑supervised encoders excel far outside it, and that fine‑tuning self‑supervised models reverses this trend. Additionally, the study shows that the silhouette score in UMAP‑reduced space correlates strongly with clustering accuracy, offering a proxy metric when labels are unavailable.

By Scott C. Lowe, Joakim Bruslund Haurum, Sageev Oore, Thomas B. Moeslund, Graham W. Taylor
arXiv Computer Vision
Sep 7

Measured Sliders: Learning Continuous Controls from Differentiable Image Measurements

Measured Sliders introduces a framework that defines continuous image controls via closed‑form differentiable measurements, enabling a unified measurement space for training and calibration. An observability test pre‑identifies usable supervision, while a measurement‑guided objective learns target movements and suppresses unwanted changes. The resulting controls, tested on SDXL and FLUX.1‑dev, are ordered, selective, and composable, achieving high monotonicity and selectivity across multiple attributes.

By Yijia Chen, Boyu Wei, Xuanhua Yin
arXiv Computation and Language
Sep 7

MedProb: Probing Internal Representations of Vision-Language Models for Medical Question Answering

MedProb is a lightweight probing framework that predicts multiple-choice medical visual question answering (Med‑VQA) answers directly from frozen vision‑language model (VLM) representations, avoiding free‑text generation. On datasets such as PATH‑VQA, SLAKE, and VQA‑RAD, MedProb extracts more answer‑relevant signal than prompting and outperforms both medical VLMs and agentic systems. The approach also narrows the performance gap between small and large models, shows that medical adaptation does not consistently improve linear decodability, and reveals positional biases in both prompting and generation.

By Erfan Nourbakhsh, Ke Yang, Anthony Rios
arXiv Computer Vision
Sep 7

Towards Neuro-Symbolic Procedural Reasoning for Long-Horizon Vision-Language-Action Manipulation

The paper proposes a neuro‑symbolic framework that augments vision‑language‑action (VLA) models with explicit task graphs and multimodal procedural memory to handle long‑horizon manipulation tasks. Task graphs encode action dependencies, valid transitions, and branch conditions, while memory tracks the active step, completed actions, textual context, and relevant visual evidence. Human demonstrations provide spatial and temporal guidance via gaze or saliency cues, which are annotated in robot‑view teleoperation videos and used to fine‑tune VLA models. The approach is evaluated on workspace clearing and surgical‑instrument handling tasks, measuring object and destination selection, subtask completion, task progress, step‑order consistency, overall success, and procedural or execution mistakes.

By Vivek Chavan, Yahuan Shi, Oliver Heimann, Kevin Haninger, J\"org Kr\"uger
arXiv Machine Learning
Sep 7

UniMate: One Unified Model to Animate Diverse Skeletons

UniMate is a unified foundation model that generates articulated motion for any skeleton from a rigged 3D asset and a text prompt, eliminating the need for test‑time optimization or per‑skeleton retraining. It uses a topology‑aware diffusion transformer that incorporates skeletal topology through graph‑aware attention bias, spectral rotary position embedding, and a global topological conditioner. Trained on the newly curated UniML3D dataset of 13,006 diverse motion sequences, UniMate outperforms existing baselines in quality, generalization, and efficiency, and supports zero‑shot cross‑topology transfer, in‑betweening, expansion, and text‑guided editing.

By Linzhan Mou, Jiahui Lei, Zhiyang Dou, Chenyue Cai, Chaoyue Song, Adam Finkelstein, Szymon Rusinkiewicz
arXiv AI
Sep 7

SciDocBench: A Workflow-Centered Benchmark and Data Pipeline for Scientific Document Understanding

SciDocBench is a workflow-centered benchmark for scientific document understanding that includes 124 expert-authored questions across seven capability groups and 19 subtasks in five scientific domains. Each question is evaluated under four conditions—English or Chinese, all-images-first or interleaved document representations—resulting in 496 evaluation instances. The benchmark is paired with SciDocIR, a typed evidence-graph representation, and SciDocDataset, a collection of 15K fine-tuning and 8K reinforcement-learning samples, forming an evaluation-to-training framework for scientific-document assistants.

By Shenxi Wu, Yuhong Liu, Haosong Zhang, Tongjin Zou, Yanxun Zhang, Gaochang Chen, Dun Liang, Jiaqi Wang, Zhecan James Wang, Yuhang Zang, Dahua Lin
arXiv Machine Learning
Sep 7

Mitra-v2 Technical Report

Mitra‑v2 is a tabular foundation model that achieves state‑of‑the‑art performance on a wide range of real‑world classification and regression tasks, including credit‑risk scoring, clinical prediction, equipment‑failure detection, and house‑price estimation. Trained solely on synthetic data with a larger and more diverse pretraining distribution than its predecessor, it uses a compact 2D Transformer backbone and improved optimization to handle longer contexts and larger feature spaces. On the TabArena and TALENT benchmarks, Mitra‑v2 outperforms leading models such as TabPFN‑3 and TabICLv2, matching the performance of a 1.6B‑parameter TabFM with only 77M parameters, and ranks first on multi‑class classification tasks with more than ten classes.

By Yefan Tao (Bernie), Xiyuan Zhang (Bernie), Xinyi Liu (Bernie), Boran Han (Bernie), Danielle Maddix (Bernie), Haoyang Fang (Bernie), Zhen Han (Bernie), Jiading Gai (Bernie), Xuanqing Liu (Bernie), Michael Bohlke-Schneider (Bernie), Yuyang (Bernie), Wang, Gerald Friedland, Kevan Mah, Chris Lee, Chris Kong
arXiv AI
Sep 7

SPD: Single Pass Decoding for Generative Reranking

The paper introduces SPD (Single Forward Pass), a decoding strategy that efficiently produces ranked lists from large language models by emitting only the ordinal positions of items. SPD extracts an item‑position score matrix from the model’s hidden states, then solves a bipartite assignment problem via the Hungarian algorithm to generate a valid permutation in a single forward pass. Experiments show that LoRA‑based fine‑tuning combined with autoregressive ranking distillation achieves 28 ms end‑to‑end inference, a 64× speed‑up while preserving ranking quality.

By Emil Laftchiev, Prachi Agrawal, Moe Kayali, Bixing Yan, Qi Xu, Zijie Lei, Chen Qiu, Zhi Hua, Ke Li, Luke Simon
arXiv Computation and Language
Sep 7

Multilingual Models for Check-Worthy Social Media Posts Detection

The paper reports a comprehensive study of transformer-based NLP models for detecting check-worthy social media posts, covering data collection, preprocessing, architecture selection, fine‑tuning, testing, and implementation. It focuses on multilingual models that can process English and low‑resource languages such as Arabic, Bulgarian, Dutch, Polish, Czech, and Slovak, and compares their performance to state‑of‑the‑art baselines. The work introduces multi‑label multilingual classifiers that simultaneously identify harmful content and posts containing verifiable factual claims efficiently.

By Sebastian Kula
arXiv AI
Sep 7

Phase Transition Frequency as a Training Time Predictor of Test Accuracy in ResNets

The study investigates whether the number of discrete class‑separability jumps (phase transitions) observed during ResNet fine‑tuning can predict final test accuracy. Across 75 experiments on four benchmarks (CIFAR‑10, CIFAR‑100, TinyImageNet, CIFAR‑10‑C) and three ResNet variants, a strong negative correlation is found on standard i.i.d. datasets (r = −0.84 on CIFAR‑10, r = −0.87 on CIFAR‑100), while the correlation weakens under distributional stress. Additional analyses show that the transition count retains predictive power after controlling for architecture depth and outperforms other training‑curve signals on in‑distribution benchmarks, though it is dominated by other signals on stressed datasets.

By Arunan J
arXiv Computer Vision
Sep 7

RefDiT: Local Attribute Guidance in Reference-Based Image Generation

RefDiT is a new framework for reference-guided image generation that addresses the shortcomings of previous methods when handling complex scenes with multiple objects. It introduces local region guidance by decomposing a single identifier token into attribute-level signals, allowing the model to learn correspondences between tokens and specific regions of a reference image. The approach incorporates a low-rank adapter (LoRA) within a diffusion transformer (DiT) to adjust the inference prompt based on user-provided guidance context, thereby enabling more precise local attribute control.

By Rameshwar Mishra, Srikrishna Karanam, A V Subramanyam
arXiv AI
Sep 7

Leveraging Low-Level Symbolic Competences for Unsupervised Grounding in Hallucination Detection

The paper explores using a language model’s low‑level symbolic skill—specifically SQL—to detect hallucinations without fine‑tuning. By having the model construct an SQL database from reference documents, it can reason over both the source and the model’s output, creating a neurosymbolic check. Experiments on RAGTruth and DiaHalu show this method outperforms direct prediction and rivals state‑of‑the‑art detectors, highlighting the value of leveraging inherent symbolic competences in LLMs.

By Renato Vukovic, Hsien-chin Lin, Carel van Niekerk, Benjamin Ruppik, Michael Heck, Shutong Feng, Nurul Lubis, Milica Gasic
arXiv AI
Sep 7

MCPO: Modality-Contrastive Preference Optimization for Multimodal Chain-of-Thought Compression

MCPO introduces a two-stage method to compress multimodal chain-of-thought reasoning. The first stage uses a step-level Normalized Cross-Modal Mutual Information pruning algorithm to remove visual-independent steps, reducing redundancy and hallucinations. The second stage fine-tunes the model with an asymmetric multimodal length-controlled preference loss, achieving up to 69.5% shorter reasoning chains and a 3.34× inference speedup while keeping accuracy.

By Guangheng Yang, Zhenliang Ni, Zhenkai Wu, Han Shu, Juan Feng, Wenming Yang, Jie Hu
arXiv AI
Sep 7

Adaptation Interfaces for In-Context Tabular Foundation Models in Time-to-Event Prediction

The paper explores how to adapt tabular foundation models (TabFMs) for censored time‑to‑event prediction by linking them with CoxPH and DeepHit and revising training procedures. It evaluates zero‑shot, classification‑based fine‑tuning, and survival‑head adaptations across 74 single‑risk and 4 competing‑risk datasets, finding that zero‑shot works best on small datasets while supervised adaptation excels as data grows. The study shows that the choice of adaptation interface and data regime critically influences TabFM transfer performance.

By Minh-Khoi Pham, Luca Cotugno, Dan Cernei, Alina Sirbu, Stefano Masi, Giuseppe Prencipe, Alessandro Pingitore, Patrizia Landi, Working Group on Uric Acid, Cardiovascular Risk of the Italian Society of Hypertension, Tai Tan Mai, Martin Crane, Marija Bezbradica
arXiv AI
Sep 7

Continual Field-Adaptive Models (CFAMs) for Post-Deployment Physical AI

arXiv:2609. 04552v1 Announce Type: cross Abstract: Unattended interactive autonomy - machines that step into danger in place of humans and complete tasks with human tools - remains a missing capability in mission-critical operations.

By Amarjot Singh, Tanmay R. Pancholi, Jainam Kothari, Shrirang Mahajan, Ketan Bansal, Zackory Erickson, Giuseppe Loianno, Alexandre M. Bayen, Jeff Schneider, Vince Nakayama
arXiv AI
Sep 7

Towards Understanding Pause Token Fine-Tuning Dynamics: A Mode Retention Perspective

The paper investigates how pause tokens affect fine‑tuning dynamics rather than just inference performance. Experiments on synthetic continual‑learning and math‑reasoning tasks show that masked pause tokens (MBP) reduce overwriting of learned modes and compress downstream‑step information. Across 1B‑8B Qwen and Llama models, MBP consistently improves reasoning scores while preserving language understanding, and the approach also benefits GRPO.

By Jaehyeon Kim, Suhwan Kim, Nakyung Lee, Yeongoon Kim, Jimin Seo, Giho Lee, Jungwoo Lee
arXiv AI
Sep 7

A Verifier-Guided Explainable Reasoning Framework with Gold-Anchored QLoRA, Task-Aware Mixture-of-Experts, and Group-Relative RLVR

The paper introduces a verifier‑guided explainable reasoning framework for educational question answering that integrates gold‑anchored QLoRA, a task‑aware symbolic router, and group‑relative RLVR. It adapts Qwen2.5‑3B‑Instruct with field‑weighted QLoRA supervision, routes logic problems to a FOL/Z3 verifier and physics problems to a symbolic solver, and uses verifier feedback for candidate evaluation, self‑revision, and reward construction. Experiments on 438 held‑out examples show that RLVR boosts reasoning depth (P3) from 50.68 % to 72.20 %, while symbolic verification improves answer reliability at the system level.

By Thi Kim Trang Vo, Nam Tien Le, Thi Kim Nguyet Vo, Minh Khang Tran, Duy Phuong Tran