Fine-tuning and adaptation

LoRA, PEFT, instruction tuning and domain adaptation — adapting a pretrained model without paying to train one.

8,205 stories · RSS feed

arXiv Computer Vision
Sep 10

VideoTIR: Accurate Understanding for Long Videos with Efficient Tool-Integrated Reasoning

VideoTIR introduces a reinforcement‑learning approach to improve long‑video understanding by encouraging multimodal large language models to use comprehensive multi‑level toolkits efficiently. It combines Zero‑RL and SFT cold‑starting strategies to help models retrieve and focus on meaningful video segments, images, and regions, thereby reducing hallucinations. The method includes Toolkit Action Grouped Policy Optimization (TAGPO) to streamline tool‑calling and a sandbox‑based trajectory synthesis framework for high‑quality data, achieving strong results on three long‑video QA benchmarks.

By Zhe Gao, Shiyu Shen, Taifeng Chai, Weinong Wang, Haotian Xu, Xing Wu, Wenbin Li, Qi Fan, Yang Gao, Dacheng Tao
arXiv Machine Learning
Sep 10

PatchFormer: A Patch-Based Time Series Foundation Model with Hierarchical Masked Reconstruction and Cross-Domain Transfer Learning for Zero-Shot Multi-Horizon Forecasting

arXiv:2601.20845v2 Announce Type: replace Abstract: Time series forecasting is a fundamental problem with applications in climate, energy, healthcare, and finance. Many existing approaches require do...

By Olaf Yunus Laitinen Imanov, Derya Umut Kulali, Taner Yilmaz
Hugging Face Trending Papers
Sep 8

Studying Image Tokenizers as Visual Languages in Unified Multimodal Models

The paper investigates image tokenizers as the visual language of unified multimodal models by creating a controlled autoregressive testbed that tracks task‑specific validation losses during multimodal continual pretraining across text, image, text‑to‑image, and image‑to‑text predictions. It shows that losses must be analyzed by task, that the loss–performance relationship varies with the token space, and that better reconstruction does not always lead to stronger downstream performance. The study also demonstrates how tokenizer design choices—such as discriminator use, semantic supervision, and vocabulary size—affect joint modeling and downstream results.

Hugging Face Trending Papers
Sep 8

Everything in Moderation: Per-Domain Coverage Optima and Alignment-Resistant Domain Gaps in Multi-Domain Mid-Training

The paper investigates how per-domain data composition during the mid‑training phase (between pre‑training and alignment) affects model performance. Experiments with Qwen3‑8B‑Base across five KOR‑Bench domains show that a moderate coverage band (10%‑40%) yields the best performance for each domain, and that alignment passes cannot fully close the gaps created by suboptimal mid‑training allocations. Additionally, zero coverage during mid‑training severely degrades accuracy, while a carefully tuned allocation can provide the largest overall pipeline improvement.

Hugging Face Trending Papers
Sep 8

SeGDeP: Semantic- and Geometric-Aware Decoupled Prompts for Reasoning Segmentation

SeGDeP introduces a decoupled prompt system that separates semantic identification from spatial grounding in reasoning segmentation. It uses a semantic prompt branch and an independent geometric projection path to produce semantic features and a DETR‑predicted box, which jointly condition a SAM mask decoder. The model achieves high cIoU and gIoU scores while adapting only a small fraction of Qwen3‑VL parameters via LoRA.

Hugging Face Trending Papers
Sep 8

Adaptive Anisotropic Attention for Axis-Structured Signals

The paper introduces Adaptive Anisotropic Attention (AAA), a method that splits self‑attention into temporal and spatial paths for axis‑structured signals like EEG. A learned gate combines the two paths for each token, and the resulting AXON model outperforms dense attention baselines on six EEG tasks and shows benefits in audio spectrogram experiments. The study demonstrates that aligning attention with natural signal axes provides a useful inductive bias.

Hugging Face Trending Papers
Sep 8

Which Forms of Caregiver Feedback Support Grammar Learning? A Reinforcement-Learning Study of Child-Like Language Models

The study investigates how different types of caregiver feedback influence grammar learning by training small GPT‑2‑style models on child‑directed language and fine‑tuning them with reinforcement learning. Four feedback categories—communicative, structural alignment, semantic contingency, and affective—were evaluated, with structural alignment showing the strongest improvement in grammaticality and communicative feedback yielding moderate gains. Semantic contingency and affective feedback did not enhance grammaticality, though they may aid other language learning aspects, indicating that various feedback forms contribute complementarily to language acquisition.

Hugging Face Trending Papers
Sep 8

FPicker: Topology-Guided Evolution for Filament Tracing in Low-SNR Microscopy

FPicker is a topology‑guided framework for filament tracing in low‑signal Cryo‑EM images. It combines a center‑endpoint representation with an open‑curve evolution module to model non‑cyclic connectivity, overcoming limitations of pixel‑wise segmenters, box‑based detectors, sequential trackers, and traditional active contours. On simulated benchmarks, FPicker improves mean spatio‑angular precision by over 40 % and reduces topological gap rates by more than 60 % under extreme noise, and it achieves state‑of‑the‑art performance on real EMPIAR data after fine‑tuning.

arXiv Computer Vision
Sep 7

RefDiT: Local Attribute Guidance in Reference-Based Image Generation

RefDiT is a new framework for reference-guided image generation that addresses the shortcomings of previous methods when handling complex scenes with multiple objects. It introduces local region guidance by decomposing a single identifier token into attribute-level signals, allowing the model to learn correspondences between tokens and specific regions of a reference image. The approach incorporates a low-rank adapter (LoRA) within a diffusion transformer (DiT) to adjust the inference prompt based on user-provided guidance context, thereby enabling more precise local attribute control.

By Rameshwar Mishra, Srikrishna Karanam, A V Subramanyam
arXiv Computation and Language
Sep 7

Uncertainty Signals for Network Intent Translation: Risk Ranking and Ambiguity Localization

The paper studies how to assess risk in intent-based networking by examining uncertainty in large language model (LLM) translations of high‑level intents into network configurations. It introduces two uncertainty signals: sampling‑based predictive uncertainty for ranking translations by risk, and token‑level entropy for locating ambiguous source tokens. Experiments on a Llama‑3.1‑8B‑Instruct model fine‑tuned for Juniper EX3300 show that predictive uncertainty can rank risky translations, while token entropy correlates with ambiguity from parameters and keywords.

By Ala' A. Alsamarneh, Omar Alhussein