Fine-tuning and adaptation

LoRA, PEFT, instruction tuning and domain adaptation — adapting a pretrained model without paying to train one.

8,205 stories · RSS feed

Hugging Face Trending Papers

Sep 10

SpecGuard: Inference-Time Backdoor Detection For Free

SpecGuard is an inference‑time backdoor detector that leverages speculative decoding—using a draft model to propose tokens and a target model to verify them—without adding extra model computation. By monitoring the draft‑token acceptance rate, SpecGuard identifies when a target model’s behavior shifts toward attacker‑controlled outputs, a signal that appears whenever a backdoor is triggered. The method works across various backdoor types and model families, reliably detecting stealthy attacks that bypass input‑level filters while avoiding the extra generation cost of existing runtime detectors.

llms fine-tuning

Hugging Face Trending Papers

Sep 10

The Eloquence submission for Task 2 of the Interspeech 2026 MLC-SLM challenge

The Eloquence team presents three strategies for the Interspeech 2026 MLC‑SLM Task 2, a multilingual MCQA challenge across 21 languages. They fine‑tune Voxtral‑Mini‑3B with LoRA and cross‑lingual augmentations, achieving 0.72 macro‑accuracy; they use multimodal in‑context learning on the frozen Voxtral‑24B to correct label bias, reaching 0.81; and they deploy a training‑free retrieval system with a voice‑anchored memory, scoring 0.68. All approaches surpass the official baseline.

llms nlp fine-tuning multimodal safety

Hugging Face Trending Papers

Sep 10

Beyond Visual Quality: Evaluating Physical Consistency under Ego-Motion with EgoGenEval

The paper introduces EgoGenEval, a benchmark that assesses visual generators’ physical consistency during ego‑motion by measuring Camera Motion Grounding and Scene State Preservation across 1,400 cases and 2,360 target views. It shows that current pose‑free models struggle to maintain both camera motion and scene state, and that pairwise supervision in training does not simultaneously improve these aspects. The authors suggest a trajectory‑centric approach that couples self‑conditioned rollouts with explicit pose and visibility supervision to address these limitations.

robotics fine-tuning benchmarks

Hugging Face Trending Papers

Sep 10

Rubric-Aligned Disentangled Evaluation of Human Simultaneous Interpreting

The paper introduces a new corpus of 1,101 human simultaneous interpreting segments, each scored on meaning transfer, delivery quality, and perceived latency. It demonstrates that standard large‑language‑model prompting and scalar supervision fail to align with these rubric dimensions, producing near‑zero correlation with human ratings. By adding dual regression heads to a LoRA‑adapted COMET‑KIWI encoder, the authors achieve modest Pearson correlations (0.388 for meaning transfer and 0.301 for delivery quality) on a held‑out test set, improving over the frozen baseline.

llms fine-tuning

arXiv Computer Vision

Sep 10

SceneHI: High-Resolution 3D-Consistent Scene Texturing with Controllable Illumination

arXiv:2609.10363v1 Announce Type: new Abstract: SceneHI is a framework that lifts high-resolution, illumination-aware priors from 2D diffusion models to perform 3D texture synthesis. It is the first...

By Athanasios Tragakis, Marco Aversa, Daniela Ivanova, Chaitanya Kaul, Roderick Murray-Smith, Daniele Faccio, Paul Henderson

diffusion fine-tuning

arXiv AI

Sep 10

NormViz: A Benchmark and Framework for Grounding Multimodal Reasoning in Global Cultures

NormViz introduces a new benchmark, NormViz‑Bench, comprising 3,268 contrastive image pairs from 16 countries that test AI’s ability to recognize culturally relevant visual norms. Each pair differs only in a behavior that changes its cultural interpretation, and images are labeled as conforming, violating, or irrelevant to local norms, requiring both images to be correctly classified. The benchmark shows current VLMs perform poorly, and a complementary training set, NormViz‑Train, offers a path to improve performance by teaching models to link visual perception with cultural significance.

By Akhila Yerukola, Fabrice Y Harel-Canada, Simran Khanuja, Abhinav Sukumar Rao, Ashima Suvarna, Nanyun Peng, Saadia Gabriel, Maarten Sap

llms fine-tuning multimodal benchmarks

arXiv Computer Vision

Sep 10

How (Mis)calibrated is your Federated CLIP and what to do about it?

arXiv:2512.04305v3 Announce Type: replace Abstract: Vision-language models (VLMs) such as CLIP are increasingly adapted across decentralized data silos, yet the reliability of their predictions under...

By Mainak Singha, Masih Aminbeidokhti, Paolo Casari, Gianni Franchi, Elisa Ricci, Subhankar Roy

llms fine-tuning multimodal

arXiv Computer Vision

Sep 10

LeCor: Learning to Be Corrected by Meta-Learned Test-Time Training for Interactive 3D Lung-Tumour Segmentation

arXiv:2609.09477v1 Announce Type: new Abstract: Delineating lung tumours on computed tomography (CT) takes a considerable share of the time spent on radiotherapy planning, and a contour proposed by a...

By Yi Luo, Yike Guo, Wenxuan Li, Zongwei Zhou, Rui Zhang, Kai Ding

computer-vision fine-tuning

arXiv AI

Sep 10

Bridging the Semantic-Utility Gap in Multimodal RAG via Generator-in-the-Loop Alignment

arXiv:2609.08188v1 Announce Type: new Abstract: Vision-language models (VLMs) augmented with retrieval-augmented generation (RAG) benefit from access to external evidence. However, standard retriever...

By Zhan-Lun Chang, Dong-Jun Han, Seyyedali Hosseinalipour, Mung Chiang, Christopher G. Brinton

llms rag reinforcement-learning fine-tuning multimodal safety

arXiv AI

Sep 10

Eliciting Self-Verification in Multimodal Reasoning Agents with Reinforcement Learning

arXiv:2609.08025v1 Announce Type: new Abstract: Reasoning agents increasingly rely on external tools such as web search to answer complex queries. Reinforcement learning (RL) finetuning algorithms su...

By Vishwas Sathish, Viresh Ranjan, Xinliang Zhu, Arnab Dhua, Douglas Gray

llms agents nlp reinforcement-learning fine-tuning multimodal benchmarks

arXiv Computer Vision

Sep 10

Arti-JEPA: Adapting Video World Model to Real-Time MRI of the Vocal Tract for Speech-Production Analysis

arXiv:2609.09757v1 Announce Type: cross Abstract: Real-time MRI (rtMRI) captures the dynamics of the entire vocal tract during speech, but labeled data are scarce and the modality - single-slice, gra...

By Hong Nguyen, Sean Foley, Christina Hagedorn, Yijing Lu, Sudarsana Reddy Kadiri, Dani Byrd, Shrikanth Narayanan

rag fine-tuning

arXiv Computer Vision

Sep 10

Guiding Image-to-3D Generation with Test-Time Partial Observations

arXiv:2609.10531v1 Announce Type: new Abstract: Image-to-3D models can generate visually compelling 3D assets from a single RGB image, but their geometry is often only loosely constrained by the avai...

By Jerred Chen, Simon Weber, Ronald Clark

fine-tuning