Fine-tuning and adaptation

LoRA, PEFT, instruction tuning and domain adaptation — adapting a pretrained model without paying to train one.

8,205 stories · RSS feed

arXiv Computer Vision
Sep 11

Beyond Visual Quality: Evaluating Physical Consistency under Ego-Motion with EgoGenEval

The paper introduces EgoGenEval, a new benchmark that assesses the physical consistency of visual generators under ego‑motion by measuring Camera Motion Grounding and Scene State Preservation across 1,400 cases and 2,360 target views. Experiments on 16 pose‑free generators and two pose‑conditioned references show that current models struggle to maintain both camera motion and scene state simultaneously. A follow‑up study using EgoGen‑Train demonstrates that pairwise supervision does not effectively improve both metrics together, suggesting the need for a trajectory‑centric training paradigm.

By Yilin Long, Chenming Zhu, Zitang Gou, Jingli Lin, Tai Wang
arXiv Computation and Language
Sep 11

LLMAR: A Tuning-Free Recommendation Framework for Sparse and Text-Rich Industrial Domains

LLMAR is a tuning‑free recommendation framework designed for sparse, text‑rich industrial B2B domains. It transforms user behavioral history into structured semantic motives using LLM inference, employs a reflection loop to self‑correct hallucinations, and operates cost‑effectively with asynchronous batch processing. Experiments on MovieLens‑1M, Amazon Prime Pantry, and a construction risk dataset show LLMAR surpasses state‑of‑the‑art learning models, achieving up to a 54.6% nDCG@10 improvement while keeping inference costs around $1 per 1,000 users.

By Ryogo Hishikawa, Ichiro Kataoka, Shinya Yuda
arXiv Computation and Language
Sep 11

Larger Context Window, Fewer Overcorrections: Optimizing Prompts and Batching for Minimal-Edit Grammatical Error Correction

The paper presents a prompt-based method for minimal-edit grammatical error correction (GEC) that reduces overcorrection in large language models (LLMs). It introduces taxonomy-based instructions, batch prompting to regularize overcorrection, and LLM-assisted prompt optimization, achieving an $F_{0.5}$ score of 78.32 on BEA-2019 with Gemini 3.1-Pro. This approach narrows the performance gap to fine-tuned models while avoiding their infrastructure demands.

By Kateryna Karpo, Artem Chernodub
arXiv Machine Learning
Sep 11

The information geometry of large language models is shared, learned, and controllable

The paper investigates how large language models (LLMs) share a common Fisher‑Rao geometry in their next‑token probability distributions, revealing that behaviour largely determines this geometry while activation geometry depends on coordinate choices. Across transformer, state‑space, and recurrent architectures, output geometries align more closely than activation geometries, and this shared structure facilitates semantic‑category transfer and improves agreement with human word choices as models scale and train. The study further demonstrates that geometry can guide minimum‑disturbance interventions, enabling reusable control that preserves behaviour better than Euclidean methods and enhances steering, editing, attribution, dictionary learning, and fine‑tuning.

By Dario Picozzi
arXiv Machine Learning
Sep 11

Enabling Knowledge Graph Understanding at Scale with the EXplore Your Graphs ENgine (EXYGEN)

The paper introduces EXYGEN, a framework that enables conversational access to large knowledge graphs by combining VoID descriptions, ShEx schemas, retrieved triples, and example question‑query pairs in a retrieval‑augmented generation pipeline. On the SciQA benchmark, this approach achieves an exact‑match score of 0.419 without fine‑tuning any large language model, and shows that larger general‑purpose LLMs can outperform smaller code‑specialized ones when provided sufficient context. To scale metadata generation for very large KGs, the authors propose a predicate‑coverage‑aware parallel graph sampling strategy that preserves structural diversity, reduces runtime by over 80× on OpenCitations Meta and GESIS, and is the only tractable method for obtaining complete metadata on ORKG.

By Harshdeep Singh, Yurui Zhu, Giovanni Colavizza, Matteo Romanello
arXiv Computer Vision
Sep 11

SenseNova-U1.5: Towards Native Unified Visual Intelligence

SenseNova-U1.5 is an 8B‑MoT native unified multimodal model that can understand, reason about, and generate visual content without using an encoder or VAE. It improves visual fidelity and text rendering through spatially coherent patch reconstruction, large‑scale training on curated generation and editing data, and native resolutions up to 4K. Post‑training, specialized experts for visual aesthetics, bilingual text rendering, infographic generation, and image editing are optimized and distilled into a multi‑expert framework, yielding advances in image fidelity, complex composition, multi‑reference editing, and instruction following.

By Haiwen Diao, Jiahao Wang, Chenjing Ding, Hanming Deng, Jiangnan Chen, Ruixi Zhang, Ruohui Wang, Wenwen Tong, Xiangyu Fan, Yubo Wang, Yue Zhu, Yuwei Niu, Zhengqi Bai, Zhiqian Lin, Zhitao Yang, Zhongang Cai, Bo Yang, Chen Feng, Chengguang Lv, Guangjia Liu, Guanlin Wang, Hanyu Zhang, Haojia Yu, Hongcan Xiao, Hongli Wang, Huan Wu, Huaping Zhong, Jian Fang, Jianan Fan, Jiaqi Li, Jiefan Lu, Jing Zuo, Jingcheng Ni, Junxiang Xu, Linjun Dai, Mutian Xu, Peishen Yan, Penghao Wu, Ruijie Mao, Ruisi Wang, Shihao Bai, Shuang Yang, Shuya Yang, Shuyan Zheng, Silei Wu, Siying Li, Tao Chu, Tianbo Zhong, Tongxi Zhou, Weichao Luo, Weichen Fan, Wenhao Jia, Wenjie Gao, Xiangli Kong, Yan Li, Yang Yong, Zimo Wen, Zixuan Qian, Wenxiu Sun, Ruihao Gong, Quan Wang, Lewei Lu, Lei Yang, Ziwei Liu, Dahua Lin
arXiv Machine Learning
Sep 11

Artificial Intelligence Algorithms for the Detection of Pathologies Related to Lung Cancer through Image Analysis using Convolutional Neural Networks and Data Augmentation: a systematic mapping of the literature

The article reviews recent applications of artificial intelligence and deep learning, particularly convolutional neural networks with transfer learning and data augmentation, for detecting lung cancer in medical images. It surveys 96 studies from 2015 onward, highlighting high sensitivity and specificity achieved by these methods. The authors also identify challenges such as data standardization, model explainability, patient privacy, and ethical concerns that must be addressed before clinical deployment.

By Pablo Ramirez Amador
arXiv Computation and Language
Sep 11

BodyCam-VQA: Enhanced Body-Worn Camera Video Captioning via Multimodal Reasoning and Probe Question Generation

The paper introduces BodyCam-VQA, an adaptive visual question answering framework designed to improve captioning of police body‑worn camera footage. By employing structured multimodal reasoning and probe question generation, the system extracts fine‑grained visual evidence that conventional captioning models miss. Experiments with various question generation models show that this VQA‑driven approach yields more reliable, objective, and detailed records of enforcement events.

By Karish Gupta, Matthew Alex, Alex Li, Yang Wu, Yun-Wei Chu, Kashif Munir, Xiaotian Zhou, Zhengping Ji, Xiaozhong Liu
arXiv Computation and Language
Sep 11

Using Semantic Uncertainty to Estimate Transition Relevance in Turn-taking

The paper proposes using semantic uncertainty, derived from large language models, to predict Transition Relevance Places (TRPs) in spoken dialogue. By sampling possible continuations of an ongoing turn and measuring changes in semantic dispersion, the authors identify moments when a listener might take the floor. Their method outperforms prompt-based and fine-tuned text-only baselines on a dataset with real-time TRP labels, supporting the idea that evolving semantic constraints inform turn‑taking opportunities in unscripted interaction.

By Muhammad Umair, Jan P. de Ruiter
arXiv Machine Learning
Sep 11

SIRF: A Spec-Internalized Risk Foundation Model for Industrial Content Risk Control

The paper introduces SIRF, a Spec-Internalized Risk Foundation Model designed for industrial content risk control. SIRF internalizes complex platform policies through EntiGraph, MAGA rewriting, and account-level chain-of-thought, then embeds them into the model weights via continued pretraining, enabling high‑precision, ultra‑low‑latency verdict‑only deployment. In controlled comparisons, SIRF‑8B‑SFT achieves a 71.3% Black Recall@P95—15.1 percentage points higher than a baseline—using only ~70 M CPT tokens and improves mis‑penalization recovery by 20% in a tree‑model adjudication layer.

By Suwan Wu, Yumeng Lin, Pengcheng Yuan, Xiaolong Jiang
arXiv AI
Sep 11

Scored vs. Generated Readouts in Behavioral Language Models: An Empirical Study of Elicitation Format

The study compares two ways of obtaining predictions from language models fine‑tuned on customer behavior: scoring answer tokens directly versus generating a written rationale and then scoring the resulting answer. Across 13 model‑domain cells covering four retail tasks, scored readouts consistently rank outcomes more accurately than generated readouts, with an AUC improvement ranging from 1.5 to 14.5 points. The authors also find that a third readout—eliciting a probability before any verdict—improves calibration but only when outcome rates are represented in training, and they recommend using generated rationales for interpretability while relying on scored heads for ranking.

By Touchapon Kraisingkorn, Krittin Pachtrachai, Wachiravit Modecrua
arXiv AI
Sep 11

Which Tokens Should SFT Actually Learn? A Token-Trimming Perspective on Mathematical Reasoning

The paper introduces Trimmed Logit-Gap SFT (TrimSFT), a token-level reweighting strategy that adjusts supervised fine-tuning loss based on the logit gap between the correct token and its strongest competitor. TrimSFT trims supervision from tokens that are either already mastered (large logit gap) or poorly supported (small or negative logit gap), focusing learning on tokens with intermediate logit gaps. Experiments on six base models across five mathematical reasoning benchmarks show that TrimSFT consistently outperforms standard SFT, achieving the best average performance on five of six models and up to +26.9 points on MATH500.

By Yaning Jia, Chunhui Zhang, Wenxuan Xu, Xingjian Diao, Xiaoyuan Wang, Soroush Vosoughi
arXiv AI
Sep 11

Reference-Based Bias Detection in LLMs via Relative Representations of Hidden States

The paper introduces a reference‑based bias detection method that audits hidden‑state representations of language models by encoding sentences as similarities to a fixed set of anchor sentences. This relative representation allows comparison across model variants, such as before and after fine‑tuning, and yields a metric called Representational Bias Shift (ΔB). ΔB correlates strongly with output‑level bias changes, can detect bias‑increasing checkpoints with high ROC AUC, and is computationally efficient, requiring only a few minutes and far less compute than traditional benchmarks.

By Marek Jeli\'nski, Jan Dubi\'nski, Maciej Chrabaszcz, Sebastian Cygert
arXiv Computation and Language
Sep 11

OpenResearcher: A Fully Open Pipeline for Long-Horizon Deep Research Trajectory Synthesis

OpenResearcher is a fully open, reproducible pipeline for generating long‑horizon deep research trajectories that interleave search, evidence aggregation, and multi‑step reasoning. It decouples corpus bootstrapping from trajectory synthesis and runs the search‑and‑browse loop offline using three browser primitives over a 15M‑document corpus. Using GPT‑OSS‑120B as a teacher, the pipeline produced over 97K trajectories, enabling a 30B‑A3B model to achieve 54.8% accuracy on BrowseComp‑Plus and providing insights into pipeline design through controlled analysis.

By Zhuofeng Li, Dongfu Jiang, Xueguang Ma, Haoxiang Zhang, Ping Nie, Yuyu Zhang, Kai Zou, Jianwen Xie, Yu Zhang, Wenhu Chen
arXiv Computation and Language
Sep 11

MindTopo: Can Foundation Models Reason in Topological Space?

MindTopo is a benchmark that tests foundation models on topological reasoning, covering five cognitive properties—continuity, separation, order, enclosure, and knots—across two cognitive levels: reasoning and planning. It contains 11,030 instances from 13 procedurally generated task types, and evaluates 14 multimodal large language models, including agent configurations with image and video generation. Results show that models perform better on reasoning than planning, and even the best model lags far behind human performance, with fine‑tuning and reinforcement learning improving reasoning more than planning.

By Yunfei Ge, Anbang Liu, Qineng Wang, Johnalbert Garnica, Jianwen Lyu, Zihan Wang, Reuben Tan, Jianfeng Gao, Ruohan Zhang, Yining Hong, Jiajun Wu, Manling Li
arXiv AI
Sep 11

Arbitrary Cipher Attacks Against Large Language Models Do Not Require Fine-Tuning

The paper reports that large language models can acquire cipher-based covert communication skills without fine‑tuning, using prompting or in‑context learning instead. This enables new jailbreak attacks that bypass alignment safeguards by encrypting harmful requests, making them appear as nonsensical text to harmfulness classifiers. The authors demonstrate successful attacks against frontier models from Anthropic, Google, and OpenAI.

By Thomas Rivasseau
arXiv AI
Sep 11

Beyond Training: A Feasibility Taxonomy for Inference-Time AI Governance

The paper "Beyond Training: A Feasibility Taxonomy for Inference-Time AI Governance" presents a taxonomy of twenty inference‑time mechanisms for monitoring, verification, and enforcement, each evaluated on a four‑point readiness scale using evidence from four vendors. It applies this taxonomy to a two‑dimensional adversary model and maps the mechanisms to four governance scenarios, finding that most mechanisms are commercially available but only adequate against cooperative or low‑to‑medium‑capability users, not high‑capability state‑level deployers. The study also links inference‑stage controls to hardware‑stage mechanisms through a substitution principle and reports a second‑rater reliability of 0.74. whyItMatters":"The work identifies the current gaps and readiness of inference‑time governance tools, highlighting that existing mechanisms are insufficient against powerful adversaries and thus informing future regulatory and technical development."

By Samar Ansari
arXiv Machine Learning
Sep 11

Your Model Already Knows Don't Teach It, Learn to Ask It: Soft Prompting for Few-Shot Adaptation of Vision-Language Models

The paper explores soft prompting for few‑shot object detection with vision‑language models, showing that optimizing a small number of continuous prompt tokens—especially when placed at the cross‑modal boundary and initialized from an empty space token—can match LoRA performance while training far fewer parameters. Soft prompting also avoids catastrophic forgetting, transfers to newer models, and can be verbalized into readable prompts. The study extends these findings to manipulation tasks, indicating that VLMs already contain much of the necessary knowledge for specialized domains, and the main challenge is learning how to ask for it.

By Gautam Rajendrakumar Gare, Siyi Li, Hewei Wang, Cesar Daniel Hernandez, Wei Zhao, Wolfgang M. Pauli, John Galeotti, Deva Ramanan
arXiv Machine Learning
Sep 11

When Synthetic Data Hurts: On Catastrophic Forgetting in Skill Retrieval for LLM Agents

The paper investigates how synthetic data used for fine‑tuning large language model (LLM) skill routers can lead to catastrophic forgetting of real and out‑of‑distribution (OOD) skills. By evaluating a production router with 34,396 skills, the authors show that synthetic‑data fine‑tuning improves in‑distribution retrieval but degrades performance on real and OOD data. They test several continual‑learning inspired mitigation methods—embedding‑anchor regularization, Learning without Forgetting, Elastic Weight Consolidation, and L2‑initialization—and find that these approaches both preserve OOD retrieval performance and boost synthetic in‑distribution retrieval by up to 13.98% for a 0.6B Qwen retriever and reranker.

By Syed Shariyar Murtaza, Yifan Nie, Utkarsh Soni, Eugene Wen, Arvid Frydenlund
arXiv Machine Learning
Sep 11

Empirical Evaluation of Membership Inference Attacks on NLP Text Classifiers: A Baseline Study on SST-2

The paper evaluates membership inference attacks (MIAs) on NLP text classifiers using the GLUE SST‑2 sentiment dataset. It compares a TF‑IDF + Logistic Regression pipeline with a fine‑tuned DistilBERT model under a loss‑threshold MIA, finding that both models leak membership signals despite high accuracy. The study also tests mitigations, showing that stronger regularization reduces leakage for Logistic Regression at a utility cost, while fine‑tuning DistilBERT for fewer epochs lowers leakage with minimal accuracy loss.

By William Novak (Minot State University), Muhammad Abusaqer (Minot State University)