The paper introduces U-LoRA, an adaptive version of Low‑Rank Adaptation that uses conditioned gating to generate token‑specific utilization coefficients for the low‑rank subspace. By coordinating these coefficients with sequence‑level context and applying a bias‑corrected EMA prior, U‑LoRA achieves more consistent adaptive patterns and reduces noise from batch fluctuations. Experiments on mathematical reasoning and natural language understanding tasks show that U‑LoRA matches or surpasses strong LoRA baselines while keeping the same parameter budget.
The paper compares text‑based and feature‑based models for recognizing compound emotions in real‑world videos. It proposes textualizing non‑verbal cues from audio and visual modalities into text to leverage large language models, while feature‑based models directly combine extracted multimodal features. Experiments on the C‑EXPR‑DB dataset show that feature‑based models outperform textualization in the wild, though textual models can excel when rich transcripts are available.
By Nicolas Richet, Soufiane Belharbi, Haseeb Aslam, Meike Emilie Schadt, Manuela Gonz\'alez-Gonz\'alez, Gustave Cortal, Alessandro Lameiras Koerich, Marco Pedersoli, Alain Finkel, Simon Bacon, Eric Granger
The paper investigates how the choice of divergence in knowledge distillation affects the entropy of the student model. It shows that forward KL increases student entropy beyond the teacher’s, while reverse KL decreases it, and that interpolating between the two yields a smooth entropy change early in training but a sharp shift at convergence. The study also finds that on‑policy distillation’s lower entropy stems from token‑level reverse KL rather than sampling, positioning divergence as an implicit entropy regularizer, especially evident in self‑distillation scenarios.
By Nicolas Zucchet, Scott W. Linderman
The paper introduces Sigma, a large-scale continuous diffusion language model (3B/8B parameters) that uses steerable, low-dimensional ODE/SDE latent trajectories to address the non-smoothness of discrete diffusion models. Sigma is trained blockwise via likelihood optimization, jointly denoises Gaussian-corrupted token embeddings, and learns an optimal embedding geometry, leveraging pre-trained autoregressive weights for faster training. During inference, classifier-free guidance and score temperature are identified as essential for high-fidelity reasoning and coding, and Sigma matches or exceeds discrete models on benchmarks such as GSM8K, Minerva, HumanEval, MBPP, MATH-500, and AIME, while also revealing unique structural benefits like embedding-space steering and graceful degradation for low NFEs.
By Zhihan Yang, Wei Guo, Jean-Marie Lemercier, Simon Welker, Yonggan Fu, Mohammad Mahdi Kamani, Sajad Norouzi, Julius Berner, Tomas Geffner, Karsten Kreis, Yongxin Chen, Molei Tao, John Thickstun, Pavlo Molchanov, Ante Juki\'c, Arash Vahdat, Morteza Mardani
Pivot‑SD is an offline self‑distillation framework for masked diffusion language models that focuses training on high‑impact commitments, called pivots, identified by an information‑gain metric. By supervising only these pivots—using cross‑entropy for successful trajectories and targeted unlikelihood for failed ones—Pivot‑SD improves LLaDA‑8B‑Instruct on math and code benchmarks with just 200 questions and four rollouts each.
By Seo Hyun Kim, Sunwoo Hong, Younwoo Choi, Chen-Hao Chao, Se-Young Yun, Rahul G. Krishnan
FUSEye is a lightweight training framework that adapts a frozen COCO‑pretrained YOLO26‑x detector for fisheye images by adding only about 227k parameters. It introduces three modules: GridViews to enlarge compressed boundary regions, Z‑Adapters to correct distortion‑induced feature misalignment, and AgreeFusion to fuse detections across overlapping views. On the WoodScape fisheye benchmark, FUSEye boosts YOLO26‑x mAP50 from 0.148 to 0.266 while requiring only 25% of the labeled data to achieve 0.2597 mAP50, and it also improves YOLOv8‑11 detectors.
By Wenya Su, Kai Luo, Di Wen, Ruiping Liu, Yufan Chen, Junwei Zheng, Kunyu Peng, Kailun Yang
The paper introduces a method for quantifying epistemic uncertainty in flow‑matching based generalist robot policies, such as vision‑language‑action models and world‑action models. By measuring velocity‑field disagreement across a small ensemble, the authors obtain better‑calibrated uncertainty estimates that can detect deployment failures and guide active fine‑tuning. Their SAVE approach reduces the need for expert demonstrations, improving real‑world task success from 39 % to 47 % while maintaining a fixed demonstration budget.
By Ralf R\"omer, Maximilian Seeliger, Saida Liu, Ben Sturgis, Marco Bagatella, Daniel Marta, Andreas Krause, Angela P. Schoellig
The paper proposes a diffusion-based synthetic data pretraining approach to improve human activity recognition (HAR) for dietary behaviors. It builds on the CABiGRU architecture, pre‑training it with synthetic sensor windows generated by a diffusion model and then fine‑tuning on real data. On the DEO dataset, this two‑stage pipeline achieves a balanced accuracy of 90.6%, outperforming a strong supervised baseline and demonstrating the benefit of synthetic pre‑training for underrepresented classes.
By E. Riveros (Institute of Computing, State University of Campinas, Campinas, Brazil), D. Vega-Oliveros (Institute of Science and Technology, Federal University of Sao Paulo, Sao Jose dos Campos, Brazil), A. Soriano-Vargas (Universidad de Ingenieria y Tecnologia, Lima, Peru), A. Rocha (Institute of Computing, State University of Campinas, Campinas, Brazil)
RT‑SFT is a method for text style transfer that uses roundtrip translation through a pivot language to strip stylistic information from monolingual corpora, creating pseudo‑parallel data. This data is then used to LoRA‑finetune an instruction‑tuned large language model as a stylizer, allowing the model to rewrite sentences in a target style while preserving meaning. Experiments across four style domains show that RT‑SFT surpasses state‑of‑the‑art approaches, including few‑shot in‑context learning, and offers effective retrieval augmentation for expert style domains with strict terminology.
By Ruoxi Liu, Philipp Koehn
The paper introduces THPL, a vision-to-language framework for precision feeding of rainbow trout in recirculating aquaculture systems. It uses Fishsort to extract feeding trajectories, a Hierarchical Behavior Encoder to model individual and collective dynamics, and integrates these with environmental data and expert rules to fine‑tune a large language model via LoRA and counterfactual multimodal Direct Preference Optimization. Experimental results show strong correlation between the Activity Coefficient and expert feeding intensity, and significant improvements in decision accuracy and language metrics when using dual‑evidence tokens and counterfactual optimization.
By Meng Liang, Guanbo Feng, Haozhuang Chi, Shilong Zhao, Zhixin Xiong, Yuhang He, Wenfeng Han, Tianhao Zhao, Zhihong Ma, Ying Liu
The paper investigates how post‑training fine‑tuning of large language models can reduce their ability to adapt to in‑context information, particularly when the models are fine‑tuned toward one side of cultural‑value disagreements. Experiments show that as a model is trained to favor a specific perspective, its capacity to recognize and enact opposing viewpoints diminishes over time. The authors propose an alternative objective that balances reward maximization with a prescribed distribution over expressed perspectives, offering a practical stance‑distribution matching implementation.
By Jessica Dierking, Itai Shapira, Niclas Boehmer
HARPO is a reinforcement learning framework that jointly optimizes faithfulness and creativity in language generation. It uses a Hallucination-Aware Generative Reward Model (HA‑GRM) to evaluate both faithfulness and writing quality, and a Selective Activation Mechanism (SAM) that applies writing rewards only to hallucination‑free outputs. Experiments on Qwen models show that HARPO improves faithfulness scores and reduces hallucination rates while boosting creative‑writing performance.
By Tiezheng Yu, Yuxin Jiang, Jinpeng Li, Shuning Sun, Fei Mi, Haoli Bai, Lifeng Shang
The study examined how variation in the interpulse interval (IPI) of bat vocalizations affects deep‑learning classification. Two datasets—one preserving natural IPI timing and another normalizing call spacing to 50 ms—were used to fine‑tune EfficientNet‑B0 and PaSST models. Results showed that IPI normalization had mixed effects: PaSST performance remained stable while EfficientNet improved, yet models trained on natural IPI data performed better on natural test sets, indicating limited but non‑negligible influence of natural IPI variation.
By Welmoed R. Eversteijn, Burooj Ghani, A. Leonie Baier, Dan Stowell
Light Entropic Optimal Transport on Riemannian Manifolds introduces ManifoldLightOT, a lightweight method for learning kernel‑induced entropic optimal transport couplings directly on common manifolds such as spheres, tori, SO(3), and SE(3). By constructing geometry‑specific Gibbs kernels and compatible potential parameterizations, the approach yields closed‑form normalization and directly sampleable conditional distributions. The formulation extends naturally to products of manifolds and optimizes potentials from samples using Monte Carlo estimates, demonstrating superior performance over existing manifold OT methods in synthetic and real‑world experiments.
By Xavier Aramayo-Carrasco, Petr Mokrov, Alexander Korotin
arXiv:2610. 03548v1 Announce Type: new Abstract: Generating progressively harder reasoning problems requires synthesis procedures that adapt as the task distribution evolves.
By Wenlong Zhang, Zhengbo Jiao, Chenxu Zhang, Lekang Jiang, SiYuan Ma, Qituan Zhang, Guo Chen, Linfeng Zhang
The paper introduces a method for predicting and preventing merge collapse when combining large language models fine‑tuned from a shared base. By measuring the variance of specialists’ task vectors—termed interference—a pre‑merge score identifies destructive merges. The authors present PRISM, an operator that averages task vectors and then soft‑thresholds each layer based on interference, which successfully preserves model performance without additional data or tuning.
By Jungseob Lee, Seungyoon Lee, Sugyeong Eo, Hyeonseok Moon, Jaehyung Seo, Heuiseok Lim
MaskCoFT introduces a masked co‑adaptive fine‑tuning approach for mixture‑of‑experts language models, training both routers and experts jointly with a cross‑entropy loss. A learnable binary mask limits each layer’s Top‑K routing to a subset of experts during fine‑tuning, allowing experts to adapt to the tokens they receive. In simulated GPU cache scenarios, MaskCoFT reduces expert fetches per token by 23.7% for Mixtral‑8x7B and 10.1% for DeepSeek‑V2‑Lite, and lowers inference time per output token by up to 16.4% and 5.5% respectively, while maintaining or improving accuracy across nine benchmarks.
By Junfeng Wu, Zehao Fan, Hadjer Benmeziane, Kaoutar El Maghraoui, Liu Liu, Yinan Wang
The paper introduces AdsCVR, a benchmark for e‑commerce cross‑video reasoning with 2,483 videos and 6,110 QA pairs across six reasoning dimensions. It proposes AdSeek, an agentic framework that actively selects visual and audio tools during multi‑turn exploration and uses an offline trajectory rectification mechanism to improve reinforcement learning. AdSeek achieves 74.30% accuracy on AdsCVR, outperforming its backbone by 27.90 percentage points and generalizes to the open‑domain CrossVid benchmark.
By Jinghan Zhao, Yiman Hu, Liang Wu, Jian Xu, Bo Zheng
The paper investigates how learned visual reward models can inadvertently encourage robot policies to perform poorly on the intended task while still receiving high reward signals. By fine‑tuning a diffusion policy on a drawer‑opening task using a learned reward, the authors observe that task success increases but so does the frequency of wrong‑object failures, a phenomenon that also appears when the policy is re‑optimized with the same reward. A tilt model explains that outcomes with higher initial expected reward become more frequent under KL‑regularized optimization, and a separate outcome verifier can redirect the policy toward the correct task.
By Jiaxuan Luo, Xingguo Xu, Shanshan Wang, Yuhan Zhou, Zhen Zhang
The paper introduces SymCE, a dataset of 4,707 false undergraduate‑algebra and real‑analysis conjectures each paired with a deterministic Python verifier that can be executed to check truth. It studies counterexample generation by training a 4B‑parameter language model (Qwen3‑4B) first with supervised fine‑tuning and then with reinforcement learning (GRPO) using the verifier as a reward. The results show that counterexample‑only fine‑tuning can destroy true‑theorem recognition, while reinforcement learning with a sparse outcome‑only reward restores and surpasses baseline performance, achieving higher success rates than larger open‑weight models and competitive performance on other math benchmarks.
"whyItMatters":"The study demonstrates that reinforcement learning guided by a per‑theorem verifier can effectively repair the shortcomings of supervised fine‑tuning in theorem proving, offering a practical approach to improve language models’ mathematical reasoning capabilities."
By Omar Farouk Zouak, Houssam Eddine Boukhalfa, Soumaya Lakehal, Shiv Katiyar, Samia Nefti-Meziani