The paper investigates whether a language model’s forgetting—worsening performance on standard web text after fine‑tuning or further pretraining—can be mitigated by simple, fixed adjustments to its final token predictions. By fitting a vocabulary‑wide bias (a token‑specific bonus or penalty) and optionally a temperature scaling, the authors evaluate recovery across 14 base‑to‑derived model pairs ranging from 1B to 7B parameters. Results show modest gains: up to 10% recovery with bias alone and up to 36% when combined with temperature, though models further pretrained on math lose the most and recover little; even in a controlled simulation with random damage, the bias‑plus‑temperature fix recovers 29%.
By Hanyu Yang
The paper introduces a federated learning framework for training multimodal foundation models on unpaired, fragmented data distributed across private nodes. It uses a small public anchor set and Gram matrices to align disjoint private manifolds via centered kernel alignment, avoiding transmission of private samples. Additional techniques include subspace‑stabilized fine‑tuning to handle large transformers and precision‑weighted averaging to downweight uncertain nodes.
By Anders Eklund
The paper introduces a new multi‑agent reinforcement learning framework that separates agent identity from behavior by using events to trigger behavioral changes. It proposes Neural Manifold Diversity (NMD) to create a continuous behavior manifold and an event‑based hypernetwork that generates Low‑Rank Adaptation (LoRA) modules for on‑the‑fly policy reconfiguration. The authors prove that this design preserves reward maximization and show empirically that it outperforms baselines, generalizes zero‑shot, and uniquely solves tasks requiring sequential behavior reassignment.
By Hannes B\"uchi, Manon Flageat, Eduardo Sebasti\'an, Amanda Prorok
HyperRepair is a framework that restores the general capabilities of fine‑tuned language models while keeping their domain specialization, using only inference time. It learns a hypernetwork from repairs on source models, which generates a Repairing Side Net (RSN) that selectively corrects the model based on activation fingerprints. Evaluated on two model families, 12 LoRA adaptation datasets, and 15 general benchmarks, HyperRepair retains 99.5% of fine‑tuned domain performance and 93.4% of the base model’s general performance.
By Junrong Tang, Jianghui Wang, Hao Kong, Yang Lin, Yibo Yang
Saidzoda‑Tajik‑LFM‑2.6B is a 2.6‑billion‑parameter hybrid model adapted for Tajik tool calling. The adaptation uses two stages: LoRA continued pretraining on 1.49 million cleaned Tajik web documents followed by one epoch of supervised fine‑tuning on machine‑translated Tajik tool calls, English tool calls, and Tajik instructions. On a 174‑item Tajik tool‑calling test set, strict full‑call accuracy rises from 20.69 % for the base model to 64.14 %, while English performance remains unchanged; however, continued pretraining removes the native tool‑call format and part of the skill, and the model shows limited general assistant capabilities.
By Tohir Saidzoda, Nekruz Maxudov, Bedil Karimov, Abubakr Abdullozoda, Amin Jalilov, Jonibek Kishvarov, Rahmonali Sharipov, Mariyamkhon Shodyeva
The paper examines how large language models (LLMs) can be tuned for grammatical error correction (GEC) by adjusting inference-time parameters. It demonstrates that the repetition penalty can be used to control the precision–recall trade-off during decoding, without changing the model architecture, training, or prompt. Experiments on standard GEC benchmarks confirm that this simple, broadly applicable method allows inference-time adjustment of the precision–recall balance, similar to prior sequence‑to‑sequence techniques such as Align‑and‑Predict Decoding.
By Ryszard Staruch, Filip Grali\'nski
The paper proposes Embedding Space Separation (ES2), a fine‑tuning technique that increases the distance between harmful and harmless embeddings in large language models. ES2 uses two regularization losses—a embedding anchor loss and a KL‑divergence loss—to preserve the model’s general capabilities while enhancing safety. Experiments show that ES2 effectively defends against various attack methods with only modest performance loss on some tasks.
By Xu Zhao, Xiting Wang, Weiran Shen
DeFT‑RAG is a retrieval‑augmented generation framework that tackles multi‑hop questions by decomposing them into self‑contained sub‑questions, each explicitly naming the entities it queries. The approach removes the need for complex planning or dependency tracking by letting the LLM focus on a single fine‑grained task, while a rule‑based QA table and a lightweight decision model handle bookkeeping outside the LLM. Experiments on MuSiQue, 2WikiMultiHopQA and HotpotQA show that DeFT‑RAG outperforms thirteen baselines with both GPT‑4o‑mini and Qwen3.6, reduces input tokens, and achieves a 12.9% reduction in LLM calls through QA caching without sacrificing accuracy.
By Dingyi Kang, Bingzhe Li
Drone-PUR is a new dataset and benchmark for vision‑language models that focuses on visual clue discovery, interpretation, and integration in UAV scenes. It contains 11 tasks with 175k training and 20k test samples across single images, image sequences, cross‑modal images, and multiple viewpoints, all of which are human‑reviewed. Evaluation of 22 general‑purpose VLMs shows persistent gaps in spatial judgment and cross‑modal/cross‑view reasoning, while fine‑tuning on Drone‑PUR improves accuracy by up to 18 percentage points.
By Jiahe Wu, Bing Cao, Xingyue Guo, Ruipu Zhao, Beiming Yu, Jiekang Feng, Qinghua Hu, Pengfei Zhu
The paper investigates whether strong generalization on a source task guarantees transferability to a related target task in transfer learning. The authors hypothesize a negative answer and conduct adversarial attacks on the transfer learning process to find counterexamples. Their empirical results reveal a strong relationship between source-task generalization and transferability, though they do not rule out the possibility that good source generalization is sufficient for successful transfer.
By Mina Remeli
The paper introduces a self‑supervised, three‑stage pose estimation model for construction machines, leveraging contrastive learning (SimCLR) in the pre‑train stage and two fine‑tuning stages for transfer learning and downstream tasks. It trains on a large unlabeled ACID dataset and fine‑tunes on smaller simulated datasets, achieving 11% and 13% accuracy improvements over baseline self‑supervised models using ResNet‑50 and HRNet‑W32 backbones, respectively.
By Alaa Alshubbak, Daniel G\"orges
The paper introduces PanoVideoBench, a manually annotated benchmark with 1,000 questions covering five categories and 15 question types for panoramic video understanding, and identifies cross‑view tracking and sphere‑time evidence integration as weak points in existing models. To address these gaps, the authors create PanoVideoCoT, a dataset of 13,465 QA‑CoT pairs generated via sphere‑time‑grounded synthesis, and train PanoVideoReasoner using panoramic CoT fine‑tuning and evidence‑privileged on‑policy self‑distillation. The resulting model achieves 55.7% accuracy on PanoVideoBench, improving the baseline by 12.5 points and surpassing the previous state‑of‑the‑art by 2.6 points, particularly in spatial, cross‑view, and temporal reasoning.
By Lidong Lu, Lidong Zou, Xinyi Mao, Jiaxin Lv, Zhenquan Li, Liangrui Dong, Tong Lu, Guo Chen
PBSR: Pseudo-Burst Super-Resolution via Zero-Shot Novel View Synthesis proposes a method to enhance single-image super-resolution by generating synthetic burst frames through zero-shot novel view synthesis. The approach uses a viewpoint sampling strategy aligned with real burst data to create realistic parallax, then processes these frames with a BSR backbone that includes geometry-aware alignment and artifact-aware adaptation. Experiments on the BurstSR dataset demonstrate consistent improvements in PSNR, SSIM, and LPIPS compared to a single-frame baseline using the same reconstruction backbone.
By Yuki Kondo, Norimichi Ukita
NP-LoRA introduces a training‑free method for fusing subject and style LoRAs by projecting the content LoRA onto the null space of the style LoRA, thereby reducing interference from overlapping low‑rank subspaces. The approach uses a soft projection that balances content preservation and style suppression, offering a single‑parameter interpolation between linear merging and hard projection. Experiments on multiple pretrained LoRA pairs demonstrate that NP-LoRA achieves more balanced content‑style composition than strong baselines without retraining.
By Chuheng Chen, Xiaofei Zhou, Geyuan Zhang, Yong Huang, Gang Xiong
The paper introduces Differential Vector Erasure (DVE), a training‑free method for removing unwanted concepts from flow matching image generation models. DVE exploits the directional structure of the velocity field that encodes semantic concepts, constructing a differential vector field to isolate and suppress target concepts during inference. Experiments on the FLUX model show that DVE outperforms existing baselines in tasks such as NSFW suppression, style removal, and object erasure while maintaining image quality and diversity.
By Zhiqi Zhang, Xinhao Zhong, Yi Sun, Shuoyang Sun, Jiahui Wang, Bin Chen, Shu-Tao Xia, Xuan Wang
PRISM is a feed‑forward single‑image 3D reconstruction framework that splits the task into a parameter‑free geometric warp and a learned residual correction, eliminating the need for iterative diffusion sampling. It uses a two‑stage training strategy—latent supervised distillation for geometric generalization and perceptual fine‑tuning for appearance quality—to generalize from synthetic data. Experiments on three benchmarks show PRISM matches diffusion‑based methods in quality while cutting inference time to just 36 seconds per scene.
By Zhijie Zheng, Xinhao Xiang, Jiawei Zhang
The paper introduces a structured transfer learning framework for learning solution operators of differential equations when target data are scarce. It leverages abundant observations from related source systems by accounting for unknown sparse differences in governing equations, using a resolvent identity and whitened instrumental variables to build unbiased source estimating equations. A three‑stage estimator—target‑only ridge pilot, source‑wise sparse calibration, and projected refit—along with an iterative recalibration scheme, achieves non‑asymptotic prediction bounds and demonstrates clear performance gains over target‑only learning, naive pooling, and existing transfer methods.
By Tongyu Li, Zhuoyuan Li, Ruoxu Tan
EvoMCP introduces a mutual evolution framework that co‑evolves Model Context Protocol (MCP) agents and their training data. The method iteratively assesses an agent’s strengths and gaps, generates new tasks by adapting verified seed tasks, refines demonstrations through trajectory evolution, and fine‑tunes the agent with these task‑trajectory pairs. Experiments on Toolathlon, MCPMark, and MCP‑Atlas show consistent performance gains of 16.5%, 10.2%, and 12.6% respectively.
By Xuanang Gao, Kepeng Lin, Xinyan Shi, Yong Deng, Shuheng Shen, Changhua Meng, Xiaofeng Wu, Jie Yang, Wei Liu
The paper investigates whether auxiliary trajectory preferences can enhance small-language-model self‑improvement and transfer when only limited supervision is available. Two reward schemes—confidence‑based (SC) and entropy‑based (UID‑Conj.)—are compared through online reward augmentation and offline training on AIME problems. UID‑Conj. yields a 2.08‑point macro‑average improvement on the adaptation benchmark and outperforms uniform weighting in offline weighted SFT by 1.93 points, with task‑dependent benefits for SC and UID‑Conj. rewards.
By Minju Gwak, Alan Ritter
The paper introduces CoFailMed, an audit framework that examines correlated failures among pretrained encoder pools used for medical image segmentation. By evaluating 11 public encoders across five 2D tasks with frozen features and 1×1 readouts, the study finds that models sharing checkpoints exhibit the highest per‑case Dice correlation, and that distinct checkpoints within the same encoder family are more correlated than cross‑family pairs. The analysis also shows that richer readouts and fine‑tuning can narrow but not eliminate these correlations, and that effective pool size at a 33rd‑percentile Dice threshold varies with encoder family and training recipe, underscoring the need to report absolute dependence and threshold‑specific effective size alongside accuracy.
By Eunseob Choi, Kyeonghun Kim, Hyuk-Jae Lee, Nam-Joon Kim