The paper identifies a specific issue in supervised fine‑tuning (SFT) of large language models called factual access failure, where models can recognize correct facts under constrained tests but fail to generate them in open‑ended settings. It demonstrates that SFT can cause both genuine wrong answers and expression‑level errors such as verbosity or formatting mismatches. To mitigate this, the authors propose Recall‑Anchored Distillation (RAD), a self‑distillation method that aligns the fine‑tuned model with the base model’s soft output distribution on unlabeled out‑of‑distribution text, thereby recovering lost factual recall without needing labeled data.
By Haodong Chen, Yadong Wang, Shengtao Wen, Dong Liang, Xiang Chen
The paper introduces Defactualize-Steer-Rehydrate (DSR), a framework that combines a typed, salience-weighted knowledge graph with activation steering to enable style-controllable yet fact-preserving generation in large language models. DSR extracts salient entities, replaces them with placeholders before steering, and then rehydrates verified values after generation. Experiments on six LLaMA-family models show that DSR improves verified-entity recovery compared to steering alone while maintaining effective style control.
By Tanmay Kumar Shrivastava, Darsh Rohit Nandu, Rajesh Kumar Mundotiya
Thermo-FL is a thermal‑aware federated LoRA fine‑tuning framework that adapts local adapter training and sparse update transmission based on device temperature. It introduces TERRA, a robust aggregation pipeline that uses norm filtering, mask‑aware directional validation, adaptive clipping, and mask‑aware aggregation to defend against Byzantine and communication‑layer adversaries. Experiments on a large‑scale emulator and a Jetson testbed show Thermo-FL improves robustness under adversarial sparse aggregation, stabilizes device temperature, reduces upload size, and preserves utility on GSM8K and BoolQ tasks.
By Shiva Shrestha, Kazi Shaharair Sharif, Zongxing Xie, Jiajing Huang, Anhao Xiang, Honghui Xu
The paper introduces VA‑DPO, a method that trains language models to generate text with a specified continuous affect point in the Valence‑Arousal plane. By using a frozen VA regressor to score candidate generations and selecting pairs with a distance margin, the approach modifies Direct Preference Optimization to better hit target emotions. Experiments on Llama‑3.1‑8B‑Instruct show a 33% reduction in mean VA distance compared to system‑prompting and 25% over few‑shot prompting, while maintaining performance on benchmarks like MMLU, HellaSwag, and TruthfulQA.
By Hyunwoo Kim
Researchers must often choose between Bayesian or neural network models of behavior, two paradigms with complementary strengths and weaknesses. An ideal paradigm would facilitate testing many kinds of...
The article explains how LoRA fine‑tuning addressed an under‑labeling issue in the SigLip model. It outlines that whether this approach is suitable depends on three specific questions. The post provides guidance on evaluating the appropriateness of fine‑tuning for your own use case.
By Miikka Silfverberg
arXiv:2509. 08022v3 Announce Type: replace-cross Abstract: Aligning large language models (LLMs) with diverse human values is essential for safe and effective deployment, yet existing benchmarks often overlook cultural and demographic variation.
By Yao Liang, Dongcheng Zhao, Feifei Zhao, Guobin Shen, Yuwei Wang, Dongqi Liang, Yi Zeng
arXiv:2608. 20099v1 Announce Type: cross Abstract: LLM-based Multi-Agent Systems (MAS) achieve strong performance on complex reasoning tasks by coordinating multiple agents, but at the cost of substantial token consumption.
By Poomphob Suwannapichat, Boonyarit Changaival, Caesar Wu, Pascal Bouvry
arXiv:2608. 19567v1 Announce Type: new Abstract: While text-to-3D generation has advanced rapidly, achieving high geometric fidelity at low inference cost remains challenging.
By Bowen Cui, Weijie Wang, Zeyu Zhang, Yefei He, Mingda Lin, Haoyu Zhao, Yuanyu He, Donny Y. Chen, Feng Chen, Bohan Zhuang
arXiv:2601. 02918v4 Announce Type: replace Abstract: Image Quality Assessment (IQA) is a long-standing problem in computer vision.
By Guoqiang Liang, Jianyi Wang, Zhonghua Wu, Shangchen Zhou, Chen Change Loy
arXiv:2608. 20281v1 Announce Type: cross Abstract: Large language models often fail to answer questions about a bounded document collection when the source documents are not retrieved at inference time.
By Qian Kou, Xiaofeng Shi, Xiaosong Qiu, Hua Zhou
arXiv:2608. 19669v1 Announce Type: cross Abstract: Latent reasoning has advanced multimodal reasoning through a two-stage training paradigm: (1) a helper image is encoded into latent tokens to teach visual chain-of-thought during a supervised fine-tuning (SFT) stage, and (2) these latent tokens are further refined with reward feedback during a reinforcement learning (RL) stage.
By Haoqiang Kang, Yinpeng Chen, Luyang Liu, Jesper Sparre Andersen, Abhijit Ogale, Baochen Sun, Lichan Hong, Ed H. Chi
arXiv:2507. 06127v2 Announce Type: replace-cross Abstract: Prefix adders are fundamental arithmetic circuits, but their design space grows exponentially with bit-width, posing significant optimization challenges.
By Dongsheng Zuo, Jiadong Zhu, Yang Luo, Yuzhe Ma
arXiv:2608. 19920v1 Announce Type: new Abstract: A lot of prior work addressed key-value (KV) cache selection and compression by sparse attention to enable long-context inference for transformer language models without excessive hardware budgets.
By Matthias Seeger, Zeyu Zhang, Vihang Patil, Konstantinos Benidis, Sebastian Schelter
arXiv:2511. 20973v2 Announce Type: replace-cross Abstract: Large Audio Language Models (LALMs) deliver strong performance across speech and audio tasks, but their audio encoders generate high-rate token sequences (e.
By Saurabhchand Bhati, Samuel Thomas, Hilde Kuehne, Rogerio Feris, James Glass
arXiv:2608. 19743v1 Announce Type: new Abstract: We describe our entry to the PhysAI Dynamic 4D Reconstruction Challenge, which placed third of 27 teams at 0.
By Nicol\`o Savioli
arXiv:2608. 19666v1 Announce Type: new Abstract: Deep learning-based whole-body PET-CT lesion segmentation can support cancer staging, treatment planning, and response assessment, but generalization is limited by scarce annotations and domain shifts.
By Bashirul Azam Biswas, Amartya Bhattacharya, Biratal Raj Wagle, Matthew E. Maeder, James B. Yu, Indrani Bhattacharya
arXiv:2608. 19680v1 Announce Type: new Abstract: Smart contract vulnerability detection with Large Language Models (LLMs) faces three causally linked challenges.
By Tenghui Huang, Jiawen Kang, Dongning Liu, Changyan Yi, Chengjun Cai, Anjia Yang, Li Li, Dong In Kim
arXiv:2608. 20326v1 Announce Type: cross Abstract: Deep neural networks are often overconfident, assigning high confidence even to incorrect predictions.
By Parampreet Singh, Anushka Singh, Sumit Kumar, Vipul Arora
arXiv:2507. 09471v4 Announce Type: replace Abstract: Continual Learning (CL) empowers AI models to continuously learn from sequential task streams.
By Lingfeng He, De Cheng, Zhiheng Ma, Huaijie Wang, Dingwen Zhang, Nannan Wang, Xinbo Gao