The paper investigates why supervised fine‑tuning (SFT) of large language models leads to increased hallucinations of factual information. It proposes a self‑distillation SFT approach that regularizes output‑distribution drift to preserve pre‑training knowledge, and shows that freezing parameter groups can reduce hallucinations when new knowledge is unnecessary. Experiments attribute the main cause to interference among overlapping semantic representations, which self‑distillation mitigates, and an associative‑memory model explains the forgetting dynamics.
By Guy Kaplan, Zorik Gekhman, Zhen Zhu, Lotem Rozner, Yuval Reif, Swabha Swayamdipta, Derek Hoiem, Roy Schwartz
V-Co investigates visual co-denoising for pixel-space diffusion models, using a unified JiT-based framework to isolate key design choices. The study identifies two essential components: a dual-stream architecture with flexible cross-stream interaction and a perceptual-drifting hybrid loss combined with RMS-based feature rescaling for stronger semantic supervision. Experiments on ImageNet-256 demonstrate that V-Co surpasses baseline pixel-space diffusion and strong prior pixel-diffusion methods at comparable model sizes while requiring fewer training epochs.
By Han Lin, Xichen Pan, Zun Wang, Yue Zhang, Chu Wang, Jaemin Cho, Mohit Bansal
SCoNE (Selective Context-aware Neuron Editing) is a training‑free method that enhances Retrieval‑Augmented Generation (RAG) by selectively strengthening context‑aware feed‑forward network neurons identified through high attribution and high cross‑input variability. It requires only a few mining samples, no fine‑tuning, and incurs no inference‑time overhead. Experiments on multiple knowledge‑intensive question‑answering benchmarks and two LLM backbones show that SCoNE consistently outperforms competitive baseline methods.
By Chaewon Kim, Seo Yeon Park
The paper introduces the Mass‑Conserving Perceptron (MCP), a physics‑aware AI framework that enforces conservation laws while learning hydrological process relationships from data. By progressively adding physically meaningful components—such as bounded soil storage, state‑dependent conductivity, variable porosity, infiltration capacity, surface ponding, vertical drainage, and nonlinear water‑table dynamics—to a single MCP storage unit, the authors demonstrate that predictive skill for daily streamflow improves across 15 U.S. catchments. The study finds that the impact of each process representation varies with hydroclimate, with vertical drainage boosting performance in arid and snow‑dominated basins but hindering it in rainfall‑dominated ones, while surface ponding has minimal effect; the best MCP configurations rival LSTM benchmarks while retaining explicit physical interpretability.
By Mohammad A. Farmani, Hoshin V. Gupta, Ali Behrangi, Muhammad Jawad, Sadaf Moghisi, Guo-Yue Niu
The paper introduces NAPHA, a lightweight post‑hoc alignment method that improves large language model (LLM) predictions of human judgment distributions (HJD) by matching LLM output distributions to HJD through entropy‑based class assignment and specialized alignment models. Experiments on five datasets show that while LLMs perform near human‑level on hard‑label tasks, they struggle with soft‑label predictions, and NAPHA consistently enhances soft‑label accuracy, especially on high‑entropy instances. The study also demonstrates that better entropy class prediction can further boost NAPHA’s effectiveness.
By Sebastian Steindl, Nikos Voskarides, Alberto Gasparin, Diego Marcheggiani
The paper introduces LEGO, a benchmark dataset pairing user text descriptions with human‑annotated fine‑grained constraints and reference 3D scenes, and LEGO‑Eval, an evaluation framework that decomposes descriptions into atomic constraints and verifies each using grounding and spatial reasoning tools. It demonstrates that LEGO‑Eval detects misalignment more accurately than existing methods and that current 3D scene synthesis approaches achieve at most a 10% success rate on this benchmark.
By Minseok Kang, Dongwook Choi, Gyeom Hwangbo, Seungwon Lim, Kai Tzu-iunn Ong, Jinyoung Yeo
The study investigates whether large language models (LLMs) can reliably detect when their own responses have been manipulated by adversarial prefill attacks. Across ten instruction‑tuned LLMs ranging from 3B to 70B parameters and four safety benchmarks, none consistently recognized compromised outputs, with models claiming intent on prefilled responses at an average of 25.3%. The research identifies that introspective signals mainly arise from safety reasoning and refusal, and that training to improve introspection can paradoxically increase attack success, underscoring the fragility of LLM self‑reporting in safety contexts.
By Quang Minh Nguyen, Uzair Ahmed, Taegyoon Kim
VerTox is a framework that turns corpus poisoning of neural ranking models into a verifiable reward‑guided reinforcement learning problem. By fine‑tuning compact large language models with reward shaping that couples ranking distortion and factual corruption, VerTox generates fluent, low‑perplexity adversarial documents that frequently outrank target items across multiple ranking architectures, including a commercial embedding model. Experiments show near‑perfect attack success and significant degradation of downstream retrieval‑augmented generation performance.
By Zhiqi Huang, Vivek Datla, Zhichao Xu, Puxuan Yu, Vivek Srikumar, Alfy Samuel
Safin-1 is a new family of foundation models that embeds safety directly into the model’s internal state through memory routing and state evolution. It uses a Memory-Anchor Routing across Context History (MARCH) architecture to maintain structured memory states and selectively retrieve relevant historical information, allowing test‑time adaptation of persistent capability states without altering the backbone. Evaluations show that this routed‑state interface improves safety on downstream tasks while also enhancing general capabilities, long‑context understanding, retrieval, and efficiency.
By Ming Zhang, Kaisen Yang, Shu Yu, Ermo Hua, Zhekai Chen, Cheng Jin, Jingnan Zheng, Yi Zhang, Zhongtian Ma, Jiawei Zhou, Sirui Chen, Qiaosheng Zhang, Xiang Wang, Ning Ding, Xia Hu, Bowen Zhou, Youbang Sun, Chaochao Lu
The paper introduces ToxicGSM, a dataset of 1.9k arithmetic problems that embed harmful or sensitive context while keeping the math reasoning intact. It audits current large language models (LLMs) on this dataset, revealing how math word problems can subtly propagate bias, unethical, or psychologically harmful content, especially in educational settings for children. The authors propose SafeMath, a safety alignment technique that reduces harmful outputs without sacrificing, and sometimes improving, mathematical reasoning performance.
By Sagnik Basu, Subhrajit Mitra, Aman Juneja, Somnath Banerjee, Rima Hazra, Animesh Mukherjee
Flow Reasoning Models (FRMs) are a new framework that turns continuous flow models into efficient recurrent reasoners for structured tasks. By self‑conditioning a flow model on its own past outputs, FRMs iteratively refine solutions, allowing parallel decision making and revision. The authors introduce Fixed‑Point Forcing (FPF) to mitigate exposure bias at deeper recursion, and report near‑perfect solve rates on Sudoku‑Extreme, Zebra, and Maze‑Unique, outperforming existing masked‑diffusion and specialized baselines while using far fewer inference FLOPs.
By Alec Helbling, Andrey Bryutkin, Mauro Martino, Duen Horng Chau, Nima Dehmamy, Hendrik Strobelt
UI‑Venus‑2 is a general‑purpose foundation GUI agent that operates across mobile, web, and desktop environments using a unified closed‑loop reasoning‑action framework. The report details how the system expands environment coverage to over 170 multilingual mobile apps and native desktop OSes, scales task generation through a deep‑research pipeline, and enhances verification with trace‑level and sample‑level evaluators that use visual keypoints and multi‑model voting. Safety‑aware mechanisms are also incorporated to control consequential actions, positioning UI‑Venus‑2 as an efficient, open‑source tool for more generalizable, verifiable, and self‑reflective agents in real‑world applications.
By Venus Team, Zhuohan Cai, Haoxing Chen, Jiaxuan Chen, Weizhi Chen, Changlong Gao, Zhangxuan Gu, Yuan Guo, Yusong Hu, Jianrong Jiang, Jianguo Li, Runze Li, Jinzhen Lin, Zhenyu Ma, Changhua Meng, Han Peng, Xinyu Qiu, Shuheng Shen, Zhongyi Shui, Weiqiang Wang, Ming Wen, Zhuoer Xu, Hang Yan, Kaiwen Yang, Ruilin Yao, Nanjun Yu, Zhengwen Zeng, Lianrui Zhang, Yunzhu Zhang, Zhe Zhao, Beitong Zhou
The paper introduces a novel framework for assessing second‑order bias in large language models (LLMs), defined as bias in how an LLM judges the acceptability of biased content. Using principles from entitlement epistemology, the authors design a reasoning task that asks LLMs to determine whether a biased text is acceptable for specific demographic groups, and propose two metrics to quantify biased judgments. Experiments on both open‑source and closed‑source models reveal that the task bypasses safety guardrails, uncovers systematic variations across target groups, and demonstrates that models still rely on demographic labels when evaluating bias.
By Ramaravind Kommiya Mothilal, Terry Jingchen Zhang, Raiyan Ahmed, Zhijing Jin, Shion Guha, Syed Ishtiaque Ahmed
MUGEN is a framework that generates unlearnable graph examples capable of protecting multiple downstream tasks—node classification, graph classification, and link prediction—simultaneously. It achieves this by perturbing a single clean dataset with a shared GNN encoder and task‑specific heads, guided by a Task‑Aligned Separability Objective (TASO) and a Type‑Adaptive Perturbation (TAP) that handles both discrete and continuous node attributes. Experiments on five benchmarks, four GNN backbones, and three learning paradigms show that MUGEN’s perturbations transfer across models and remain effective even under adversarial training and data augmentation.
By Ziyan Liu, Chengshuai Zhao, Huan Liu
The paper investigates when online adaptation benefits edge time‑series forecasting under distribution drift, using a leakage‑free streaming protocol on six public multivariate datasets. It shows that the warmup budget for static baselines and the choice of learning rate can bias perceived adaptation gains, and that a validation‑only procedure selecting warmup and optimizer rates yields Adam outperforming SGD with momentum in most settings. The study also examines accuracy versus adaptation‑state memory and per‑update latency for different adaptation strategies, highlighting parameter‑efficient variants that are nondominated on the memory axis.
By Takumi Fujimoto, Hiroaki Nishi
arXiv:2609.00796v1 Announce Type: new
Abstract: As one of the most critical challenges in large language models, contextual faithfulness directly determines their reliability in knowledge-intensive a...
By Guanqiao Chen, Di Wang, Lijie Hu
The paper identifies a problem called Gradient-Update Mismatch (GUM), where optimizers can alter conflict-free gradient directions produced by gradient surgery, leading to conflicts between physics residual and boundary condition losses in Physics-Informed Neural Networks (PINNs). To address this, the authors propose Gradient-Update Alignment (GUA), which projects the optimizer’s update onto the conflict-free cone and adjusts internal optimizer state accordingly. Experiments show GUM is common across many optimizers, and GUA consistently eliminates conflicts and significantly reduces error in PINN training.
By Jing Xiao, Xinhai Chen, Qinglin Wang, Menghan Jia, Zhiquan Lai, Dongsheng Li, Jie Liu, Tiejun Li
The paper presents a cumulative turn‑based risk assessment framework for detecting financial scams targeting older adults, which aggregates conversational turns and updates risk estimates at each step. A multi‑turn dialogue dataset covering investment, charity, and tech support scams is created, with annotations for risk level, score, rationale, and safety recommendation at every cumulative stage. Four small language models (Phi‑4, LLaMA‑3.2, DeepSeek‑R1, Qwen3) are fine‑tuned; Phi‑4 and LLaMA‑3.2 outperform others in turn‑aware risk estimation, demonstrating that compact models can effectively support incremental scam detection in resource‑constrained, privacy‑aware deployments.
By Parviz Ghafariasl, Weimin Fu, Xiaolong Guo, Shing I. Chang
AI Morbidity and Mortality (AI M&M) is a structured, blameless framework designed to review clinical AI failures. It combines standardized case intake, evidence preservation, investigator reconstruction, tool‑in‑loop attribution, and corrective‑action tracking, classifying each event across four linked dimensions: Trigger, Mechanism, Clinical Pathway, and Corrective Action. The authors demonstrate the framework with five outpatient medication and clinical decision‑support cases, achieving full agreement among reviewers on all classification axes.
By Paulius Mui, Dean F. Sittig, Steve Labkoff, Sanjay Basu
The paper investigates how large language and visual‑language models used in autonomous vehicles inherit human driver biases when deciding whether to yield to pedestrians. It introduces two new bias‑testing methods—All Else Being Equal and Self‑Consistency tests—to evaluate these models. Results reveal that the models’ yielding decisions are influenced by pedestrian attributes such as gender, ethnicity, religion, disability, age, skin tone, and socio‑economic status, with varying patterns across models.
By Irem Yoldas, Martim Brand\~ao, Jie Zhang, Odinaldo Rodrigues