AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,798 stories · RSS feed

arXiv Machine Learning
Oct 5

Context-Tower Conversion Preserves Generation While Freezing Retains Knowledge: Low-Budget AR-to-Diffusion Conversion of MoE LLMs

The paper compares two low‑budget methods for converting a 30B Mixture‑of‑Experts autoregressive language model into a diffusion language model. One method updates a subset of the model’s weights in‑place, while the other freezes the context tower and conditions on a frozen causal copy via cross‑attention. With only 1B training tokens, the frozen‑tower approach achieves a HumanEval pass@10 score of 71.60 versus 6.19 for the in‑place method, and retains 95% of the parent’s GSM8K and 99% of its MMLU‑Pro performance.

By Wentao Lu, Jesse Clark, Tianyu Zhu
arXiv AI
Oct 5

LPS-Bench: Benchmarking Safety Awareness of Computer-Use Agents in Long-Horizon Planning under Benign and Adversarial Scenarios

LPS-Bench is a benchmark designed to evaluate the safety awareness of computer‑use agents (CUAs) in long‑horizon planning tasks that involve tool workflows. It uses a template‑guided multi‑agent pipeline to generate user instructions, simulated toolkits, and case‑specific safety criteria, followed by human review, allowing scalable expansion without building separate application environments. The benchmark includes 570 cases from 65 scenarios across seven task domains and nine planning‑risk types, and an LLM‑based evaluator assesses tool choices, arguments, and responses throughout execution. Evaluations of 13 LLM agents show persistent safety failures in both benign and adversarial settings, with prompt‑based interventions providing only model‑dependent improvements.

By Tianyu Chen, Chujia Hu, Dongrui Liu, Xia Hu, Wenjie Wang
arXiv Computer Vision
Oct 5

TerrainForge: Physics-Grounded road geometry Editing for Counterfactual Autonomous Driving

TerrainForge is a framework that generates counterfactual road geometry edits—such as crests, sags, speed humps, and friction changes—by linking a unified road model to four‑wheel vehicle dynamics. It propagates these edits through vehicle motion, camera viewpoint, and inter‑vehicle clearance, and validates the dynamics against CarSim and reconstructed Waymo scenes. The system also simulates thousands of road edits to train a surrogate model that predicts changes in the ego vehicle’s terminal gap, enabling efficient screening of safety‑critical edits.

By Yang Chen, Yicheng Zhu, zhenning Li, Tao Li, Zilin Bian
arXiv AI
Oct 5

Certified Mechanistic Edits: Behavioral Guarantees for Skill Removal and Preservation

Certified Mechanistic Edits: Behavioral Guarantees for Skill Removal and Preservation proposes a method to formally certify that disabling a specific circuit in a neural network removes a targeted skill while preserving another, for every input within a continuous region. The approach extends beyond testing by providing sound guarantees over continuous embedding-space regions, demonstrated on toy ReLU networks and a standard transformer. It also shows that no finite deterministic black-box test can guarantee removal, highlighting the necessity of formal verification.

By Md Sazid Uddin, Md. Khairul Alam Mazumder, M. F. Mridha
arXiv Computer Vision
Oct 5

ProAR: Learning Prospective Reasoning with Autoregressive Video Models

ProAR introduces a goal‑oriented reasoning framework for autoregressive video models, adding goal‑frame prediction via an asymmetric attention mask and future representation self‑alignment to guide intermediate states. These mechanisms combine sparse target supervision with dense step‑wise guidance, enabling coherent, goal‑directed generation while keeping computational costs modest. Experiments demonstrate consistent performance gains across visual reasoning benchmarks and training efficiency, achieving comparable results to fully trained baselines with only 25% of the training steps.

By Linghui Shen, Tinghui Zhu, Sheng Zhang, Muhao Chen
arXiv AI
Oct 5

Morality is Contextual: Learning Interpretable Moral Contexts from Human Data with Probabilistic Clustering and Large Language Models

The paper introduces COMETH, a framework that learns interpretable moral contexts from human judgments using probabilistic clustering and large language models. It builds a dataset of 300 scenarios involving six core actions and three moral rules, and employs an LLM-based preprocessing pipeline to cluster actions and scenarios. COMETH achieves roughly double the alignment with human judgments compared to end‑to‑end LLM prompting, while providing transparent explanations of the contextual features driving its predictions.

By Geoffroy Morlat, Marceau Nahon, Augustin Chartouny, Raja Chatila, Ismael T. Freire, Mehdi Khamassi
arXiv AI
Oct 5

Many Preferences, Few Policies: Compact Portfolios for Multi-Objective LLM Alignment

The paper introduces PALM, an algorithm that constructs a compact portfolio of aligned large language models (LLMs) capable of near‑optimal performance across a wide range of reward weightings for objectives like helpfulness, harmlessness, and conciseness. PALM uses a structured grid of weight vectors, lazy search, and pruning to efficiently identify a small set of policies that provably cover all target reward configurations within specified tolerances. Experiments demonstrate that PALM achieves smaller approximation gaps than portfolios built from uniformly spaced or randomly sampled weights and scales effectively to higher‑dimensional reward spaces.

By Cheol Woo Kim, Jai Moondra, Roozbeh Nahavandi, Andrew Perrault, Milind Tambe, Swati Gupta
arXiv AI
Oct 5

DNAlign: Dynamic Null-Space Safe Alignment for LLMs

DNAlign is a lightweight alignment framework that uses control‑theoretic optimization and null‑space projection to steer large language models toward safe behavior while preserving core knowledge and response quality. By treating the LLM as a dynamic system, it introduces controllable perturbations that are restricted to a harmful‑related subspace derived from neutral hidden states, and a value function trained on human preference data adaptively optimizes these control signals. Extensive evaluations across multiple LLM backbones show that DNAlign consistently reduces harmful outputs while maintaining fluency, coherence, and factual utility, outperforming prior alignment baselines without sacrificing generation diversity.

By Jisheng Dang, Yushuo Zhao, Dewei Liu, Junfeng Fang, Bimei Wang, Tiantian Rao, Hong Peng, Bin Hu, Tat-Seng Chua
arXiv Computer Vision
Oct 5

OpenBox: Annotate Any Bounding Boxes in 3D

OpenBox is a two‑stage automatic annotation pipeline that uses a 2D vision foundation model to associate 2D image cues with 3D point clouds. It then classifies instances by rigidity and motion state to generate adaptive bounding boxes using class‑specific size statistics, eliminating the need for self‑training. Experiments on Waymo Open Dataset, Lyft Level 5 Perception, and nuScenes show improved accuracy and efficiency over existing baselines.

By In-Jae Lee, Mungyeom Kim, Kwonyoung Ryu, Pierre Musacchio, Jaesik Park
arXiv AI
Oct 5

Assessing Rule Adherence of LLM Adjudicators in Call of Cthulhu TRPG

The paper introduces CoC‑Seduce, a multi‑agent adversarial benchmark for evaluating how well large language models (LLMs) adhere to the rules of the tabletop role‑playing game Call of Cthulhu when acting as adjudicators. Using three LLMs to generate 5,376 samples across diverse settings and skill categories, the study benchmarks 22 target adjudicators and finds that newer releases and explicit reasoning do not guarantee robustness, with pseudo‑logic framing emerging as the most effective manipulation. The benchmark highlights the vulnerability of LLM adjudicators to rhetorical injection attacks that exploit narrative framing to bypass rule enforcement.

By Weiying Chen, Junlong Shen, Zhanyuan Guo, Xiaoou Zhou
arXiv AI
Oct 5

Safeguarding Mutual Correction in Source-Free Domain Adaptation via Cut Statistics

The paper introduces SafeCut, a method for Source-Free Domain Adaptation that uses Vision‑Language models as external knowledge. SafeCut employs a cut statistic to gauge prediction reliability, enabling a dynamic, reliability‑gated supervision between the source‑pretrained model and the ViL model. This approach selectively amplifies correct mutual corrections while suppressing error propagation, achieving state‑of‑the‑art performance on multiple SFDA benchmarks.

By Seongjun Lee, Changhee Lee
arXiv Machine Learning
Oct 5

PowerBench: Measuring Language Model Bias in Power-shifting Requests

PowerBench is a new evaluation framework that measures how language models respond to power‑shifting requests, distinguishing self‑empowerment, disempowerment, and power grabbing while controlling for neutral requests. The authors built and released a dataset covering different power domains, contexts, scales, and prior power standings, and tested 24 models from US and Chinese developers across three experimental conditions. Results show models are more likely to refuse power‑grabbing than disempowerment, and that biases exist toward helping US users gain power while resisting US users taking power from others, with additional effects when the user is an AI agent or when requests are in different languages.

By Nicolas Martorell, Wendy Brau, Gonzalo A. Heredia, Tom\'as Pablo Korenblit, Gaspar Labasti\'e, Tom\'as Gimenez Molina
arXiv Computation and Language
Oct 5

Writerslogic at PAN 2026: Process over Content for Robust Detection under Domain Shift

The paper presents the Writerslogic systems for three PAN 2026 shared tasks—Reasoning Trajectory Detection, Voight‑Kampff Generative AI Detection, and Multi‑Author Writing Style Analysis—using a unified analytical framework that prioritizes feature robustness under distribution shift. The framework distinguishes domain‑anchored, domain‑portable, and domain‑invariant features, explaining why generator‑specific traits fail while vocabulary fingerprints, compression measures, and character n‑grams remain effective. The authors report first‑place source detection and third‑place safety classification on Reasoning Trajectory Detection, a top‑scoring ensemble for Voight‑Kampff, and a detailed design for Multi‑Author Writing Style Analysis that was not evaluated due to a platform mix‑up.

By David L. Condrey
arXiv AI
Oct 5

Mitigating Private Data Leakage in LLMs with Whiteout

The paper introduces Whiteout, a tool that prevents large language models from revealing personally sensitive information (PSI) by overwriting such data with carefully crafted obfuscation samples. Whiteout is evaluated on various LLMs, including an OpenAI model, and shows effective PSI protection with minimal impact on model utility and safety. The study also tests Whiteout against multiple attack vectors and discusses its security and ethical implications.

By Anna Yoo Jeong Ha, Ronik Bhaskar, Haitao Zheng, Ben Y. Zhao
arXiv Machine Learning
Oct 5

Differentiable Koopman Operator for Contrastive Learning on Dynamic Graphs

The paper introduces KAIROS, a self‑supervised framework that embeds a differentiable Koopman operator into a dynamic graph contrastive learning loop. By linearizing temporal evolution in the embedding space, KAIROS jointly optimizes raw node features and a graph‑diffused structural view using multi‑granularity contrastive objectives across time windows. For anomaly detection, it combines Koopman prediction residuals with temporal inconsistency and local neighborhood deviation, achieving state‑of‑the‑art results on nine dynamic graph benchmarks and competitive performance on unsupervised node classification.

By Md Abrar Jahin, Taufikur Rahman Fuad, Md Rizwan Parvez
arXiv AI
Oct 5

Follow the Winners: Conservative Policy Improvement with the Cross-Entropy Method for Critic-Free RFT

The paper introduces Follow the Winners (FTW), a critic‑free policy‑learning algorithm that adapts the cross‑entropy method for reinforcement fine‑tuning of agentic large language models. FTW replaces group rollouts with an ordinal filter on replay‑buffer samples, achieving polynomial concentration in the order statistic of returns and offering a bounded risk‑seeking offset that balances variance reduction. Experiments on Sokoban and Search‑R1 show that FTW matches the performance of GRPO and PPO while reducing reliance on value models or repeated rollouts.

By Joery Ari\"en de Vries, Neil David Lawrence, Zhenwen Dai
arXiv Computer Vision
Oct 5

EVEWorld: Physical Evolution Supervision for Embodied World Models

EVEWorld introduces a physical evolution-supervision framework for embodied world models, addressing the issue of Model Laziness by focusing on physical consistency rather than visual fidelity. The framework comprises Instance-Guided Restoration (IGR) to enforce instance consistency and Temporal Instance Alignment (TIA) to align target instances across adjacent frames. Experiments on DreamGenBench, EWMBench, and PBench show an 87.5% reduction in the Model Laziness Rate (MLR) compared to GigaWorld-0, and the model ranks 6th in JEPA Similarity on the WorldArena 2.0 Track 1 leaderboard.

By Kaiqi Wang, Songxin Zhang, Zejian Xie, Xiao Xiong, Zhuoyang Song, Ziwei Wu, Jun Yu Lu, Yitan Teng, Ziying Song, Jiaxing Zhang
arXiv Computer Vision
Oct 5

Kinematics-Induced Multimodal 3D Human Pose Estimation with Subject-Level Privacy

The paper introduces a unified framework for multimodal 3D human pose estimation that fuses RGB, LiDAR, and mmWave radar data while incorporating kinematics-based sensor fusion. It presents a black-box subject membership inference attack and a pointwise maximal leakage analysis to assess privacy risks, and proposes a user-level differential privacy method called Action Temporal Stratification to mitigate these risks. The framework is evaluated on the MM-Fi dataset under three experimental protocols, with source code to be released upon acceptance.

By Kaushik Bhargav Sivangi, Fani Deligianni