AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,798 stories · RSS feed

arXiv Machine Learning
Oct 9

CSF: Contextual Safety Filtering for Motion Generators

The paper introduces Contextual Safety Filtering (CSF), a training‑free method that enforces natural‑language safety rules on text‑conditioned motion generators by comparing generated trajectories to safe and unsafe reference paths. CSF uses an affine safety value derived from these references and a safe reference tracking CBF‑QP to activate rules in both explicit and scene‑triggered unsafe scenarios. Experiments on four pretrained generators show CSF reduces danger‑event rates by up to 90% while preserving 88‑100% of benign motions, and a real‑world Unitree G1 robot demonstrates successful prevention of unsafe motions in diverse human and object interaction settings.

By Lizhi Yang, Yiling Hou, Yao Tang, Junheng Li, Daniel Weng, Blake Werner, Aaron D. Ames
arXiv Machine Learning
Oct 9

The Sample Complexity of Membership Inference and Privacy Auditing

The paper investigates the sample complexity required for membership‑inference attacks, focusing on Gaussian mean estimation. It shows that, in this setting, an attacker may need at least Ω(n + n^2 ho^2) reference samples to match a fully informed adversary, which can be far more than the n samples used by the learning algorithm. This demonstrates that practical attacks, which typically use only O(n) samples, might underestimate the true risk of membership inference.

By Mahdi Haghifam, Adam Smith, Jonathan Ullman
arXiv Machine Learning
Oct 9

Self-sufficient Independent Component Analysis for Demixing Flows

The paper introduces a method for learning disentangled signals via non‑linear Independent Component Analysis (ICA) by defining self‑sufficient signals, where the conditional distribution of a missing value given the rest remains unchanged when other signals are observed. It formulates this as a conditional KL divergence minimization and proposes a sequential, prior‑free, likelihood‑free algorithm that learns a demixing flow model at each iteration. The authors provide theoretical guarantees of local descent for an idealized Wasserstein‑gradient‑flow variant and demonstrate the method’s effectiveness on toy and real‑world datasets.

By Song Liu
arXiv Machine Learning
Oct 9

V-ECE: Estimating General Expected Calibration Errors

The paper introduces V-ECE, a variational estimator for general Expected Calibration Errors (ECE) that avoids binning and works with any convex divergence, including Lp distances. By using prediction‑dependent proper scores and a residual‑fitted temperature scaling via gradient boosting, V‑ECE provides a lower‑bound estimate of true calibration error. Experiments on semi‑synthetic benchmarks show V‑ECE outperforms existing binary and multiclass estimators, with additional theoretical insights on bias and confidence estimation.

By Eug\`ene Berta, Sacha Braun, Francis Bach, Michael I. Jordan, David Holzm\"uller
arXiv Computation and Language
Oct 9

Diffu-LoRA: A Novel Low-Rank Adaptation for Personalized Diffusion Models

Diffu-LoRA is a parameter‑efficient method for personalizing text‑to‑image diffusion models. It inserts trainable low‑rank components into Transformer linear layers and assigns a learnable gate to each component, allowing the model to learn how to distribute adaptation capacity across layers. Using bilevel optimization and progressive pruning, Diffu-LoRA keeps the pretrained backbone frozen while improving subject fidelity and prompt alignment compared to fine‑tuning baselines.

By Tianjing Li, Wei Zhu
arXiv Computation and Language
Oct 9

Large Language Model-Assisted Preparation of Transportation Management Plans: A Case Study with WisDOT WisTMP System

The paper presents a Large Language Model (LLM)-assisted framework for automating the creation of Transportation Management Plans (TMPs) using the WisDOT WisTMP system. It fine‑tunes several open‑source LLMs of varying sizes on a domain‑specific dataset derived from historical WisTMP documents, and deploys the models locally to maintain data security. Experiments show that fine‑tuning improves text‑generation metrics, though LLMs still over‑generate strategies and struggle with project‑specific justifications and cost estimates, and scaling from 7B/8B to 14B models offers limited gains.

By Zihao Sheng, Pei Li, Zilin Huang, Yen-Jung Chen, Yuhao Luo, Zhengyang Wan, Steven T. Parker, David A. Noyce, Sikai Chen
arXiv Computation and Language
Oct 9

Local Prototype Reconstruction for Text-Compatible Speech-to-LLM Bridge Pretraining

The paper introduces Local Prototype Reconstruction (LPR), a lightweight regularizer that encourages bridge embeddings in speech‑to‑LLM systems to stay close to the frozen LLM’s input‑embedding neighbourhoods. By measuring lexical manifold compatibility with a fixed, head‑free, timestamp‑free diagnostic, the authors show that next‑word prediction and sentence‑level contrastive pretraining do not fully capture token‑level lexical compatibility. Experiments on multilingual ASR and speech translation demonstrate that LPR improves transfer, especially for translation and low‑resource adaptation, and that the diagnostic correlates with downstream gains across objectives.

By Xinnian Zhao, Chia-Hua Wu, Pu Wang, Hugo Van Hamme
arXiv Computation and Language
Oct 9

When Do We Need On-Policy Distillation? Distilling on Offline Student Rollouts Is Often Better

The paper investigates whether on‑policy distillation (OPD) is always advantageous for transferring knowledge from a teacher to a student model. It introduces Semi‑OPD, which uses offline rollouts from the initial student, and demonstrates that across 17 teacher‑student pairs (1.5B–235B parameters) Semi‑OPD outperforms OPD in 14 cases, achieving up to +13.6% accuracy and an 11.4× speedup. The authors show that the benefit of OPD depends on the alignment between teacher and student, measured by an output‑token overlap ratio, and argue that effective distillation requires on‑policy behavior relative to both models.

By Siyan Zhao, Yonggan Fu, Jindong Jiang, Shih-Yang Liu, Song Bian, Byung-Kwan Lee, Sharath Turuvekere Sreenivas, Wenliang Dai, Hanrong Ye, Aditya Grover, Pavlo Molchanov
arXiv Computation and Language
Oct 9

Beyond Sequences: Distilling Structured Decision Memory for LLM Recommendation

The paper introduces MARI, a Memory-Augmented Recommendation system that uses a Decision Memory Bank to store Structured Decision Memories (SDMs) capturing users’ goals, constraints, and trade-offs. Unlike prior methods that flatten heterogeneous behaviors into token sequences, MARI retrieves relevant SDMs to enhance LLM reasoning, improving performance on next-item prediction and a new Difficult Choice Prediction task while keeping latency low. The approach also provides interpretable, human-readable insights into user decision-making.

By Leikun Liang, Guoshuai Wang, Xingsheng He, Yushan Han, Yunyi Xuan, Xiaoxiao Xu, Lin Qu
arXiv Computer Vision
Oct 9

Collaboratively Guided Adversarial Robust Distillation with Teacher-Favorable Examples

Collaboratively Guided Adversarial Robust Distillation (CGARD) is a method that transfers robustness from high‑capacity teachers to compact students by jointly optimizing student‑adversarial and teacher‑collaborative examples within the same perturbation neighborhood. The teacher‑collaborative example is constrained so that the teacher’s cross‑entropy loss does not exceed that of the clean input, combining collaborative guidance with adversarial supervision. Experiments on CIFAR‑10 and CIFAR‑100, including white‑box and black‑box evaluations, show consistent robustness improvements over strong adversarial distillation baselines.

By Zhi Li, Haowei Liu, Hongchen Yang, Xiaoxuan Wang, Song Gao, Shaowen Yao, Wei Zhou
arXiv Computer Vision
Oct 9

Learning Which Correspondences to Trust: Confidence-Weighted Event-Camera Localization in LiDAR Maps

The paper introduces CELL, a confidence-weighted event-camera localization method that learns per-correspondence confidence end-to-end using a differentiable probabilistic PnP. It reweights flow supervision, drives probabilistic correspondence selection at test time, and refines edge matching, while also proposing a partial-completion depth representation. Experiments on M3ED and DSEC show significant reductions in median translation and rotation errors compared to the LEAR baseline.

By Panagiotis Kiousis, Kuangyi Chen, Jun Zhang, Friedrich Fraundorfer
arXiv Computer Vision
Oct 9

Spatial-Frequency-Aware Implicit Neural Representation of Multidimensional Signals via MLP-KAN Fusion

The paper introduces a spatial-frequency-aware implicit neural representation that fuses a Multi‑Layer Perceptron (MLP) branch with a Kolmogorov‑Arnold network (KAN) branch. The MLP captures low‑frequency, smooth structures while the KAN captures high‑frequency, localized details, and their outputs are combined using discrete wavelet transform (DWT) and inverse DWT. A wavelet‑domain band‑separation regularization encourages complementary frequency behavior, and experiments on 1D–4D signals show improved reconstruction fidelity.

By Wen Yan, Ligen Shi, Jun Qiu, Haimiao Zhang, Lina Wu, Chang Liu
arXiv Computer Vision
Oct 9

Seek-and-View Reasoning for Multi-View Spatial Understanding

The paper introduces Vantage, a training‑free, model‑agnostic framework that pairs a vision‑language model with a 3D foundation model to perform Seek‑and‑View reasoning for multi‑view spatial understanding. Vantage first analyzes a question to plan a relevant viewpoint, then augments reasoning with geometry‑grounded visual evidence from that viewpoint. Experiments across six VLMs on five benchmarks show consistent improvements without fine‑tuning, demonstrating that revealing spatial evidence through view‑grounded reasoning reduces reliance on language‑based cross‑view alignment.

By Qixiang Chen, Cheng Zhang, Fucai Ke, Chi-Wing Fu, Jianfei Cai, Jingwen Ye
arXiv Computer Vision
Oct 9

Does Target Alignment Mean Target Recovery? An Evidence-Ladder Study of Adversarial Claims on Contrastive Encoders

The paper investigates whether the similarity score (victim‑space target alignment, VTS) reported by adversarially attacked vision‑language models actually indicates that the target text can be recovered by an independent model. Using a new measurement instrument that includes supervised judges, blend controls, shuffled negatives, and a reference level, the authors conduct two preregistered studies across six contrastive encoders and three perturbation budgets. Robustly trained encoders (FARE, TeCoA, PMG, TRADES) transfer more independent evidence than vanilla CLIP or SigLIP, yet none reach the blend‑derived reference level, leaving practical recovery uncertain; VTS is only informative within a fixed robust encoder, prompting the authors to propose a new reporting protocol.

By Tao Yang, Jianying Zhou
arXiv Computer Vision
Oct 9

Right Screen, Wrong Transition: World Models as Verifiers for GUI Agents

The paper introduces LGWM, a decoder‑free, action‑conditioned world model that predicts the representation of the next GUI screen directly, enabling efficient verification of transitions without semantic annotation. By comparing predicted and observed representations, the model achieves high AUC scores on the RSWT‑BENCH diagnostic, outperforming existing generative world models and matching closed‑source VLMs while operating at much lower latency. The study demonstrates that world models can serve a third role—verification—by predicting in representation space rather than generating text, code, or images.

By Jiaming Zhang, Xuan Wang, Fuyao Zhang, Yang Cao, Lingjuan Lyu, Wei Yang Bryan Lim
arXiv Computer Vision
Oct 9

WorldAlign: Decoupled 4D Reward for World-Consistent Video Generation

WorldAlign introduces a decoupled 4D reward framework for improving world consistency in generated videos. It separates static regions from dynamic subjects, aligning each with a suitable world prior—geometric priors for static geometry and a vision‑language model for dynamic subjects—using masked reprojection and a VLM‑as‑a‑judge reward. The method enhances both static and dynamic consistency in pretrained image‑to‑video generators without relying on human preference data.

By Jing He, Kaixin Ding, Xingye Tian, Guibao Shen, Wenhang Ge, Xin Tao, Pengfei Wan, Ying-Cong Chen
arXiv Computer Vision
Oct 9

LEGO: A Lifting-Free Approach for Exocentric-to-Egocentric Video Generation

The paper introduces LEGO, a lifting‑free method for generating egocentric videos from a single exocentric recording. Instead of reconstructing depth and point clouds, it trains a transformer‑based view synthesizer that directly predicts the egocentric view, providing a probabilistic correspondence map that balances structural alignment with texture averaging. This approach outperforms existing explicit pipelines and generalizes across datasets without retraining.

By Suhwan Cho, Yonwoo Choi, Soongjin Kim, Jicheol Park, Taegyu Lim
arXiv Computer Vision
Oct 9

Image Recognition with Vision and Language Embeddings of VLMs

The paper evaluates how vision-language models (VLMs) perform in image classification tasks, comparing language-guided and vision-only approaches across various dual-encoder VLMs such as SigLIP 2 and RADIOv2.5 on ImageNet-1k and its label-corrected variant. It identifies key factors influencing accuracy—including prompt design, class diversity, k‑NN neighbors, and reference set size—and demonstrates that language and vision provide complementary strengths. A simple, learning‑free fusion method based on per‑class precision is proposed to further improve classification performance.

By Illia Volkov, Nikita Kisel, Klara Janouskova, Jiri Matas
arXiv Computer Vision
Oct 9

Comparing Object Detection Models for Electrical Substation Component Mapping

The paper evaluates 16 computer‑vision models for mapping electrical substation components, training them on a manually labeled dataset of U.S. substation images. The best performer, RF‑DETR‑large, achieved mAP@50 of 0.881 and mAP@50:95 of 0.632, with alternate energy systems detected most accurately and transformers/reactors posing the greatest challenge. Applying this model nationwide produced about 22,591 component detections across 11,083 substations, with Florida and the MISO region showing the highest counts.

By Namish Bansal (George Mason University), Haley Mody (George Mason University), Dennies Kiprono Bor (George Mason University), Dante Groccia (George Mason University), Edward J. Oughton (George Mason University)
arXiv Computer Vision
Oct 9

LearnPruner: Rethinking Attention-based Token Pruning in Vision Language Models

LearnPruner rethinks token pruning in vision‑language models by addressing shortcomings of attention mechanisms in vision encoders and large language models. The method uses a two‑stage approach: first a learnable pruning module removes redundant vision tokens after the vision encoder, then only task‑relevant tokens are kept in the LLM’s middle layer. Experiments show that LearnPruner retains about 95% of performance while using only 5.5% of vision tokens, achieving a 3.2× speed‑up.

By Rinyoichi Takezoe, Yaqian Li, Zihao Bo, Anzhou Hou, Mo Guang, Kaiwen Long