← Back to all news

AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,798 stories · RSS feed

arXiv Computer Vision
Oct 8

FaceKit: a Toolkit for Interpretable Facial Phenotyping, Synthetic Image Generation and Privacy Analysis in Rare Diseases

arXiv:2610.09288v1 Announce Type: cross Abstract: Many rare genetic diseases are associated with recognizable craniofacial features. However, traditional approaches for describing facial morphology r...

By Hongzhuo Chen, Zhanliang Wang, Florent Pollet, Mian Umair Ahsan, Joshua Bie, Tzung-Chien Hsieh, Peter Krawitz, Cong Liu, Wendy K Chung, Chunhua Weng, Gamze G\"ursoy, Kai Wang
diffusionsafety
More like this →
arXiv Computer Vision
Oct 8

Generative AI for Autonomous Driving: Frontiers and Opportunities

arXiv:2505.08854v2 Announce Type: replace Abstract: Generative Artificial Intelligence (GenAI) constitutes a transformative technological wave that reconfigures industries through its unparalleled ca...

By Yuping Wang, Shuo Xing, Cui Can, Renjie Li, Hongyuan Hua, Kexin Tian, Zhaobin Mo, Xiangbo Gao, Keshu Wu, Sulong Zhou, Hengxu You, Juntong Peng, Junge Zhang, Zehao Wang, Rui Song, Mingxuan Yan, Walter Zimmer, Xingcheng Zhou, Peiran Li, Fangzhou Lin, Peizheng Li, Zhaohan Lu, Chia-Ju Chen, Yue Huang, Ryan A. Rossi, Lichao Sun, Hongkai Yu, Zhiwen Fan, Frank Hao Yang, Yuhao Kang, Ross Greer, Chenxi Liu, Eun Hak Lee, Xuan Di, Xinyue Ye, Liu Ren, Alois Knoll, Xiaopeng Li, Shuiwang Ji, Masayoshi Tomizuka, Marco Pavone, Tianbao Yang, Jing Du, Ming-Hsuan Yang, Hua Wei, Ziran Wang, Yang Zhou, Jiachen Li, Zhengzhong Tu
llmsagentsdiffusionroboticsmultimodalsafety
More like this →
arXiv Machine Learning
Oct 8

Average-Reward Reinforcement Learning for Multichain MDPs: A Hierarchical Decomposition Approach

The paper introduces an asynchronous value‑iteration RL algorithm for average‑reward multichain MDPs that requires only the transition graph and uses Bather’s decomposition to partition the state space into communicating subsystems and transient states. This hierarchical decomposition transforms the global decision problem into structured subproblems, enabling the algorithm to converge to the optimal gain and produce gain‑optimal policies in finite time. Two additional algorithms are presented: one that approximately solves the multichain average optimality equations for near gain‑optimality, and another that approximates the optimal bias function to achieve near bias‑optimality, both with almost‑sure convergence guarantees and improved transient performance. whyItMatters":"The work offers the first essentially model‑free average‑reward RL algorithms for general multichain MDPs, avoiding reductions to discounted problems."

By Huizhen Yu, Isaiah Heidt
reinforcement-learningsafety
More like this →
arXiv Machine Learning
Oct 8

Adversarial RL for Port-Scan Evasion: Attacker Feature Visibility in Edge-Deployed IDS

The paper studies how an adaptive attacker can evade machine‑learning intrusion detection systems (IDS) deployed on a Raspberry Pi 3B+ by using a Deep Q‑Network (DQN) to learn port‑scan tactics that vary probe timing, TCP flags, and payload size. Three IDS models—XGBoost, a multi‑layer perceptron, and a 1D convolutional neural network—trained on TON_IoT telemetry were evaluated under black‑box, gray‑box, and white‑box feature‑visibility scenarios. Evasion success rates ranged from 61.9 % to 98.3 %, showing that even limited feature knowledge can enable high‑rate evasion and that the impact of feature visibility depends on the specific IDS model.

By Logan Andrew North, Priya Sanjay Kaluskar, Shasi Kumar Ramachandran Prabhu, Peilong Li, Suman Saha
agentsreinforcement-learningsafety
More like this →
arXiv Computer Vision
Oct 8

HDR Video Generation via Latent Alignment with Logarithmic Encoding

arXiv:2604.11788v2 Announce Type: replace Abstract: High dynamic range (HDR) imagery offers a rich and faithful representation of scene radiance, but remains challenging for generative models due to...

By Naomi Ken Korem, Mohamed Oumoumad, Omer Hagage, Amir Gam, Matan Ben Yosef, Harel Cain, Urska Jelercic, Ofir Bibi, Yaron Inger, Or Patashnik, Daniel Cohen-Or
diffusionfine-tuningsafety
More like this →
arXiv Machine Learning
Oct 8

ORCA: Hunting Compositional Failures in Text-to-Image Diffusion

ORCA (Orthogonal Residual Compositional Alignment) is a method that improves text-to-image diffusion models by aligning the diffusion transformer’s latent space with a low‑rank target derived from a frozen visual encoder. It introduces an auxiliary loss that uses a predictor with an orthogonal basis parameterised by a learned residual between T5 and CLIP embeddings, providing a prompt‑dependent signal for selecting the visual readout subspace. Experiments on three diffusion‑transformer backbones show that ORCA improves FID and GenEval scores, especially on attribute binding, spatial relations, and multi‑object prompts, without adding inference‑time cost.

By Arshia Hemmat, Amirhossein Vahidi, Amitis Shidani, Mohammad Vali Sanian, Hesam Asadollahzadeh, Aryan Yazdan Parast, Mohammad Lotfollahi
llmsragdiffusionmultimodalsafety
More like this →
arXiv Computation and Language
Oct 8

Is Word Error Rate Enough? Rethinking Privacy Evaluation in Speech with Entity-Aware Metrics

The paper examines whether word error rate adequately measures privacy in speech by applying entity‑aware metrics from NLP to evaluate two obfuscation techniques. It focuses on protecting named entities and explores how fine‑tuning on entity‑rich data affects attack success across different entity categories. The authors also offer guidance on choosing metrics depending on whether the obfuscation preserves temporal alignment.

By Anjana Rajasekhar, Jule Pohlhausen, Nayana Jacob Alappattu, Anna Leschanowsky
nlpfine-tuningsafety
More like this →