AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,884 stories · RSS feed

arXiv Computer Vision
Sep 3

Stereo 4D Radar for 3D Object Detection: Integrating Geometric Alignment and Absolute Velocity Estimation

The paper presents a stereo 4D Radar framework for 3D object detection that uses geometric disparity between left and right radars to estimate absolute velocity and fuse complementary features. It addresses clutter, ghost reflections, and sparse data issues inherent in raw 4D Radar signals. Experiments on an in‑house dataset show significant gains, improving AP 3D by 8.82 points and AP BEV by 9.0 points over mono‑radar baselines.

By Seung-Hyun Song, Dong-Hee Paek, Woong-Chan Byun, Seung-Hyun Kong
arXiv Computer Vision
Sep 3

AffectDelta: Beyond Emotion Labels for Image Editing

AffectDelta is a new image editing framework that moves beyond single emotion labels by modeling edits as transitions between eight‑dimensional emotion distributions. It uses a frozen Emotion Distribution Predictor to estimate the source state and a signed difference vector to encode the desired change, which is then translated into context‑dependent semantic and appearance modifications via a transition encoder and a diffusion backbone. The authors introduce AffectPair‑249K, a dataset of 248,841 source‑target pairs covering both cross‑category and within‑category transitions, and show that AffectDelta outperforms six baselines in affective alignment and content preservation.

By Xingzu Zhan, Lin Gu, Ruogu Fang
arXiv AI
Sep 3

LLM-as-a-Judge Is Not an Oracle: Why Self-Improving Agents Need Deterministic Guardrails

The paper argues that using a large language model (LLM) as the sole judge in self‑improving agent pipelines is problematic, as the judge can be biased or manipulated, leading to false confidence in system performance. The authors propose a new framework, PROCTOR, which replaces the oracle judge with a deterministic, teacher‑student loop that enforces guardrails such as sandboxing, role separation, and acceptance checks to prevent cheating and ensure reliable evaluation. Experiments across contract analysis, compliance review, and code quality demonstrate that PROCTOR mitigates eleven identified failure modes that previously allowed agents to achieve perfect scores while hiding significant capability gaps.

By Vansh Wahi
arXiv AI
Sep 3

A Data-Driven Multimodal Method for Early Detection of Coordinated Abnormal Behaviors in Live-Streaming Platforms

arXiv:2609. 01649v1 Announce Type: cross Abstract: With the rapid growth of live-streaming e-commerce and digital marketing, abnormal marketing behaviors have become increasingly concealed and coordinated across heterogeneous modalities, challenging platform governance and early risk identification.

By Jingwen Luo, Pinrui Zhu, Yiyan Wang, Zilin Xiao, Jingqi Li, Xuebei Kong, Yan Zhan
arXiv AI
Sep 3

Convergence Theory of Knowledge Distillation in Asynchronous P2P Gossip Learning Network

arXiv:2609. 01952v1 Announce Type: cross Abstract: Decentralized, serverless learning increasingly connects devices running different architectures, where the standard tool, decentralized SGD, is undefined as models with different parameter counts cannot be averaged.

By Lucas Qingyang Fang, Tiyao Liu, Jinhao Jing, Zeji Li, Kaijie Chen, Harikrishna Kuttivelil, Katia Obraczka
arXiv AI
Sep 3

OBJECTION! Lawyer Agents Mitigate Guilty Bias in Legal Judgment Prediction

The paper introduces OBJECTION, an inference-time pipeline that adds an Adversarial Lawyer Agent to each of the three reasoning steps—offense, unlawfulness, and culpability—in legal judgment prediction models. By actively injecting defense arguments, the agent challenges the model’s default assumption of guilt, which is common in datasets biased toward guilty outcomes. Using a new Natural Innocent dataset of 3.4k real cases, OBJECTION reduces the False Guilty Rate from 82.93% to 16.69%, demonstrating significant improvement in substantive legal reasoning.

By Jaehoon Jeong, Jay-Yoon Lee
arXiv AI
Sep 3

TUX: Measuring Human--AI Tacit Understanding

The paper introduces TUX, a Tacit Understanding Index that measures how similarly humans and large language models (LLMs) place concepts along subjective spectra in a task inspired by the game Wavelength. Using 241 human participants and 200 profile-conditioned LLM agents across four models, the study finds that human–agent pairs with similar traits achieve higher TUX scores, indicating that tacit alignment is linked to person-level characteristics. Regression analyses show that richer predictor sets—including individual traits, decision-making styles, and confidence—improve the explainability of TUX beyond simple trait-distance baselines.

By Yueshen Li, Hanyi Min, Vedant Das Swain, Koustuv Saha
arXiv Machine Learning
Sep 3

Similarity-Aware Personalized Federated Learning in Heterogeneous Environments

The paper introduces SAPE-FL, a personalization framework for Federated Learning that anchors each client’s model to both a global model and a similarity-weighted peer-averaged model. By applying dynamic, client-specific regularization based on model and output similarity, SAPE-FL balances global knowledge transfer with peer collaboration, filtering out dissimilar clients. The authors provide theoretical convergence guarantees and demonstrate empirically that SAPE-FL outperforms state‑of‑the‑art methods in highly heterogeneous and low‑data scenarios.

By Arun Kumar A V, Sunil Gupta, Dang Ngyuen, Bao Duong, Dat Phan Trong
arXiv Machine Learning
Sep 3

Online Reinforcement Learning in the Met Office Unified Model through Distributed Model-Agent Coupling

The study couples the Met Office Unified Model with distributed reinforcement learning agents, using a DDPG actor that applies bounded potential‑temperature corrections across 70 vertical levels. Training is performed on ten nudged forecasts, after which the frozen policy is evaluated in a non‑nudged forecast, demonstrating numerical stability. The learned policy reduces Z₅₀₀ MAE in four of six latitude bands—up to 45.8% in the northern tropics—and decreases MSLP error by up to 27.3% in certain bands, indicating promising bias‑correction potential.

By Pritthijit Nath, Sebastian Schemm, Peter Haynes, Emily Shuckburgh, Mark Webb
arXiv Machine Learning
Sep 3

Breadth Beats Depth: Improving GCG-Based Jailbreak Optimization with Breadth-Oriented Suffix Search

The paper introduces BOSS, a plug‑and‑play framework that enhances Greedy Coordinate Gradient (GCG) jailbreak attacks by employing a breadth‑oriented suffix search. BOSS incorporates Tail‑Focused Adversarial Loss, standard source loss, and behavior coverage to select terminal suffixes, then explores multiple short trajectories and selectively continues promising suffixes. Experiments on public benchmarks demonstrate that BOSS increases attack success rates across various GCG‑based methods while reducing optimization time.

By Shiliang Xiao, Jingsong Wei, Yuzhi Liang, Yufan Zheng, Xia Li, Qiliang Lin
arXiv Machine Learning
Sep 3

Explainable Information Processing in Particle Swarm Optimization through Landscape and Search Behavior Analysis

The paper introduces a dual‑perspective explainability framework for Particle Swarm Optimization (PSO). From a landscape viewpoint, it uses Exploratory Landscape Analysis (ELA) and machine‑learning classifiers to predict topology‑specific hyperparameters for unseen problems. From an algorithmic viewpoint, it incorporates IOHxplainer and Search Trajectory Networks (STN) with new metrics—Connectivity Density, Fragmentation Score, and Search Efficiency—to visualize and quantify PSO’s search organization and transition effectiveness across 24 benchmark functions and multiple topologies.

By Nitin Gupta, Bapi Dutta, Anupam Yadav
arXiv Machine Learning
Sep 3

Enhancing Road Safety Through Multi-Camera Image Segmentation with Post-Encroachment Time Analysis

The paper introduces a multi‑camera computer vision system that computes Post‑Encroachment Time (PET) in real time to assess intersection safety. Using four synchronized cameras and YOLOv11 segmentation on NVIDIA Jetson AGX Xavier devices, vehicle detections are mapped to a unified bird’s‑eye view and processed with a pixel‑level PET algorithm to generate high‑resolution heatmaps. The system records PET data in an SQL database and demonstrates real‑time throughput (2.68 FPS) with 800 × 800 heatmaps, validating a scalable, decentralized approach for high‑resolution intersection safety evaluation.

By Shounak Ray Chaudhuri, Arash Jahangiri, Christopher Paolini
arXiv Machine Learning
Sep 3

PRISM: An Agentic Multi-Model Architecture for Proactive Safety in Autonomous Transportation Systems

PRISM (Proactive Risk Intelligence and Safety Management) is an agentic multi-model architecture designed to shift autonomous transportation safety from reactive crash avoidance to proactive, continuous risk management. It uses inverse crash‑probability modeling to transform binary crash classifiers into dynamic safety scores, and runs three specialized models—trajectory kinematics, environmental risk, and VRU interaction—coordinated by a reinforcement‑learning reasoning layer. Across 1,296 naturalistic driving scenarios, PRISM achieved a mean safety score of 68/100, classified 77.6% of situations as advisory, and flagged 3.8% as near‑misses, with 11% requiring intervention or emergency response, highlighting trajectory risk and VRU proximity as key safety factors.

By Joyjit Roy, Samaresh Kumar Singh, Sushanta Das
arXiv Machine Learning
Sep 3

SPADE: SPaT Attack Detection from the Connected Vehicle's Perspective

The paper introduces SPADE, a labelled, multi‑modal dataset for detecting attacks on Signal Phase and Timing (SPaT) messages from the perspective of connected vehicles. Generated via Eclipse MOSAIC, SPADE includes 1.89 million timestep records across six attack classes and one benign class, combining SPaT fields, camera confidence scores, and V2V peer data over 40 features. The dataset, along with generation code and scenario configurations, is publicly released on GitHub to enable reproducible deep‑learning intrusion detection research in C‑V2X security.

By James Di Novo, Hany Ragab, Sylvain P. Leblanc
arXiv Computation and Language
Sep 3

CLASE: A Hybrid Method for Chinese Legalese Stylistic Evaluation

CLASE is a hybrid evaluation method for Chinese legal text that combines linguistic feature-based scores with experience-guided LLM-as-a-judge scores. It learns from contrastive pairs of authentic legal documents and their LLM-generated counterparts, enabling transparent, reference-free assessment of stylistic quality. Experiments on 200 Chinese legal documents show that CLASE aligns better with human judgments than traditional metrics and offers interpretable score breakdowns and improvement suggestions.

By Yiran Rex Ma, Yuxiao Ye, Huiyuan Xie
arXiv Computer Vision
Sep 3

Geometry-Guided Modeling of Foundation Features Enables Generalizable Object Shape Deformation Learning

The paper introduces a generalizable deformation learning framework that reconstructs 3D objects by deforming a category-level shape template to match a monocular observation. It employs a geometry-guided feature modeling mechanism to enrich foundation features with template topology, creating a geometry-aware representation that is explicitly correlated with the target observation for precise deformation. A view-adaptive feature aggregation module further bridges the gap between the fixed template and arbitrary target views by leveraging multi-view template features and camera poses, ensuring robust feature alignment across diverse viewpoints.

By Yiyao Ma, Kai Chen, Zhongxiang Zhou, Zhuheng Song, Dongsheng Xie, Zelong Tan, Rong Xiong, Qi Dou
arXiv Computer Vision
Sep 3

Geometric Distillation from Rectified Stereo: Leveraging Epipolar Cues for Monocular Depth

The paper introduces Epipolar Distillation (EpiDistill), a method that transfers scale‑aware geometric priors from multi‑view models to monocular depth foundation models using Rectified Stereo Tokens. By preserving epipolar attention patterns, the single‑view model maintains geometric consistency without needing multi‑view inputs during inference. Experiments show significant improvements in zero‑shot metric depth estimation on challenging datasets such as ETH3D and DIODE, and the approach consistently boosts performance of state‑of‑the‑art ViT‑based models like UniDepthV2 and DepthPro.

By Jung-Hee Kim, Xiaoming Liu
arXiv AI
Sep 3

The Memory Trust Gap: Capability-Dependent Failures in Persistent-Memory Agents

The paper investigates how persistent memory in AI agents can lead to over‑trust in stale facts, creating a "Memory Trust Gap" that worsens as model capability increases. Using a benchmark with Benefit and Safety suites across Qwen3 models of varying sizes, the authors show that larger models are more prone to harmful over‑trust, especially when metadata is absent or misleading. They also demonstrate that mitigation strategies such as exposing metadata or pre‑resolving conflicts improve accuracy, but the effectiveness depends on model size and dataset.

By Jundong Hu, Shekar Ramachandran
arXiv AI
Sep 3

FUSE: An Evaluating Framework for Dangerous Capabilities of LLMs

The paper introduces FUSE, a modular framework that evaluates large language models (LLMs) for dangerous capabilities across three orthogonal pipelines: Knowledge (K), Defense (D), and Harm (H). Using a chemical‑biological module, the authors assess 12 commercial LLMs, revealing divergent profiles among models and families, and showing that newer models increase knowledge while only partially improving defense. The framework’s reliability is supported by high cross‑judge consistency and low inter‑pipeline correlations.

By Zhengyi Jin, Ru Zhang, Xiao Chen, Xinbo Liu, Jiaxuan Lin, Jia Huang, Jianyi Liu, Zhen Yang