AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

9,648 stories · RSS feed

arXiv Machine Learning
Jul 3

Rethinking Post-Hoc Calibration in Semantic Segmentation

arXiv:2607. 01902v1 Announce Type: cross Abstract: Reliable confidence estimates are essential in semantic segmentation, especially in safety-critical settings where overconfident errors can mislead downstream decisions.

By Tristan Kirscher (ICube), Kim-Celine Kahl (DKFZ), Balint Kovacs (DKFZ), Maximilian R. Rokuss (DKFZ), Klaus Maier-Hein (DKFZ), Xavier Coubez (ICube), Philippe Meyer (ICube), Sylvain Faisan (ICube)
arXiv AI
Jul 3

Conformal Policy Control

arXiv:2603. 02196v3 Announce Type: replace Abstract: An agent must try new behaviors to explore and improve.

By Drew Prinster, Clara Fannjiang, Ji Won Park, Kyunghyun Cho, Anqi Liu, Suchi Saria, Samuel Stanton
arXiv Machine Learning
Jul 3

WBMM: Windowed Batch Matrix Multiplication for Efficient Large Receptive Field Convolution

arXiv:2607. 02097v1 Announce Type: cross Abstract: Large kernel depthwise convolutions achieve strong performance but suffer from significant degradation as kernel size grows due to irregular memory access from gather-based computation; while Large Kernel Acceleration (LKA) helps on small feature maps, it becomes counterproductive on large feature maps, even slower than non-accelerated implementations.

By Wan Song, Wei Zhou, Rui Wang, Jun Yu, Toru Kurihara, Jiajia Xu, Shu Zhan
arXiv Machine Learning
Jul 3

Multilayer Q-Matrix-Embedded Neural Network for Cognitive Diagnosis (M-QCDNet): Structure-Aware Deep Learning Architecture for Psychometric Interpretability

arXiv:2607. 01278v1 Announce Type: new Abstract: The research proposes a multilayer Q-matrix-embedded neural network for cognitive diagnosis (M-QCDNet), which integrates the structural interpretability of cognitive diagnostic models (CDMs) with the deep learning neural network (NN).

By Yiyao Yang
arXiv Machine Learning
Jul 3

Quantifying the Uncertainty of Blindly Estimated Room Embeddings Using a Dispersion-Calibrated Score

arXiv:2607. 01527v1 Announce Type: cross Abstract: Room embeddings derived from reverberant speech are often unreliable: speech content and recording degradation can alter the representation even when speaker, room, and source-receiver geometry remain unchanged, degrading downstream task performance.

By Yang Xiang, Philipp G\"otz, Emanu\"el A. P. Habets, Andreas Walther, Wenwu Wang, Philip J. B. Jackson
arXiv AI
Jul 3

DriveVLM-RL: Neuroscience-Inspired Reinforcement Learning with Vision-Language Models for Safe and Deployable Autonomous Driving

arXiv:2603. 18315v2 Announce Type: replace-cross Abstract: Traditional reinforcement learning (RL) methods rely on manually engineered rewards or sparse collision signals, which fail to capture the rich contextual understanding required for safe driving and make unsafe exploration unavoidable in real-world settings.

By Zilin Huang, Zihao Sheng, Zhengyang Wan, Yansong Qu, Junwei You, Sicong Jiang, Sikai Chen
arXiv AI
Jul 3

Robust for the Wrong Reasons: The Representational Geometry of LLM Robustness to Science Skepticism

arXiv:2607. 01951v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly consulted on contested scientific questions, raising the concern that they will sycophantically retreat from established consensus when a user signals doubt -- drifting toward a false balance that treats settled science as one view among several.

By Minjong Cheon