AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

11,407 stories · RSS feed

arXiv AI
Jun 2

Prototype Transformer: Towards Language Model Architectures Interpretable by Design

arXiv:2602. 11852v2 Announce Type: replace Abstract: While state-of-the-art language models (LMs) surpass most humans in certain domains, their reasoning remains largely opaque, reducing trust and increasing the risk of deception and hallucination.

By Yordan Yordanov, Matteo Forasassi, Bayar Menzat, Ruizhi Wang, Chang Qi, Markus Kaltenberger, Amine M'Charrak, Tommaso Salvatori, Thomas Lukasiewicz
arXiv AI
Jun 2

How Hard Can It Be? Hardness-Aware Multi-Objective Unlearning

arXiv:2606. 02119v1 Announce Type: cross Abstract: Machine unlearning aims to remove the influence of specific forget training data due to privacy, copyright or bias concerns while maintaining the model performance on the remaining retain data.

By Jiangwei Chen, Xinyuan Niu, Rachael Hwee Ling Sim, Zhengyuan Liu, Nancy F. Chen, Bryan Kian Hsiang Low
arXiv Machine Learning
Jun 2

Measurement Geometry and Design for Trustworthy Generative Inverse Problems

arXiv:2606. 02309v1 Announce Type: new Abstract: Generative models are increasingly used as priors for inverse problems, but their ability to produce realistic images creates a basic trust problem: a plausible reconstruction may be supported by the measurements, or it may be filled in by the prior along unobserved directions.

By Pengfei Jin, Na Li, Quanzheng Li
arXiv AI
Jun 2

Paradoxical noise preference in RNNs

arXiv:2601. 04539v2 Announce Type: replace-cross Abstract: In recurrent neural networks (RNNs) used to model biological neural networks, noise is typically introduced during training to emulate biological variability and regularize learning.

By Noah Eckstein, Manoj Srinivasan