AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

9,539 stories · RSS feed

arXiv AI
Jul 10

MAVEN: A Multi-stage Agentic Annotation Pipeline for Video Reasoning Tasks

arXiv:2605. 21917v2 Announce Type: replace-cross Abstract: Training Vision Language Models (VLMs) for video event reasoning requires high-quality structured annotations capturing not only what happened, but when, where, why, and with what consequence, at a scale manual labelling cannot support.

By Han Zhang, Wanting Jiang, Tomasz Kornuta, Tian Zheng, Vidya Murali
arXiv Machine Learning
Jul 10

EdgeRefine: Privacy-Utility Balance for Graphs via Jaccard Sampling under Edge Differential Privacy

arXiv:2607. 08659v1 Announce Type: new Abstract: Graph Neural Networks (GNNs) have shown considerable success in learning from graph-structured data, but their use in privacy-sensitive areas remains difficult because graph structure can leak sensitive link information.

By Wenxiu Ding, Muzhi Liu, Zheng Yan, Mingjun Wang, Yifan Zhao, Qiao Liu
arXiv AI
Jul 10

When Synthetic Speech Is All You Have: Better Call GRPO

arXiv:2607. 08409v1 Announce Type: cross Abstract: LLM-based ASR adapted to regulated domains such as banking is bottlenecked by privacy: real speech is costly and legally constrained to collect, making synthetic text-to-speech (TTS) an attractive substitute.

By Shashi Kumar, Yanis Labrak, Hasindri Watawana, Sergio Burdisso, Esa\'u Villatoro-Tello, Kadri Hacio\u{g}lu, Petr Motlicek, Andreas Stolcke
arXiv Machine Learning
Jul 10

Communication-Efficient Byzantine-Robust Federated Conformal Prediction via Partial Model Sharing

arXiv:2602. 18396v2 Announce Type: replace Abstract: We propose PRISM-FCP (Partial shaRing and robust calIbration with Statistical Margins for Federated Conformal Prediction), a communication-efficient Byzantine-robust federated conformal prediction framework that uses partial model sharing to mitigate stochastic model-poisoning attacks during training and histogram-based filtering to mitigate adversarial calibration submissions.

By Ehsan Lari, Reza Arablouei, Stefan Werner
arXiv AI
Jul 10

Persona Cartography: Charting Language Model Personality Traits in Weight Space

arXiv:2607. 07916v1 Announce Type: new Abstract: Large language models exhibit recurring behavioural patterns -- personas -- that shape generalisation and safety, but we lack reliable tools for decomposing, measuring, and controlling them.

By Luke Baines, Anton Gonzalvez Hawthorne, Mariia Koroliuk, Irakli Shalibashvili, Cl\'ement Dumas, Konstantinos Voudouris, David Demitri Africa