AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,638 stories · RSS feed

arXiv AI
1d ago

TherapyGym: Evaluating and Aligning Clinical Fidelity and Safety in Therapy Chatbots

arXiv:2603.18008v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are increasingly used for mental-health support; yet prevailing evaluation methods--fluency metrics, preference...

By Fangrui Huang, Souhad Chbeir, Arpandeep Khatua, Sheng Wang, Sijun Tan, Kenan Ye, Lily Bailey, Merryn Daniel, Ryan Louie, Sanmi Koyejo, Ehsan Adeli
arXiv AI
1d ago

Agent Security is a Systems Problem

arXiv:2605.18991v3 Announce Type: replace-cross Abstract: We take the position that agent security must be approached as a systems problem: the AI model powering the agent must be treated as an untru...

By Mihai Christodorescu, Earlence Fernandes, Ashish Hooda, Somesh Jha, Johann Rehberger, Kamalika Chaudhuri, Xiaohan Fu, Khawaja Shams, Guy Amir, Jihye Choi, Sarthak Choudhary, Nils Palumbo, Andrey Labunets, Nishit V. Pandya
arXiv Machine Learning
1d ago

The Confidence Score: a New Measure on Kernels

arXiv:2611.05528v1 Announce Type: new Abstract: Across a wide range of applications, kernel measures are used to tune hyperparameters, select among candidate kernels, and assess the suitability of ke...

By Elizabeth Hofer, Martin v. Mohrenschildt
arXiv Machine Learning
1d ago

Alignment Myopia in Generative AI

arXiv:2611.06830v1 Announce Type: cross Abstract: Current generative AI alignment relies on a myopic paradigm that prioritizes immediate conversational satisfaction. By relying on short-horizon feedb...

By Remi Denton, Aida Davani, Cynthia L Bennett, Michael Madaio, Roma Patel, Himabindu Lakkaraju