AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,638 stories · RSS feed

arXiv Machine Learning
1d ago

Self-Directed Apple Tasting

arXiv:2611.06677v1 Announce Type: new Abstract: We study the classical apple-tasting setting, a partial feedback model in which the learner observes the true label only when it predicts $1$. A combin...

By Tanmay Devale, Reed Phillips, Arvind Ramaswami
arXiv Machine Learning
1d ago

Regret Bounds for Market Making with Persistent Orders

arXiv:2611.06159v1 Announce Type: cross Abstract: We study adversarial market making with orders that remain active until execution. The maker operates on two markets: a primary market where orders a...

By Nicol\`o Cesa-Bianchi, Tommaso Cesari, Roberto Colomboni, Luigi Foscari, Vinayak Pathak
arXiv Machine Learning
1d ago

Estimating Implicit Regularization in Deep Learning

arXiv:2605.05436v2 Announce Type: replace-cross Abstract: Deep learning systems are known to exhibit implicit regularization (alt. implicit bias), favoring simple solutions instead of merely minimizi...

By Joseph H. Rudoler, Kevin Tan, Giles Hooker, Konrad P. Kording
arXiv Computer Vision
1d ago

ImagineAgent: What if Agents Could Imagine? Reinforcing Open-Vocabulary HOI Comprehension through Generation

arXiv:2602.11499v3 Announce Type: replace Abstract: Multimodal Large Language Models have shown promising capabilities in bridging visual and textual reasoning, yet their reasoning capabilities in Op...

By Zhenlong Yuan, Yue Wang, Dapeng Zhang, Kejin Cui, Rui Chen, Jing Tang, Lei Sun, Hongwei Yu, Chengxuan Qian, Xiangxiang Chu, Shuo Li, Yuyin Zhou
arXiv Machine Learning
1d ago

Role-based Effectiveness of an Epistemic Uncertainty Estimate in Training Models for Semantic Segmentation of 3D LiDAR Point Clouds

The paper studies how epistemic uncertainty estimates can be used in different ways—either as an additional input feature or as a per‑sample confidence weight in the loss function—to improve 3D LiDAR point cloud segmentation. Experiments on four large‑scale LiDAR benchmarks (ISPRS Vaihingen, AHN3, Paris‑Lille‑3D, and Toronto‑3D) show that uncertainty‑aware models outperform a baseline ANN in mean Intersection over Union, recover minority classes, and better align prediction error with estimated uncertainty. The results suggest that uncertainty can serve multiple roles in the segmentation pipeline beyond merely boosting accuracy.

By Krishnakumar N, Jaya Sreevalsan-Nair