AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,788 stories · RSS feed

arXiv Machine Learning
Oct 15

When Non-Adversarial Noise Hurts Learning: Benchmarking Skill Self-Evolution in Agents

arXiv:2610.19910v1 Announce Type: new Abstract: Large language model agents increasingly improve by distilling experience into reusable skills, but errors in that experience can persist in learned ru...

By Tao Yu, Zhongtian Luo, Yuchen Long, Yiheng Lu, Yuxuan Zhou, Yanpei Gong, Siyi Cao, Yuqing Wu, Haopeng Jin, Zhenghao Zhang, Yujia Yang, Jingmin Zhu, Yunyun Hou, Xinming Wang, Hongzhu Yi, Jiachen Liu, Sirui Wang, Nan Duan, Yan Huang, Liang Wang, Tianjin Huang, Yanyan Zou
arXiv AI
Oct 15

GeoFix: Geometry Fixing for 4D Consistent Generation

arXiv:2610.14293v1 Announce Type: cross Abstract: Video diffusion models increasingly serve as priors for 4D generation, but the generated videos often lack geometric consistency. Recent approaches a...

By Yeobin Hong, Hyojun Go, Dohun Lee, Jeongjae Lee, Sunghwan Hong, Federico Tombari, Zhaochong An, Jong Chul Ye, Konrad Schindler, Dominik Narnhofer
arXiv AI
Oct 15

Revealing the Gap in Human and VLM Scene Description through Counterfactual Semantic Saliency

The paper introduces Counterfactual Semantic Saliency (CSS), a black-box, model-agnostic method that measures how the removal of objects from a scene shifts the semantic description produced by vision‑language models (VLMs). Using CSS, the authors compare prominent VLMs to a large human psychophysics baseline (16,603 responses across 307 scenes) and find that VLMs over‑rely on large, centrally located, and highly salient objects, while under‑rely on people compared to humans. The study highlights a significant scene‑comprehension gap between VLMs and human perception, with size bias emerging as a key factor driving model‑human divergence.

By Ziqi Wen, Parsa Madinei, Miguel P. Eckstein