AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,788 stories · RSS feed

arXiv AI
Oct 26

Concept Acupuncturist: An Efficient and Precise Concept Erasure Method for Diffusion Models via Sparse Autoencoders

Concept Acupuncturist (ConceptACP) is a machine‑unlearning framework that removes targeted concepts from diffusion models efficiently and precisely. It identifies the most relevant cross‑attention block in the U‑Net denoiser, fine‑tunes only that block, and uses Sparse Autoencoders to generate semantically meaningful correction signals for concept erasure. Applied to SDXL‑Turbo, ConceptACP boosts forgetting success by over 36 percentage points while maintaining image quality and prompt fidelity, and works across other Stable Diffusion backbones.

By Yiran Xiahou, Siliang Qin, Kai Chen, Shengzhi Zhang, Renyang Liu
OpenAI Blog
Oct 23

Securing our frontier model training environment

OpenAI details the measures taken to enhance containment in its frontier model training environment. The company highlights the use of offline environments and high‑compute red‑teaming to strengthen security during training and evaluation. These steps aim to protect the integrity of the model development process.

arXiv Computer Vision
Oct 23

AgentDreamer: Agentic 3D Motion Reasoning for Text-Controllable Physics Simulation

AgentDreamer is an agent-guided optimization framework that aligns text instructions with simulated 3D motion by using an off-the-shelf agent to generate a structured semantic motion prior. The prior specifies what should move, in which direction, and in what temporal order, and is represented as parametric functions over temporal chunks to produce dense 3D trajectories. These trajectories are then used to optimize a differentiable MPM simulator so that its rollout matches the prior while respecting scene dynamics, and the framework introduces Motion-IG, an information‑theoretic metric for measuring text‑motion alignment.

By Mijeong Kim, Suho Park, Junha Park, Bohyung Han
arXiv Machine Learning
Oct 23

LLM2Cert: LLM Synthesis of Certificates with SMT/Lean Verification

The paper introduces DynaCertBench, a benchmark of 105 diverse systems for certifying stability, safety, and related properties, and presents LLM2Cert, a framework that uses large language models to generate and verify candidate certificates via SMT solvers or Lean. LLM2Cert achieves a 77.1% verification rate on the benchmark, improving by 17.1 percentage points over a single-shot approach, and successfully verifies 22 certificates that All-SMT could not resolve. The authors provide code and data on GitHub for reproducibility.

By Zhipeng Cao, Peiyang Song, Nikolaus Vertovec, Weiru Cui, Zhihao Yang, Mingkang Liu, Luxia Lin, Bai Xue, Alessandro Abate
arXiv AI
Oct 23

DeepLook: Uncertainty-Guided Branching with Lookahead Confidence for Efficient Reasoning

DeepLook is a training‑free decoding framework that uses segment‑level confidence to guide branching during inference. It only expands the search tree at points of high uncertainty and ranks candidate continuations by Average Lookahead Confidence (ALC), a mean confidence over fixed‑horizon rollouts, before finalizing and voting on a pruned subset. Experiments on four mathematics benchmarks show that DeepLook matches or surpasses DeepConf‑low in 12 of 16 settings while reducing token usage by 87.3%, though its advantage disappears on the knowledge‑intensive GPQA‑Diamond task due to hallucinations receiving high ALC.

By Tingxin Yang, Zefeng Wang, Mengyue Wang, Jinhe Bi, Xingcheng Zhou, Volker Tresp, Yunpu Ma