AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,884 stories · RSS feed

arXiv AI
Aug 25

Balancing Safety and Optimality in Robot Path Planning: Algorithm and Metric

The paper introduces the Unified Path Planner (UPP), a graph‑search algorithm that balances safety and optimality by adaptively weighting heuristics and using a local inverse‑distance safety field. UPP auto‑tunes its parameters during search, guaranteeing suboptimality bounds while improving obstacle clearance. Evaluation on ten simulated environments shows UPP achieving a 0.94 OptiSafe score—significantly higher than existing methods—while adding only 0.5–1% to path length and maintaining a 100% success rate, with hardware validation on a TurtleBot confirming practical benefits.

By Jatin Kumar Arora, Soutrik Bandyopadhyay, Sunil Sulania, Shubhendu Bhasin
arXiv Computation and Language
Aug 25

DIAG: Diagnostic Iterative Alignment and Generation for Data-Efficient Mathematical Preference Distillation

DIAG is a Diagnostic Iterative Alignment and Generation framework designed to improve data efficiency in aligning large language models for mathematical reasoning. It adaptively reshapes the practice distribution by first diagnosing valid preference-pair yield to calibrate exploration and exploitation, then generating targeted practice from the model’s failure traces. The approach is theoretically framed as a teacher‑mediated approximation to KL‑regularized reweighting, and experiments show that DIAG increases preference-pair yield and reasoning performance under the same training budget.

By Guhan Chen, Songtao Tian, Bohan Li, Hejin Wang, YeXin Xie, Zixiong Yu
Hugging Face Trending Papers
Aug 24

AI Finds A Way

Artificial Intelligence (AI) algorithms frequently learn creative and unexpected solutions, surprising even expert researchers who develop and study them. They often astonish practitioners by discover...