AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,896 stories · RSS feed

arXiv AI
Aug 21

SoftVTBench: A Safety-Aware Visuo-Tactile Benchmark for Physically Constrained Robotic Manipulation of Deformable Objects (Early Version)

arXiv:2607. 04234v2 Announce Type: replace-cross Abstract: Deformable object manipulation poses challenges beyond task completion: successful execution must also maintain safe physical interaction, holding the object stably without slip or drop while avoiding excessive deformation.

By Bowen Jing, Mingxin Wang, Ruiyang Hao, Chenchen Ge, Hanwen Shen, Junjie He, Yang Cui, Yiming Hou, Weitao Zhou, Jiawei Wang, Minglei Li, Dandan Zhang, Ding Zhao, Houde Liu, Xiaofan Li, Si Liu, Ping Luo, Haibao Yu
arXiv AI
Aug 21

CVSD-Reg: Cross-Modal Visual Semantic Prior Distillation for Robust LiDAR Registration

arXiv:2608. 19536v1 Announce Type: cross Abstract: Learning-based global point cloud registration has achieved remarkable progress, yet its reliance on geometric representations makes existing methods sensitive to variations in point density, scan pattern, viewpoint, and sensor characteristics.

By Eunsoo Im, Junghun Suh, Gyeonggwan Lee, Seunghwan Hong
Hugging Face Trending Papers
Aug 20

DICS: Data-Informed Centroid Splitting for Decision Tree Classifiers

The paper introduces DICS, a clustering-based framework that uses data-informed priors to construct a compact set of candidate splits for decision tree classifiers. By incorporating class-aware structure, DICS reduces the split search space, preserving predictive performance while cutting training time. The authors provide theoretical analysis and experimental results showing comparable accuracy to exhaustive search across synthetic and benchmark datasets.

Hugging Face Trending Papers
Aug 20

DecoVAE: a Lightweight Interpretable Trend-Seasonal VAE Framework for Efficient Probabilistic Time Series Forecasting

Probabilistic time series forecasting remains challenging, largely because modeling distinct trend and seasonal dynamics requires specialized approaches. Existing methods often fail to capture the unique inner properties of these components, lack interpretability, or suffer from heavy memory and runtime overhead.

Hugging Face Trending Papers
Aug 20

Auditing Cross-Lingual Fairness in Language Model Watermarking

Watermarking schemes for large language model output are evaluated almost exclusively on English text using each scheme's detection threshold and a narrow set of quality measurements. Multilingual deployment exposes evaluation-design choices that are inconsequential on English but determine conclusions cross-lingually.

Hugging Face Trending Papers
Aug 20

DIFFCZSL: Compositional Zero-Shot Learning Regularized by Diffusion Representations

Compositional Zero-Shot Learning (CZSL) aims to recognize unseen attribute-object compositions by leveraging knowledge of primitive concepts learned from seen compositions. Although recent works achieve impressive performance in CZSL by leveraging large vision-language models, they primarily rely on discriminative representations that may not explicitly preserve the structured relationships between primitive concepts and their compositions.

Hugging Face Trending Papers
Aug 20

Inadvertent Context Leakage in Language Models

For AI agents to be useful beyond simple chat, they must hold sensitive user context such as calendars, credentials, health records, and financial data. We study whether the mere presence of such secrets in a model's context window introduces hidden correlations into the model's benign outputs, allowing reconstruction even when the model correctly refuses direct extraction.

Hugging Face Trending Papers
Aug 20

TT-net: Quantum Inspired Tensor Network Denoising in Conditional GANs

TT‑Net introduces a two‑cut tensor‑train decomposition to replace the per‑channel SVD denoising block in conditional GANs, enabling direct cross‑channel information access. In controlled experiments, TT‑Net outperforms SVD‑Net on PSNR and SSIM for Gaussian, motion blur, and salt‑and‑pepper noise, and surpasses EigenGAN and Pix2pix for Gaussian noise. Training dynamics reveal that TT‑Net’s adversarial loss saturates while reconstruction quality continues to improve, raising questions about the role of the adversarial component.

Hugging Face Trending Papers
Aug 20

Distilling Aggregate Mobility Statistics into a Language Model Policy for Post-Event Crowd Simulation

The paper presents a method to align a language‑model‑based crowd agent with aggregate mobility data by fine‑tuning it to match observed destination compositions derived from origin‑to‑destination flows. The approach uses iterative proportional fitting to reweight the model’s destination distribution and corrects for dominant destination inflation by training a low‑rank adapter on resampled trajectories. Experiments on mobile network counts from two baseball games show a 25% reduction in destination‑share error while maintaining similar grid correlation across policies.