AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,798 stories · RSS feed

arXiv AI
Oct 7

Lineage-Aware Memory Governance: A Derivation-Gated Framework for Privacy-Preserving Column-Level Access Control in Enterprise AI Agents

The paper proposes the Analytical Memory Unit (AMU), a memory schema that attaches a full derivation (lineage) graph to every cached result in enterprise AI agents. By gating retrieval with a policy that requires authorization for every column touched, the authors prove that sensitive columns cannot be leaked through derived results, achieving up to 90% lineage completeness to eliminate leakage. Experiments show lineage‑gated retrieval removes 18.8‑25.5% of cross‑department leakage while maintaining 81.5‑82.6% memory reuse with minimal overhead, and a real‑agent proof‑of‑concept demonstrates zero leaks over multiple interactions.

By Venkata M Sangaraju, Sudhir Vissa
arXiv AI
Oct 7

Seeing the Invisible: Physics-Guided Visual Prompting for Temperature- and Radiation-Aware VLA Navigation

The paper introduces Physics‑Guided Visual Prompting (PG‑VP), a plug‑and‑play module that overlays a virtual obstacle onto the input of a frozen Vision‑Language‑Action model to guide navigation around invisible hazards such as radiation or temperature spikes. PG‑VP performs a physics‑based risk assessment to determine the avoidance direction and dynamically renders the same virtual obstacle across frames, allowing the existing navigation policy to detour without retraining. Experiments on OmniNav with R2R‑CE and RxR‑CE datasets show that PG‑VP steers the policy toward low‑risk actions in 84.9% and 83.2% of cases, while real‑world tests on a robot demonstrate significant safety improvements against thermal and radiation sources.

By Hojoon Son, Fan Zhang
arXiv AI
Oct 7

Exploiting Acoustic and Content-Oriented Speaker Verification Attacks Against Multilingual Voice Anonymization

The paper investigates how acoustic and content-oriented speaker verification attacks perform against multilingual voice anonymization. It evaluates both attacker types on anonymized speech, builds a multilingual voice-converted dataset to enhance cross‑lingual generalization, and finds that attack success depends on how much linguistic utility is preserved. Acoustic‑oriented attacks generally outperform content‑oriented ones, but the performance gap narrows when linguistic information is well maintained, and the new dataset improves results while reducing cross‑lingual disparities.

By Ridwan Arefeen, Ze Li, Rong Tong, Ming Li, Xiaoxiao Miao
arXiv AI
Oct 7

How High Is 0.6? Floors, Ceilings, and Headroom in Interpretability Probing

The paper argues that probe scores lack intrinsic meaning and should be interpreted relative to two reference points: a floor (what simple inputs predict) and a ceiling (what the full input predicts). The difference, called headroom, indicates the range where a probe can reveal that a model computes beyond what the input already provides. Experiments on transformers and real models show that headroom can vanish when the target no longer depends on hidden variables or when the input no longer reveals them, and that some previously claimed representations are largely explained by the input text alone.

By Pranjal Garg
arXiv AI
Oct 7

Systemization of Knowledge (SoK): Human-Centered AI Safety for Youth

The paper systematically reviews 100 empirical HCI studies on children and youth interacting with AI in various settings. Using the YAIR risk taxonomy and MIT Mitigation Taxonomy, it maps identified risks to countermeasures, finding that most risks are only matched with proposed countermeasures, few are implemented, and even fewer are evaluated. The authors highlight gaps in coverage and testing, and suggest concrete research directions to improve youth AI safety.

By Pratyasha Saha, Yaman Yu, Yang Wang
arXiv AI
Oct 7

FedDermaSeg: Federated Learning for Dermatological Image Segmentation

FedDermaSeg explores federated learning for skin lesion segmentation, aiming to preserve privacy by avoiding centralized data aggregation. Using the ISIC 2018 dataset in a simulated distributed setting, the federated model matches centralized training performance and outperforms locally trained models. The study demonstrates that collaborative, privacy‑preserving segmentation is feasible without central image storage.

By Anabik Pal, Ganesh Patidar, Bikash Santra
arXiv AI
Oct 7

WorldSonus: Bringing Sound to Worlds

arXiv:2610.08760v1 Announce Type: cross Abstract: Recent advances in world models have enabled increasingly realistic visual synthesis. However, these generated environments remain largely silent. Br...

By Pengjun Fang, Jingyi Fa, Kam Man Wu, Jiaming Wang, Haoyuan Huang, Yaguang Wu, Xiangjun Huang, Ziyang Ma, Weijia Chen, Hongyu Liu, Zeyue Tian, Qifeng Chen