AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

10,300 stories · RSS feed

arXiv Machine Learning
Jul 2

ZO-Act: Efficient Zeroth-Order Fine-Tuning via One-Shot Activation-Informed Low-Rank Subspaces

arXiv:2607. 01125v1 Announce Type: new Abstract: Zeroth-order (ZO) optimization enables fine-tuning large language models when backpropagation is unavailable or memory-prohibitive, but existing methods often perturb full model weights or randomly constructed low-dimensional subspaces, yielding high-variance estimates and limited performance.

By Xun Dong, Yibo Xu, Naigang Wang, Xin Li, Penghang Yin, Zi Yang
arXiv Machine Learning
Jul 2

A Filtered Mixture-of-Generators for Fully Synthetic Survival Training

arXiv:2607. 00127v1 Announce Type: new Abstract: Survival analysis models time-to-event data, but in clinical settings training data are costly and scarce: events accrue over years of follow-up, cohorts are small, and privacy regulations restrict sharing across institutions.

By Niccol\`o Maria Rizzi, Eugenio Lomurno, Alberto Archetti, Matteo Matteucci
arXiv AI
Jul 2

LLM-Guided ODE Discovery and Parameter Inference from Small-Cohort Aggregate Data

arXiv:2607. 00733v1 Announce Type: cross Abstract: Mechanistic modeling via ordinary differential equations (ODEs) provides interpretable descriptions of complex dynamics and enables inference of underlying mechanisms, which is particularly valuable in clinical settings.

By Hanning Yang, Meropi Karakioulaki, Lennart Purucker, Tim Litwin, Cristina Has, Moritz Hess
arXiv Machine Learning
Jul 2

A Mechanistic View of Authority Hierarchy in LLM Sycophancy

arXiv:2607. 00415v1 Announce Type: cross Abstract: Authority bias poses a critical safety concern in language models: models systematically prioritize social cues from authority figures over factual consistency, swaying their answers based on source credibility rather than evidence.

By Emil Joswin, Srujananjali Medicherla, Priyanka Mary Mammen
arXiv Machine Learning
Jul 2

Explainability in mulimodal deep transformation models for stroke outcome prediction

arXiv:2504. 06299v2 Announce Type: replace-cross Abstract: Multimodal prediction models based on imaging and clinical data are increasingly used for clinical decision support, yet their interpretability remains limited.

By Lisa Herzog, Jonas Br\"andli, Maurice Schneeberger, Loran Avci, Nordin Dari, Martin H\"ansel, Hakim Baazaoui, Pascal B\"uhler, Susanne Wegener, Beate Sick
arXiv AI
Jul 2

Managed Autonomy at Runtime: Gear-Based Safety and Governance for Single- and Multi-Agent Cyber-Physical Systems

arXiv:2607. 00334v1 Announce Type: new Abstract: Autonomous agents, whether LLM-driven software agents or robotic physical agents, face a common class of failure modes when operating without continuous human oversight: safety violations from unverified actions, behavioral instability from unconstrained loops, and continuity loss from unhandled error states.

By Srini Ramaswamy, Wang Miaosheng
arXiv AI
Jul 2

Mapping the Evaluation Frontier: An Empirical Survey of the Bias-Reliability Tradeoff Across Eleven Evaluator-Agent Conditions

arXiv:2607. 00304v1 Announce Type: cross Abstract: The bias-reliability tradeoff conjectures that LLM evaluation systems are constrained in (gamma, H, CV) space, where evaluator coupling (gamma), strategy diversity (H), and small-sample measurement reliability (CV(N)) cannot be simultaneously optimized at fixed sample size N.

By Zewen Liu