← Back to all news

AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,788 stories · RSS feed

arXiv Machine Learning
Oct 15

Discovering Undisclosed Concepts in LoRA Adapters for Diffusion-Based AIGC Models

The paper introduces a two‑stage method for uncovering hidden concepts encoded in Low‑Rank Adaptation (LoRA) adapters used to personalize diffusion models. First, Subspace‑Guided Evolutionary Search (SGES) identifies promising prompts by evaluating their alignment with the LoRA subspace without rendering images. Second, Paired Visual Identification (PVI) renders only the shortlisted prompts and compares LoRA outputs to the base model to confirm coherent, distinctive generation, thereby efficiently revealing undisclosed concepts.

By Yupeng Qiu, Han Fang, Dhananjai Sharma, Ee-Chien Chang
diffusionfine-tuningsafety
More like this →
arXiv Computation and Language
Oct 15

AdaptFuse: Training-Free Sequential Preference Inference via Externalized Bayesian Computation

arXiv:2604.03925v2 Announce Type: replace Abstract: Large language models (LLMs) struggle to accumulate evidence across multiple rounds of user interactions, failing to update their beliefs in a mann...

By Fangzhou Lin, Zilin Dai, Shuo Xing, Peiran Li, Siyuan Yang, Qianwen Ge, Kazunori Yamada, Yang Zhou, Ziming Zhang, Haichong Zhang, Zhengzhong Tu
llmsfine-tuningsafety
More like this →
arXiv Machine Learning
Oct 15

Who's Behind the Text? Robust Model Authorship Attribution

The paper introduces ATLAS, a likelihood‑based method for model authorship attribution (MAA) that learns how each source family generates text and attributes new text by comparing token‑level likelihoods. It also presents MAABench, a benchmark for evaluating MAA under out‑of‑distribution scenarios such as unseen domains and future model versions. Experiments show ATLAS outperforms existing baselines, achieving 74.27% accuracy on the hardest MAABench setting and correctly identifying teacher models in distillation experiments.

By Minseok Ko, Hyunseok Lee, Woomin Song, Beomjun Kim, Jinwoo Shin, Kunwoong Kim
llmsefficiencybenchmarkssafety
More like this →
arXiv Machine Learning
Oct 15

Calibration and Risk-Budgeted Decisions with System One Models

The paper investigates how large language model (LLM) workflows and agents make binary decisions—such as allowing or blocking an input—when human deferral is not possible. It proposes using System One decision scores that default to a conservative fallback in uncertain cases and control false omission risk within the subset of inputs that receive the utility‑maximizing action. Experiments on six datasets spanning moderation, safety, and reranking show that naive threshold baselines often exceed target risk, whereas a Learn‑Then‑Test approach achieves only 7.8% risk, demonstrating the limitations of probability‑like interfaces and the benefits of risk‑budgeted selection.

By Soundouss Messoudi, Youness Mansar
llmsragagentsfine-tuningsafety
More like this →
arXiv Machine Learning
Oct 15

Contrastive uncertainty learning for robust multi-structure segmentation in lower-pelvic MR

The paper introduces Contrastive Uncertainty Learning (CUL), a supervised framework that leverages voxel‑wise predictive uncertainty to guide representation learning for multi‑structure segmentation in lower‑pelvic MRI. Using Monte Carlo Dropout, CUL identifies uncertain voxels and shapes the embedding space so that low‑uncertainty, correctly predicted features become stable while high‑uncertainty or misclassified features are repelled, preventing feature collapse at challenging boundaries. Evaluated on 589 prostate MRI scans with a ViT‑based UperNet decoder, CUL achieves comparable Dice scores to a baseline while significantly reducing surface distance for most structures, and ablation studies confirm the necessity of both uncertainty‑guided sampling and the contrastive objective.

By Bilal Sidiqi, Yipei Wang, Wen Yan, Shaheer U. Saeed, Daniel C. Alexander, Yipeng Hu
ragcomputer-visionbenchmarkssafety
More like this →