AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,884 stories · RSS feed

arXiv AI
Aug 25

Evaluating Inference-Time Defenses Against Package Hallucination in LLM-Generated Code

The paper investigates how large language models (LLMs) hallucinate nonexistent software packages during code generation and evaluates methods to mitigate this issue. It finds that current evaluation practices overestimate hallucination rates, especially for Python, and that Retrieval-Augmented Generation (RAG) and Self-Refine reduce hallucinations across multiple models and languages. The study also introduces Package Utility (PU) to measure whether defenses preserve useful recommendations and shows that Greedy decoding offers the best trade‑off between mitigation and utility, while adversarial prompts significantly increase hallucination rates, particularly in Ruby.

By Alberick Euraste Djire, Iyiola E. Olatunji, Melissa Tessa, Earl T. Barr, Jacques Klein, Tegawend\'e F. Bissyand\'e
arXiv AI
Aug 25

From Isolation to Alignment: Unified LoRA for Efficient Multi-Task Learning

The paper introduces Align‑LoRA, a unified LoRA framework for multi‑task learning that replaces complex, isolated adapter designs with a single‑adapter model enhanced by a higher rank and an explicit alignment loss. It demonstrates that a router‑free, multi‑head model with high inter‑head redundancy can outperform more elaborate baselines, and that a unified LoRA can achieve competitive performance while enabling weight merging and zero inference latency. Extensive experiments and theoretical analysis confirm that Align‑LoRA surpasses prevailing approaches, offering a simpler, production‑friendly paradigm for parameter‑efficient fine‑tuning of large language models.

By Jinda Liu, Yi Chang, Yuan Wu
arXiv Computation and Language
Aug 25

Can LLMs Truly Forget? Revealing Unlearning Gaps Through Adversarial Evaluation

arXiv:2608.21606v1 Announce Type: new Abstract: Machine unlearning aims to remove the influence of targeted training data from a model while preserving its remaining capabilities, but evaluating whet...

By Ayush Gupta, Hima Varshini Surisetty, Sreevidya Bollineni, Varad Ingale, Tuhina Tripathi, Abhishek Lalwani, Somya Chatterjee, Sadid Hasan
arXiv AI
Aug 25

Alignment midtraining for animals

The paper studies how midtraining with synthetic documents can align AI models to the value of animal compassion. It introduces ANIMA, a 26‑question benchmark covering 13 ethical dimensions, and shows that training on 3,000 documents yields 77% accuracy versus 40% for instruction‑tuning. The improvement disappears after 5,000 samples, indicating that document‑based interventions may need explicit preservation strategies.

By Jasmine Brazilek, Miles Tidmarsh
arXiv AI
Aug 25

Mitigating Sample-Level Imbalance via Probabilistic Separation for Adaptive Multimodal Fusion

The paper introduces a framework to tackle modality imbalance in multimodal learning by focusing on sample-level variations. It defines a Modality Gap metric to measure prediction discrepancies, models the resulting bimodal distribution with a Gaussian Mixture Model, and uses Bayesian probabilities for soft separation of balanced and imbalanced samples. A two‑stage training process—Warm‑up and Adaptive Training—reallocates loss weights based on the GMM, strengthening alignment for imbalanced samples while favoring fusion for balanced ones, and shows superior performance over existing baselines.

By Zhiwen Yu, Zhaocheng Liu, Xiaoqing Liu, Huanqiang Zeng, C. L. Philip Chen
arXiv Computation and Language
Aug 25

Checkup2Action: A Multimodal Clinical Check-up Report Dataset for Patient-Oriented Action Card Generation

arXiv:2605.11533v4 Announce Type: replace Abstract: Routine clinical check-up reports combine laboratory measurements, physiological assessments, imaging findings and visually structured information,...

By Sike Xiang, Shuang Chen, Kevin Qinghong Lin, Jialin Yu, Yijia Sun, Philip Torr, Amir Atapour-Abarghouei
arXiv Computation and Language
Aug 25

RASET: Router-Agnostic Safety-Critical Expert Tuning Exposes Localized Safety Enforcement Failures in Mixture-of-Experts LLMs

The paper introduces RASET, a router‑agnostic safety‑critical expert tuning framework for Mixture‑of‑Experts (MoE) large language models. RASET identifies a small subset of experts that are responsible for safety enforcement and applies parameter‑efficient tuning only to those experts, preserving the model’s intrinsic routing behavior. Experiments on five open‑weight MoE backbones show that RASET achieves a high safety‑bypass yield, outperforming existing baselines by a significant margin.

By Zhibo Zhang, Yuxi Li, Zhen Ouyang, Ling Shi, Kailong Wang
arXiv Computation and Language
Aug 25

GRACE: Step-Level Benchmark for Faithful Reasoning over Context

GRACE is a step‑level benchmark for evaluating the faithfulness of chain‑of‑thought reasoning over context. It provides human annotations for each step in CoT traces from 10 models across 4 datasets, labeling faithfulness, error category, and natural‑language explanations. The benchmark introduces a data‑driven taxonomy that splits errors into GRACE‑Inference (deductive) and GRACE‑Grounding (factual) tracks, each with four categories, and demonstrates that incorporating step‑level faithfulness signals can improve downstream accuracy and reasoning reliability.

By Hoang Pham, Dong Le, Anh Tuan Luu
arXiv AI
Aug 25

LLM-Based Adversarial Persuasion Attacks on Fact-Checking Systems

The paper introduces a new type of adversarial attack on automated fact‑checking systems that uses large language models to rephrase claims with persuasive techniques. By applying 15 persuasion methods across five categories, the authors evaluate how these rewrites affect claim verification and evidence retrieval on the FEVER and FEVEROUS benchmarks. Results show that persuasive rewrites significantly degrade both verification accuracy and evidence retrieval performance, underscoring the vulnerability of current fact‑checking systems to such attacks.

By Jo\~ao A. Leite, Olesya Razuvayevskaya, Kalina Bontcheva, Carolina Scarton