AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,788 stories · RSS feed

arXiv Machine Learning
Oct 15

Lost in Translation: Why Lower Bias Scores Don't Mean Fairer Models

The paper investigates whether lower stereotype bias scores in multilingual language models truly indicate greater fairness. By auditing ten open models across eight languages, the authors show that lower average scores often arise from translation artifacts rather than genuine bias reduction, as item‑level correlations between English and translated scores are weak. They also identify pipeline errors that can mask bias signals and provide code to replicate their findings.

By Harsh Kumar
arXiv Machine Learning
Oct 15

Hierarchical Residual Policy Learning for Real-World Mobile Manipulation with Sparse Human Guidance

The paper introduces HiRe-MoMa, a hierarchical residual policy learning framework that fine‑tunes a frozen base policy for mobile manipulation. It uses a policy‑agnostic residual split into mobile‑base and manipulator‑arm heads to reduce gradient interference, and incorporates expert‑guided value alignment to improve sample efficiency with sparse human interventions. Experiments on the TIAGo++ platform show that HiRe‑MoMa boosts task performance after only about 30 minutes of online interaction, regardless of the underlying base policy.

By Zhefei Gong, Hefeifei Jiang, Zehao Wang, Jiachen Li
arXiv AI
Oct 15

Scaling Vision Transformers Without Training: Prune-and-Merge Token Compression

The paper introduces Prune-and-Merge ViT, a training‑free token compression framework for Vision Transformers that combines class‑attention pruning with similarity‑based merging to reduce computational cost while preserving semantic information. It evaluates robustness using two new metrics—Relative Accuracy Drop (RAD) and Prediction Flip Rate (PFR)—and shows that the method maintains or improves accuracy on pretrained ViT and DeiT models compared to existing token reduction techniques. The authors also argue that the hybrid approach does not induce hallucination‑like behavior and keeps predictions stable under common image transformations.

By Vatsal, Soumil Malhotra, Priyanshu Chahal, Sanjeev Kumar
arXiv AI
Oct 15

Cost-Efficient Privacy Enforcement for Email-Generating LLM Agents under Strategic Information Requests

The paper proposes a cost‑efficient privacy enforcement method for email‑generating large language model (LLM) agents that intercepts drafts, creates an information‑release contract, rewrites the draft locally with a Gemma rewriter, and then optionally calls a final PrivacyChecker. Experiments on normal and strategically malicious prompts show that the Gemma + Checker approach reduces leakage from 11.41 % to 4.03 % and improves usable output rates, while also cutting the cost amplification of checker‑only pipelines from 1.32× to 1.02×. The results demonstrate that moving repair to a cheaper local layer can significantly lower both privacy leakage and operational cost, yet still rely on a checker for high‑risk sends.

By Qikun Cai, Tao Huang, Chen Hou
arXiv AI
Oct 15

Checking the Checkers: A Synthesis of Verifier Noise and Robustness in RLVR

The paper surveys 58 studies on verifier errors in reinforcement learning with verifiable rewards (RLVR), examining how such errors affect training and which mitigations are effective. It finds that the type of error—rather than its frequency—has a larger impact on learning dynamics, with random errors mainly slowing progress and false positives potentially causing learning plateaus or collapse. The authors compile findings into comparison tables, reconcile conflicting reports, and propose a reporting protocol to standardize future research.

By Savyasachi Gupta, Aniruddha Srinivas Joshi
arXiv AI
Oct 15

Visual Dependence Does Not Imply Answerability: A Systematic Analysis of Answerability Signals in Frozen InstructBLIP

The study evaluates whether signals from a frozen vision‑language model, InstructBLIP, can reliably indicate answerability in visual question answering. Using a 10K subset of UNK‑VQA, the authors test predictive uncertainty, cross‑modal alignment, visual grounding, visual dependence, internal representations, and learned probes, finding that all signals yield weak discrimination (AUROC 0.50‑0.54). Even interventions that remove question‑relevant visual evidence or increase probe capacity do not substantially improve answerability detection, underscoring that visual dependence and related metrics are not robust proxies for answerability.

By Minjun Kim, Hyunmin Song, Changwon Lim
arXiv AI
Oct 15

SpecCloak: Probing Harmful Code Generation through Engineering Specifications

SpecCloak investigates how presenting coding tasks as engineering specifications—via the PseudoSpec template—affects large language models’ refusal to generate harmful code. Experiments on five open-weight models show that specification prompting significantly lowers refusal rates even when risk keywords remain, shifting internal representations toward code-output contexts. Building on this, SpecCloak combines PseudoSpec with discriminator-guided optimization to achieve a 28.91% attack success rate, while SpecReveal offers a defense that detects such attacks with 96.88% accuracy without fine‑tuning the target model.

By Wenbin Chen, Jiexin Wang, Guanting Wu, Jiaxi Liang, Yi Cai
arXiv AI
Oct 15

LinMorph: Learning to Morph Weights for Efficient Video DiT Linearization

LinMorph proposes a learnable softmax‑to‑linear weight transformation for Video Diffusion Transformers, replacing traditional random initialization or manual weight reuse. By learning a shared morphism that maps each target Gated DeltaNet parameter to a combination of pretrained query, key, value, and output projections, it generates layer‑specific initializations that preserve pretrained knowledge. The method enables faster recovery and higher generation quality across pretrained Video DiTs, especially when applied to larger video scales after training on short, low‑resolution clips.

By Lingyi Long, Congyue Zhao, Xiaoou Li, Yanmin Qian, Tao Huang
arXiv AI
Oct 15

MemoryVLN: From Ego-Aware Memory Representation to Cross-Context Memory Reuse in Vision-Language Navigation

MemoryVLN introduces a memory-driven navigation system that compresses long navigation histories and reuses past spatial experience. It uses forgetting-curve-inspired temporal sampling for long sequence compression, an ego-aware memory representation that fuses 2D semantic, 3D geometric, and 6D pose tokens, and a hierarchical memory bank for in-episode and context-level memory management. The system retrieves top‑K relevant memories to augment current visual tokens, achieving significant improvements on R2R‑CE and RxR‑CE benchmarks.

By Xueyao Wei, Yifei Wang, Delun Li, Shaoan Wang, Junzhi Yu, Wenjie Song, Xingyu Chen
arXiv AI
Oct 15

inLoRA-MINT: Internal Representations for Auditing Training Data in Domain-adapted LLMs

The paper introduces inLoRA-MINT, an extension of the LoRA-MINT framework that uses intermediate model representations—such as activations and parameter-dependent signals—to perform Membership Inference Tests on large language models fine‑tuned with LoRA. By leveraging these representations, the method can distinguish training from non‑training samples without full parameter access, achieving 95–98% accuracy across four LLMs (Qwen, Phi‑4, DeepSeek‑V3, LLaMA 3.2‑3B) on medical, mathematical, and academic reasoning datasets. The approach offers a scalable way to audit sensitive training data and supports compliance with emerging privacy and AI governance regulations.

By Gonzalo Mancera, Julian Fierrez, Aythami Morales, Ruben Tolosana, Daniel DeAlcala, Francisco Jurado
arXiv AI
Oct 15

How Well Do LLMs Perform in Peer Review? Evidence from Finance and Accounting

The study evaluates how well large language models (LLMs) can perform peer review tasks in finance and accounting. Using GPT‑4o, the authors had the model review papers presented at the 2024 AFA and NBER finance meetings and compared its assessments with subsequent publication outcomes in top finance journals. They found that higher model scores on correctness and exposition predict a 44% higher likelihood of publication, while judgments on originality and significance do not influence outcomes, suggesting LLMs are useful for technical checks but not for evaluating a paper’s broader impact.

By Hai Lu, Yichen Luo, Luping Yu
arXiv AI
Oct 15

FALLNEST: Dual-Domain Dataset Condensation of Imbalanced IMU Data for Wearable Fall Detection

The paper introduces FALLNEST, a data‑centric framework that condenses large, imbalanced inertial measurement unit (IMU) datasets into tiny, class‑balanced synthetic training sets for wearable fall detection. FALLNEST builds on dual‑domain time‑series condensation, adding a fall‑specific 7‑channel input, class‑balanced K‑Means initialization, normalized gradient matching with clipping, MMD‑based embedding alignment, and multi‑view Fourier‑domain augmentation. Experiments on SisFall, KFall, and MobiFall show that a 102K‑parameter TinyCNN trained on only 32 to 146 synthetic windows (≈1% of the original data) achieves 93.5–95.8% mean accuracy and 0.89–0.98 PR‑AUC with low variance.

By Rashi Niyas P, Ritika Jain, Hitika Tiwari
arXiv AI
Oct 15

Agent-Environment Alignment via Automated Interface Generation

The paper introduces ALIGN, a lightweight framework that automatically generates enriched interfaces to reduce agent‑environment misalignment in large language model agents. By augmenting static environment information and step‑wise observations, ALIGN improves performance across diverse tasks such as embodied navigation, web browsing, and tool use, achieving up to a 45.67% success rate increase in ALFWorld. The method works without altering agent logic or environment code and generalizes across different agent architectures and LLM backbones.

By Kaiming Liu, Xuanyu Lei, Ziyue Wang, Peng Li, Yang Liu
arXiv AI
Oct 15

Multi-agent Undercover Gaming: Hallucination Removal via Counterfactual Test for Multimodal Reasoning

The paper introduces Multi-agent Undercover Gaming (MUG), a protocol that extends the Multi-Agent Debate (MAD) framework to address hallucinations in large language models. MUG reframes MAD as a detection task for ‘undercover’ agents—those that hallucinate—by using multimodal counterfactual tests that modify reference images to reveal hallucinations. The approach enhances factual verification, cross‑evidence reasoning, and active discussion among agents, aiming to improve reliability in multimodal reasoning.

By Dayong Liang, Xiao-Yong Wei, Linhui Xiao, Changmeng Zheng
arXiv AI
Oct 15

LaCache: Robust Semantic Caching for LLM Serving

LaCache introduces a redesigned semantic caching mechanism for large language model (LLM) serving that defends against cache‑collision attacks. By shifting integrity checks from queries to the first $k$ speculatively decoded tokens of the LLM response, LaCache can detect adversarial cache pollution with provable high probability. Experiments across multiple LLMs and benchmarks show that LaCache reduces attack hit rates to near zero while retaining over 90% of benign cache utility.

By Jiacheng Liang, Yuhui Wang, Tanqiu Jiang, Ting Wang
arXiv AI
Oct 15

CockpitHAT: Dependency-Graph-Driven Hierarchical Attribution for Embodied Multi-Agent Cockpits

CockpitHAT is a hierarchical attribution framework for embodied multi‑agent systems that replaces positional windows with dependency‑distance thresholds derived from interaction DAGs, incorporates multi‑channel evidence through an embodied adapter, and applies a safety‑uplift to high‑risk failures during confidence‑weighted analyst consensus. The authors also release CockpitBench, a benchmark of 212 annotated failure traces labeled with ISO 26262 ASIL severity. On both the public Who&When benchmark and CockpitBench, CockpitHAT outperforms the text‑only state‑of‑the‑art ECHO, achieving higher agent‑level and step‑exact accuracies across multiple splits.

By Wei Wang, Shuanghe Liu, Zhu Zhuo, Jiaqi Zhong, Xiaozhao Zhao, Xiaojie Zuo, Jie Su
arXiv AI
Oct 15

Right Answer, Wrong Reason: Accuracy, Consistency, and Consensus Are Misleading Indicators of LLM Faithfulness in Clinical Decision Support

The paper introduces three interpretable metrics—Explanation Stability Index (ESI), Causal Faithfulness Score (CFS), and Perturbation Stability Score (PSS)—to assess the faithfulness of large language models (LLMs) in clinical decision support. Evaluating six LLMs on 150 MedQA-USMLE questions, the study finds that only 23.3% of cited clinical concepts are causally necessary, and that correct answers often have lower CFS than incorrect ones, with answer consistency negatively predicting faithfulness. These findings suggest that accuracy, consistency, and consensus are incomplete safety signals for clinical decision-making support.

By Bharath Kumar Bolla, Chaitali Deb Purkayastha, Vishnu Surya Reddy Nandi