AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,808 stories · RSS feed

arXiv Machine Learning
Sep 14

Fed-Equilibrium Framework for Topological Pareto Control in Robust and Fair Clinical Federated Learning

Fed-Equilibrium is a federated learning framework designed to address the knowledge dominance problem in multi-center clinical networks, where high-volume hubs overpower minority community nodes. It employs a two-stage gradient control cascade: Stage I uses a cosine similarity funnel for geometric quality assurance, while Stage II applies topological Pareto control to identify an optimal knee point for contribution modulation. Experiments on Canadian and U.S. registries show that the framework secures the network against adversarial divergence and enables minority U.S. nodes—comprising less than 3% of the data—to achieve deep convergence comparable to the data-rich Canadian hub.

By Ting Xu, Henry Leung
arXiv Computation and Language
Sep 14

I Am No One: Style-Aware Paraphrasing for Text Anonymization

The paper introduces a style-aware paraphrasing method for text anonymization that leverages pretrained large language models to build compact stylistic profiles from minimal samples and rewrite text to suppress identifiable style markers while preserving meaning. It demonstrates that this approach reduces authorship attribution F1 scores by 60‑70% on blog and review datasets, outperforming both differential privacy‑based and non‑DP baselines, and maintains content quality and readability.

By Ahmed Sohair Khan, Estrid He, Monica Wachowicz, Elham Naghizade
arXiv Computer Vision
Sep 14

Semantically Aligned Gradient-Driven Context-Preserving Image Editing

Semantically Aligned Gradient-Driven Context-Preserving Image Editing (IABEdit) is a model‑agnostic framework that embeds differentiable semantic verification into the training of generative image editors. By using a frozen vision‑language model to extract spatially‑aware descriptors from ground‑truth edits and a trainable aligner to reproduce them from generated outputs, the residual becomes a gradient that teaches the generator both what to edit and where, without adding inference‑time VLM cost. IABEdit is compatible with various backbones (e.g., U‑Net in Stable Diffusion and MMDiT in FLUX) and improves structural fidelity on MagicBrush, achieves state‑of‑the‑art instruction adherence on RealEdit and EMU Edit, and outperforms the proprietary Gemini agent on the D‑LORD surveillance benchmark under heavy occlusion. "whyItMatters":"IABEdit demonstrates that incorporating semantic verification during training can produce more accurate, well‑localized edits and outperform existing methods even in challenging surveillance scenarios, as shown by its superior metrics and human/GPT‑4o evaluations."

By Chiranjeev Chiranjeev, Muskan Dosi, Mayank Vatsa, Richa Singh
arXiv Computer Vision
Sep 14

PAVXploreRL: Physical-Action-Visual World Model Reinforcement Learning with Action Exploration

PAVXploreRL introduces a reinforcement learning framework that builds on a pretrained latent world model to explicitly optimize Physical Plausibility, Action Adherence, and Visual Fidelity (PAV) objectives. By combining in‑distribution expert trajectories with noise‑driven out‑of‑distribution action exploration, the method avoids reliance on paired video supervision and improves generalization. Experiments demonstrate a 5.6% average performance gain over pretrained baselines and more reliable policy evaluation with reduced overestimation bias.

By Han Wang, Zijun Wang, Shuoshuo Xue, Rui Cao, Fengjiao Chen, Xiaodan Liang, Roy Ka-Wei Lee
arXiv Computer Vision
Sep 14

BodhiPromptShield: Pre-Inference Prompt Mediation for Surface-Form Privacy Propagation in LLM Agent Pipelines

BodhiPromptShield is a policy‑aware mediation layer for LLM agent pipelines that detects sensitive text spans before they propagate, replacing them with typed placeholders, semantic abstractions, or secure tokens and restoring them only at authorized execution boundaries. In evaluations on AI4Privacy, PrivacyLens, and AgentDojo datasets, the system reduces identifier exposure to 7.4% and 1.8% respectively, and limits exact identifier leakage in final actions to 2.1–3.1%. While mediation preserves factual content according to automated metrics, human annotations show a significant drop in inferability from 100% to 24–53%, indicating the need for human validation of semantic‑leakage measures.

By Bo Ma, Jinsong Wu, Weiqi Yan
arXiv Machine Learning
Sep 14

Hidden in Rounds: Predicting the Time Cost of 802.11 Contention in Federated Learning

The paper investigates how the time cost of 802.11 contention affects federated learning. Using ns-3 simulations, the authors measure frame-delivery ratios and saturation throughput across various client densities and offered loads, then employ a FedAvg trainer that uses these ratios to estimate communication time. Across 720 runs with diverse datasets, partitions, densities, loads, and seeds, all models reached target accuracy within the round budget, with communication time-to-target increasing significantly as client density rose. The study also compares uniform and persistent heterogeneous participation, finding no statistically significant accuracy gap, though confidence intervals are wide. The results are specific to the evaluated configurations and do not generalize to all convergence or fairness scenarios.

By Satwat Bashir, Tasos Dagiuklas
arXiv Machine Learning
Sep 14

Robust Prototypical Networks for Few-Shot Sensor Fault Diagnosis

The paper introduces Multi-Episode Prototypical Networks (MEPN), a variant of prototypical networks that aggregates class prototypes from multiple disjoint support episodes to reduce prototype variance in few-shot sensor fault diagnosis. MEPN is evaluated on the DeFACTO sensor dataset with synthetic fault injections, achieving significantly higher one-shot accuracy than single-episode baselines while matching ProtoNet performance under a 10-sample support budget. The approach demonstrates that prototype accumulation improves stability without altering the encoder architecture.

By Mohammed Ayalew Belay, Amirshayan Haghipour, Pierluigi Salvo Rossi
arXiv Machine Learning
Sep 14

When Bias Pretends to Be Truth: How Spurious Correlations Undermine Hallucination Detection in LLMs

The paper examines a specific type of hallucination in large language models caused by spurious correlations—unintended, statistically prominent associations in training data such as surnames linked to nationalities. These hallucinations are confidently produced, persist regardless of model scaling or refusal fine‑tuning, and evade existing detection methods like confidence filtering and inner‑state probing. The authors use controlled synthetic experiments and evaluations on both open‑source and proprietary LLMs, including GPT‑5, to demonstrate the failure of current detection techniques and provide a theoretical explanation for why statistical biases undermine confidence‑based approaches.

By Shaowen Wang, Yiqi Dong, Ruinian Chang, Tansheng Zhu, Yuebo Sun, Kaifeng Lyu, Jian Li
MIT News AI
Sep 14

New method enables AI for safety-critical situations

The article introduces the "HardFlow" algorithm, a new method designed to improve the reliability of generative AI models in safety‑critical contexts. By enforcing strict requirements, HardFlow aims to ensure that the AI’s outputs meet high‑quality standards even when approximate solutions are insufficient.

By Adam Zewe | MIT News