AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,788 stories · RSS feed

arXiv Computer Vision
Oct 19

Stereo LWIR-RGB Automatic Labeling for Low-Light Camera-Based VRU Detection

The paper introduces an automated pipeline for labeling vulnerable road users (VRUs) in low-light conditions using stereo LWIR-RGB image pairs. By sourcing labels from LWIR data, which highlights VRUs through body temperature, the method reduces hardware complexity compared to previous setups. Experiments on a Finnish dataset show that models trained on these auto-generated labels outperform off-the-shelf baselines and hand-labeled fine-tuned models on several metrics.

By Dimitrios Bouzoulas, Pyry J\"appinen, Pejman Habibiroudkenar, Eerik Alamikkotervo, Risto Ojala
arXiv AI
Oct 19

RECREG: Recovery-based Shielding for Regular Safety Properties in Reinforcement Learning

The paper introduces RecReg, a safe reinforcement learning framework that enforces non‑Markovian regular safety properties through recovery‑based shielding. It provides theoretical foundations and rigorous safety and convergence guarantees in the tabular setting, and demonstrates empirical effectiveness using the MASA‑Safe‑RL library. The authors also outline an extension of RecReg to deep RL with state‑of‑the‑art model‑based methods.

By Alexander W. Goodall, Francesco Belardinelli
arXiv Computation and Language
Oct 19

VISTA: Steering LLMs with Latent User Values Inferred from a Few Survey Answers

VISTA is a method that steers large language models by inferring a latent user value profile from a few survey answers and applying a hypernetwork to generate user‑specific interventions. It outperforms or matches in‑context profiling on synthetic and real datasets (Food2D, World Values Survey, PrimeX) without adding prompt tokens. VISTA also adaptively selects which survey questions to ask, stopping after only 6% of the questionnaire while preserving 94% of the full‑history benefit, and identifies a clear value axis that accurately positions unseen users.

By Yinuo Xu, Jadie Adams, Brian Hu, Shivani Kumar, David Jurgens, Arslan Basharat
arXiv AI
Oct 19

LocalIntent: Outcome-Specific Structured Generation for Backend-Compatible Smart Building Automation with Local LLMs

The paper introduces LocalIntent, a structured generation framework that routes smart building automation requests to either ordinary or persistent-rule branches using locally deployed large language models. LocalIntent improves outcome-specific prompting, yielding higher accuracy in intent understanding, device identification, and action/parameter correctness compared to generic two-stage prompting, especially for rule-based requests. Experiments on Gemma and Qwen models (2B‑9B) across 101 non‑persistent and 54 persistent‑rule records demonstrate significant performance gains in rule-mode correctness and overall intent handling.

By Tianyang Liu, Jiayue Wang, Zhanhao Cheng, Changyang Wang, Shaoshu Zhu, Zhenghao Wu, Hadi Tabatabaee Malazi
arXiv Computer Vision
Oct 19

SCALE: Improving Guidance-Free Visual Generation via Self-Contrastive Alignment for Likelihood Estimation

The paper introduces Self-Contrastive Guidance (SCG), a method that contrasts a model’s predictions on real versus self-generated samples to directly guide generation toward the true data manifold. SCG is incorporated into the likelihood objective through reparameterization, resulting in the Self-Contrastive Alignment for Likelihood Estimation (SCALE) framework, which produces SCG-enhanced predictions in a single forward pass. Experiments on class-conditional and text-to-image generation with diffusion and autoregressive models show that SCALE consistently outperforms existing guidance-free baselines and further benefits from combining with Direct Preference Optimization (DPO).

By Jiwan Hur, Dong-Jae Lee, Dongyeun Lee, Gyojin Han, Junmo Kim
arXiv Computer Vision
Oct 19

LithoWorld: An Endpoint World Model for Lithography Process-Window Recovery

LithoWorld is an endpoint world model designed to recover lithography process parameters using scanning electron microscopy (SEM) guidance. It jointly uses interior representative targets to define qualified reference appearances and correction supervision, aligning current image features with endpoint representations via a causal predictive branch and generating two‑dimensional corrections through a conditional flow policy. Trained on 173 groups of measured SEMs, the model achieves 88.5% qualified‑region success and 46.0% interior success on a 200‑pair test set, outperforming the best comparison method by 2.5 and 6.0 percentage points, respectively.

By Botong Zhao, Fang Yu, Xinyuan Chen, Yue Lu
arXiv Computer Vision
Oct 19

ESC: Escaping 2D Diffusion Ambiguity with Geometric Constraints for Consistent Novel View Synthesis

The paper introduces ESC, a framework that improves 3D-consistent novel view synthesis from a single 2D image by jointly generating a target view and its depth map. By conditioning the diffusion model on the source image, source depth, and target camera rays, ESC forces the model to reason in 3D even under extreme viewpoint changes. A per‑pixel depth consistency constraint is added to enforce spatial fidelity, resulting in superior visual accuracy and spatial alignment compared to current state‑of‑the‑art methods.

By Ge Ya Luo, Daniel Severo, Brian Karrer, Christopher Pal, Adriana Romero-Soriano, Michal Drozdzal
arXiv Computer Vision
Oct 19

Generative AI Produces and Transmits Beauty Bias

The paper audits 27 text‑to‑image generators released between 2022 and 2026, examining 51,300 AI‑generated faces and 160,196 real photographs. Using automated metrics and 550 human raters, it finds that AI‑generated faces consistently score higher on attractiveness than real faces, with female‑prompted images often surpassing professional models and darker‑skinned groups receiving lower scores. A preregistered hiring simulation shows that exposure to AI‑generated attractive faces reduces hireability and trustworthiness ratings for real candidates, whereas retouched photographs have no such effect.

By AJ Alvero, Deepak Varuvel Dennison, Alexandra Crespin, Heeyon Kim, Alejandra Regla-Vargas, Avani Akerkar, Jasmine McNealy, Ren\'e Kizilcec
arXiv AI
Oct 19

A Gradient-Optimized TSK Fuzzy Framework for Explainable Phishing Detection

A novel phishing URL detection system is introduced that employs a first-order Takagi-Sugeno-Kang (TSK) fuzzy inference model optimized via gradient-based techniques, specifically using the Adam optimizer. The framework combines fuzzy logic’s interpretability with gradient optimization’s precision, achieving an average accuracy of 99.95% and a perfect AUC of 1.00 across 5 cross-validation folds on a dataset of over 235,000 URLs. The optimized fuzzy rules and membership functions provide clear decision pathways, enhancing transparency for cybersecurity applications.

By Lohith Srikanth Pentapalli, Jon Salisbury, Josette Riep, Kelly Cohen
arXiv AI
Oct 19

Beyond Uncertainty Reduction: Calibrated Planning for Interactive Diagnosis

The paper introduces CaRDI, a calibrated planning framework for interactive medical diagnosis that uses a clinical loss combining diagnosis error and severity-weighted omission. Unlike prior methods that focus solely on uncertainty reduction, CaRDI balances expected risk reduction with interaction cost, employing diagnostic and safety beliefs along with conformal residual calibration. Experiments on multiple medical benchmarks demonstrate that CaRDI improves diagnostic accuracy, reduces the number of questions asked, and lowers clinical loss by 21.5% while decreasing severe omissions.

By Yue Zhou, Haiyang Zhou, Jin Zhang, Kong Wang, Zihao Rao, Maolin Wang, Hanwen Du, Youhua Li
arXiv Computation and Language
Oct 19

Identify, Classify, Act: Per-Instruction Controlled Compliance for LLMs over External Data

The paper introduces EDIC, a framework for per‑instruction compliance in large language models that interact with external data. It decomposes the task into identifying, classifying, and acting on each embedded directive, using a scene‑grounded pipeline and a generative judge trained with GRPO to attribute compliance at the instruction level. Experiments on two backbones and five benchmarks show that EDIC improves safety and utility, achieving up to a 7.7% increase in harmful‑rejection and a 12.5% boost in Arena‑Hard‑Auto utility on Qwen3‑4B.

By Zesheng Shi, Xue Li, Chenrui Dai, Jing Li
arXiv AI
Oct 19

Beyond Boundaries: A Comprehensive Survey of Transferable Attacks on AI Systems

The paper surveys transferable attacks on AI systems, noting that such attacks can be crafted in one setting and then applied to others, thereby lowering costs and increasing security risks. It introduces a unified taxonomy across six axes—cross-instance, cross-domain, cross-modality, cross-model, cross-task, and cross-deployment—while distinguishing source-target changes from attacker resources. The survey also links transfer mechanisms to enhancement methods, highlights inconsistencies and gaps in current research, and proposes a research agenda for better benchmarking and defense strategies.

By Guangjing Wang, Ce Zhou, Yuanda Wang, Bocheng Chen, Hanqing Guo, Qiben Yan
arXiv AI
Oct 19

Learning to Reason About Implicit Multimodal Harm: From Diagnosis to Generalization

The paper introduces MuPHI, a diagnostic benchmark that isolates multimodal harm arising solely from image‑text composition, providing benign counterfactuals and annotated harm rationales. Evaluation with MuPHI shows that vision‑language models can ground individual modalities but struggle to bind evidence into a coherent harm mechanism, and that existing harm detection methods transfer poorly to new harm distributions. To address this, the authors propose Compositional Harm Interpretation Learning (CHIL), which rewards evidence identification, cross‑modal binding, and verdict consistency, leading to significant improvements in out‑of‑distribution transfer and counterfactual robustness.

By Anisha Saha, Varsha Suresh, Teodora Kamova, Sophia Wiedmann, Timothy Hospedales, Vera Demberg
arXiv Computer Vision
Oct 19

STC-MGNet: Spatial-Temporal-Channel Attention-Enhanced Multimodal Graph Network for Pedestrian Intention Prediction

The paper introduces STC-MGNet, a spatial‑temporal‑channel attention‑enhanced multimodal graph network designed to predict pedestrian crossing intentions. It employs a pose graph encoder with three attention branches that focus on skeletal joints, observation frames, and feature channels, and integrates visual and ego‑vehicle motion data through a multi‑stage cross‑modal channel modulation mechanism. Experiments on JAADbeh, JAADall, and PIE datasets show accuracy rates of 70%, 87%, and 88%, surpassing several baseline methods.

By Yuanzhe Li, Meng Wang
arXiv AI
Oct 19

Rethinking the design space of Vanilla Pixel Diffusion Transformers

The paper revisits pixel-space diffusion models, showing that a vanilla diffusion transformer can generate directly in pixel space by swapping velocity prediction for clean image prediction. It introduces MFlow, an adaptive parameterization that interpolates between these two prediction modes based on data representation and model capacity, and demonstrates competitive performance on ImageNet, UCF‑101, and text‑to‑image tasks. Additionally, the authors add ConvRefiner, a lightweight convolutional module trained post‑hoc to restore high‑frequency details while preserving FID.

By Soumik Mukhopadhyay, Long Mai, Abhinav Shrivastava, Aniruddha Mahapatra
arXiv AI
Oct 19

HEM-Med: A Hierarchical Agentic Memory Framework for Safe Multi-Center Medication Recommendation

HEM-Med introduces a hierarchical agentic memory framework for safe multi‑center medication recommendation, addressing two key gaps in existing methods. It disentangles cross‑hospital clinical experience from hospital‑specific prescribing patterns via a hierarchical memory module, and incorporates a safety‑aware medication set generation module that considers drug‑drug interactions and local formulary constraints. Experiments on a real‑world multi‑center eICU dataset demonstrate that HEM‑Med outperforms representative transfer learning, LLM‑based, and MR baselines in both recommendation accuracy and safety.

By Min Xiao, Jin Li, Shoujin Wang, Yeguang Zhen, Yishuo Li, Wenpeng Lu
arXiv AI
Oct 19

Cross-Layer Attention Probing for Fine-Grained Hallucination Detection

Cross-Layer Attention Probing (CLAP) is a new activation‑probing method that examines the entire residual stream of Large Language Models (LLMs) to detect hallucinations. Experiments on five LLMs across three tasks demonstrate that CLAP outperforms baseline detectors for both greedy and temperature‑sampled outputs, enabling fine‑grained discrimination between hallucinated and non‑hallucinated responses. The approach also supports a detect‑then‑mitigate strategy that reduces hallucinations and maintains reliability even on out‑of‑distribution data.

By Malavika Suresh, Rahaf Aljundi, Ikechukwu Nkisi-Orji, Nirmalie Wiratunga
arXiv Machine Learning
Oct 19

Reinforcement Learning in MDPs with Information-Ordered Policies

The paper introduces a framework for reinforcement learning in Markov decision processes where policies are ordered by the information they provide, allowing data from one policy to estimate the performance of others. An epoch‑based algorithm is developed for infinite‑horizon average‑cost MDPs with continuous state and action spaces, achieving regret that scales with the width of the policy class’s information order. The authors apply the framework to lost‑sales inventory, dual‑sourcing, and state‑dependent queueing systems, demonstrating improved regret bounds and empirical performance without requiring global convexity or specific arrival‑rate structures.

By Zhongjun Zhang, Shipra Agrawal, Ilan Lobel, Sean R. Sinclair, Christina Lee Yu
arXiv Machine Learning
Oct 19

Towards Hyperparameter Transfer for Differentially Private Optimization

The paper introduces DP‑µP, a framework that extends Maximal Update Parametrization to differential privacy settings. By analyzing gradient clipping, noise injection, and feature learning, it derives a signal‑to‑noise ratio condition and spectral scaling rules for DP‑SGD, DP‑Adam, and DP‑Muon optimizers. Experiments show that DP‑µP enables zero‑shot transfer of learning rates across model widths on MLP, ViT, and GPT‑2, matching or surpassing tuned baselines while cutting computational costs.

By Tianze Wang, Zhiqi Bu, Linjun Zhang