AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,884 stories · RSS feed

arXiv Machine Learning
Sep 3

Scalable Direction-Following TTS via Voice Impression-Guided Pseudo Triplet Construction

The paper introduces a scalable method for creating pseudo-triplet data—comprising a reference utterance, a direction text, and a modified utterance—to train direction‑following text‑to‑speech (TTS) systems. It uses an impression‑controllable TTS model to generate style variations and a large language model to generate natural language directions from estimated impression differences. Experiments show that these pseudo‑triplets enable stable speaker‑preserving modifications, and combining them with recorded data further improves direction alignment while maintaining speaker similarity.

By Kenichi Fujita, Yusuke Ijima
arXiv Machine Learning
Sep 3

Beyond State Consistency: Behavior Consistency in Text-Based World Models

The paper introduces a new training paradigm for text-based world models that prioritizes behavior consistency over traditional state consistency metrics. It proposes the Behavior Consistency Reward (BehR), a step-level metric that evaluates how the likelihood of a logged next action changes between real and predicted states using a frozen Reference Agent. Experiments on WebShop and TextWorld demonstrate that BehR-based training improves long-term alignment, reduces false positives in offline evaluation, and yields modest gains in lookahead planning while maintaining or enhancing single-step prediction quality.

By Youling Huang, Guanqiao Chen, Junchi Yao, Lu Wang, Fangkai Yang, Chao Du, ChenZhuo Zhao, Pu Zhao, Qingwei Lin, Saravan Rajmohan, Dongmei Zhang
arXiv Machine Learning
Sep 3

Conditional Diffusion Posterior Alignment for Sparse-View CT Reconstruction

The paper introduces Conditional Diffusion Posterior Alignment (CDPA), a method that scales diffusion-based sparse‑view CT reconstruction to large 3D volumes by conditioning a 2D U‑Net diffusion model on an initial 3D reconstruction and enforcing data‑consistency alignment. CDPA addresses high memory demands, limited 3D training data, and slice‑wise inconsistencies, achieving state‑of‑the‑art performance on synthetic and real Cone Beam CT data. The authors also demonstrate that the same approach improves fast denoising U‑Nets, delivering near‑diffusion quality at a fraction of the computational cost.

By Luis Barba, Johannes Kirschner, Benjamin Bejar
arXiv Computation and Language
Sep 3

Grounded, Compute-Efficient LLM Policy Agents for Energy-Poverty Equity in Physically-Constrained Peer-to-Peer Energy Markets

The paper introduces EqGrid, a closed‑loop simulation that uses a low‑frequency, open‑weight LLM policy agent to set price, carbon limits, and subsidies for a community of empirically‑grounded household personas, while high‑frequency multi‑agent RL traders clear a continuous double auction on a physically constrained IEEE‑33‑bus grid. It demonstrates that the LLM can reduce energy‑poverty inequality—lowering the Gini of energy burden from 0.351 to 0.305 and mean burden by 28%—without increasing net grid cost, and that a compressed sub‑1B model retains 92–95% of this benefit at dramatically lower inference energy. The study also establishes a compute‑efficiency frontier and a decoupled‑safety design that eliminates grid violations. whyItMatters":"By showing that a lightweight LLM can effectively manage energy markets to reduce poverty and inequality while staying energy‑efficient, the work offers a practical, low‑carbon AI solution for humanitarian energy‑poverty interventions."

By Kunal Jadhav, Siddhesh More
arXiv Computation and Language
Sep 3

Improving Health Literacy through Lay Summarization of Radiological Reports: An Evaluation of BioNER and Retrieval-Augmented Generation

The paper examines how Retrieval-Augmented Generation (RAG) and Named Entity Recognition (NER) affect the quality of lay summaries of radiology reports. Using a framework that extracts clinically relevant findings via NER and grounds them with RAG, the authors evaluate few‑shot and fine‑tuned versions of Qwen and BioBART. Results show that NER consistently improves readability and overall quality, RAG alone offers no benefit and can introduce hallucinations, and the best performance comes from fine‑tuned BioBART with NER.

By Egecan \c{C}elik Evgin, \.Ilknur Karadeniz, Olcay Taner Y{\i}ld{\i}z
arXiv Computer Vision
Sep 3

Chameleon: Style-Content Disentangled Framework for Cross-Domain Object Compositing

The paper introduces Chameleon, a two‑stage training framework for cross‑domain image compositing that separates style and content representations. It first trains a ChameleonEncoder using Joint Hard Contrastive Learning to disentangle style and content, then applies Spatio‑Temporal Attention Gating within a diffusion transformer to stylize the foreground while preserving its identity. The authors also release ChameleonDataset, the first large‑scale training set for cross‑domain compositing, and demonstrate that Chameleon outperforms existing in‑domain, cross‑domain, and commercial models in both plausibility and stylistic fidelity.

By Sukhun Ko, Soo Ye Kim, Jihyong Oh
arXiv Machine Learning
Sep 3

FairLens: Benchmarking Fairness in Vision-Language Models for High-Stakes Decision-Making

FairLens is a benchmark and evaluation framework that measures fairness and validity of vision‑language models (VLMs) in high‑stakes domains such as hiring, legal, and healthcare. It uses over 100,000 face‑image and question pairs covering gender, race, and age, and assesses responses through demographic parity, soundness, demographic association, and bias in free‑text generation. The study finds that VLMs often make unwarranted inferences from faces rather than abstaining, especially in legal and healthcare contexts, and that small parity gaps can still hide unsafe treatment across groups.

By Vahid Reza Khazaie, Ahmed Y. Radwan, Shaina Raza
arXiv Machine Learning
Sep 3

CAPTURE: Disentangling Preference Drift from Memory Poisoning in Personalized LLM Agents

The paper introduces CAPTURE, a system designed to help personalized language agents distinguish genuine preference changes from temporary context shifts or malicious memory poisoning. CAPTURE employs a neural differential-equation belief tracker, a multi-timescale memory ledger, uncertainty-triggered clarification, and counterfactual auditing to resolve ambiguity. Experiments on 480 episodes from 96 users show CAPTURE outperforms baseline methods, limiting poisoning success while accepting most real preference updates.

By S M Asif Hossain, Ruksat Khan Shayoni, Md Kishor Morol
arXiv Computation and Language
Sep 3

Efficient GUI Agents: A Systems Survey of Observation, Memory, Action, and Runtime Optimization

The paper surveys efficient GUI agents, emphasizing that practical deployment requires more than task success—it must also minimize context, computation, action budget, and runtime overhead. It reviews key efficiency dimensions—observation, memory, action, and planner/system—and identifies recurring strategies such as selective reading, global-to-local visual allocation, recoverable memory, verification-aware control, and hybrid runtimes. The authors highlight open challenges, including accurate verifier cost accounting, benchmark comparability, and co-design of observation, memory, and execution layers under real latency and privacy constraints.

By Bizhe Bai, Jiakang Yuan, Hongming Wu, Xinyue Wang, Jie Ren, Siyao Chen, Yuchen Ya, Fan Bai, Pai Peng, Huafeng Qin, Tao Chen
arXiv Computation and Language
Sep 3

Debias-SparseGPT: Bias-Aware Pruning for Large Language Models

Debias‑SparseGPT is a post‑training pruning technique that adds a representational debiasing term based on demographically contrasting inputs to mitigate bias amplification caused by weight sparsification. The method is validated across various generative LLMs and sparsity levels (25%, 50%, and structured 2:4), consistently reducing pruning‑induced bias while maintaining perplexity and zero‑shot accuracy. In the most aggressive 2:4 sparsity regime, enriching the calibration set with long‑context, content‑rich examples further improves both downstream performance and fairness.

By Irina Proskurina, Guillaume Metzler, Antoine Gourru, Julien Velcin
arXiv Machine Learning
Sep 3

Hearing the Whispers: Black-Box Membership Inference Attacks on Finetuned TTS Models

The paper introduces a black-box membership inference attack framework tailored for fine-tuned text-to-speech models, addressing challenges in query generation and representation engineering. It evaluates five query types, finding recitation queries most effective, and uses multi-level speech embeddings with temporal alignment for fine-grained comparison. Experiments on CosyVoice2, F5-TTS, and XTTS-v2 trained on VCTK and British Dialect datasets show high privacy leakage, with speaker-level AUC above 0.80 and record-level AUC between 0.80 and 0.90.

By Kunlin Cai, Kaiyuan Zhang, Zihang Xiang, Jinghuai Zhang, Abeer Alwan, Fnu Suya, Yuan Tian
arXiv Machine Learning
Sep 3

Linear Fusion MultiDiffusion for Fast Training-Free Spherical Panorama Generation

LF-MultiDiffusion is a training‑free method for generating spherical panoramas that extends MultiDiffusion by adding linear projections between target and reference image spaces. It reformulates latent aggregation as a regularized least‑squares problem and solves it with a Krylov‑based iterative solver during denoising, enabling denser and more natural mappings. The approach reduces the number of generator evaluations, improves inference speed by 15.36×, and yields better visual quality, text alignment, and panoramic consistency compared to the strongest training‑free baseline.

By Akio Hayakawa, Yusuke Mukuta, Tatsuya Harada
arXiv Machine Learning
Sep 3

IDEEA: training-free Input-Dependent stEEring via Activation cluster matching

IDEA is a training‑free, input‑dependent steering method for large language models that matches activations to cluster‑specific directions aligned with a target concept. It clusters positive and negative activation supports per attention head, solves an optimal‑matching problem to create a pool of cluster‑conditional directions, and selects the best match for each input at inference time. This approach preserves the input’s original representation while improving the truth × info rate on TruthfulQA by an average of 9.9% (up to 23.5%) over input‑independent baselines.

By Zheng Wang, Muchen Li, Renjie Liao, Yan Leng
arXiv Machine Learning
Sep 3

GenCAR: Generative Counterfactual Alignment with Risk-Controlled Selection for Out-of-Distribution Recommendation

GenCAR introduces a method for out‑of‑distribution recommendation that balances utility and risk by controlling the proxy‑label false discovery rate (FDR). It frames the problem as an α‑Valid Counterfactual Recommendation (α‑VCR) task, coupling counterfactual supervision with calibrated set selection using conformal p‑values and Benjamini–Hochberg filtering. The approach theoretically bounds counterfactual approximation error and guarantees finite‑sample, distribution‑free FDR control under various dependence assumptions, and empirical results show improved OOD candidate recovery across benchmarks.

By Qianqian Wang, Yunshan Li, Jiawen Zeng, Wenwu Gong, Lili Yang
arXiv Machine Learning
Sep 3

Hardware-Accelerated Instance Segmentation for Resource-Constrained Space Robotics with Criticality Analysis

The paper presents a hardware‑accelerated instance segmentation framework tailored for resource‑constrained lunar robotics, addressing low‑light perception, limited compute, and radiation‑induced hardware faults. It introduces Activation Variance Informative Sampling (AVIS), a label‑free calibration method that selects samples based on activation variance, and deploys a YOLO‑based model on a Deep Learning Processor Unit with architectural tweaks to reduce CPU fallback and ensure bounded latency. A software‑level criticality analysis estimates fault exposure, guiding mitigation that reduces global criticality by 31.7%, while AVIS with bias correction recovers 69.8% of quantization‑induced accuracy loss at 309 ms latency and 5.7 W power consumption.

By Siddhant Shete, Hilmi Dogu K\"uc\"uker, Udo Frese, Frank Kirchner
arXiv Machine Learning
Sep 3

Orthogonal Ensembles and Tested Explanations for Performer-Independent Body-Motion Emotion Recognition

The paper investigates 12‑class body‑only emotion recognition from skeleton motion using a leave‑performer‑out evaluation, where chance accuracy is 8.3% and a reproduced STGCN++ baseline scores 25.73% Macro‑F1. By ensembling eleven models with orthogonal error modes, the authors achieve 36.80% Macro‑F1, a 43% relative improvement over the baseline. They also introduce a tested explanation suite that demonstrates the ensemble’s decisions rely on motion‑grounded body‑region evidence, aligning strongly with Laban Movement Analysis attributes rather than classical kinematics, while showing diffuse temporal saliency.

By Naoto Nishida, Yoshio Ishiguro
arXiv Machine Learning
Sep 3

Eliciting ESG Preferences for Reinforcement Learning-Based Portfolio Optimization

The paper proposes a Multi-Objective Reinforcement Learning framework for portfolio optimization that incorporates ratings from three ESG agencies, addressing the divergence in ESG rating methodologies. It couples this with a Preference Elicitation system using Gaussian Processes, allowing users to infer latent utility functions via pairwise comparisons of portfolios based on Sharpe ratios and ESG scores. Experiments with LLM-generated portfolio managers show that regional background influences preference weights, with European personas prioritizing ESG alignment and Texas personas favoring risk‑adjusted returns.

By Giovanni Dispoto, Marcello Restelli, Carmine Ventre
arXiv Machine Learning
Sep 3

CodePoisonRAG: Knowledge Poisoning Attacks on Retrieval-Augmented Code Generation

The paper introduces CodePoisonRAG, a framework that poisons retrieval-augmented code generation systems by transforming benign code artifacts into malicious ones. It injects CWE-specific vulnerabilities and false safety claims into a single task-matched artifact, achieving high success rates across multiple generators and even against a defense system. The study demonstrates that attackers can target and propagate specific weaknesses without altering the underlying language model.

By Varun Gadey, Ziad Marey, Alexandra Dmitrienko
arXiv Machine Learning
Sep 3

Full-Model Optimality for Tunable Linear Generative Priors in Compressed Sensing

The paper investigates compressed sensing using a family of tunable linear generative priors linked through their singular value decompositions. It proves that in noiseless Gaussian compressed sensing, the full-dimensional linear prior yields the lowest expected reconstruction error across the entire family, meaning lower-complexity priors do not improve performance in this idealized setting. This contrasts with denoising, where lower complexity priors can reduce error due to bias‑variance tradeoffs, suggesting that the experimental gains seen with neural network priors stem from their nonlinearities.

By Zhaoming Li, Paul Hand
arXiv Machine Learning
Sep 3

Exchange Policy Optimization Algorithm for Semi-Infinite Safe Reinforcement Learning

The paper introduces Exchange Policy Optimization (EPO), a framework for semi‑infinite safe reinforcement learning that handles infinitely many constraints by iteratively solving finite subproblems. EPO expands or deletes constraints based on tolerance violations and Lagrange multipliers, maintaining computational tractability while converging to an optimal policy with bounded safety violations. The authors prove finite convergence, provide iteration bounds, and quantify the suboptimality gap under mild assumptions.

By Jiaming Zhang, Yujie Yang, Haoning Wang, Liping Zhang, Shengbo Eben Li