AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,884 stories · RSS feed

arXiv Computation and Language
Sep 1

Context-Aware Interleaved Batching for WhisperX

Context-Aware Interleaved Batching for WhisperX proposes a new batching strategy that combines WhisperX’s fast intra-audio batching with Whisper’s sequential context retention. By using voice activity detection (VAD) to define segment boundaries, the algorithm stabilizes Whisper’s text conditioning and preserves continuous historical context across batched audio segments. Experiments on long‑form audio benchmarks show reduced Word Error Rate and improved proper noun transcription while maintaining high‑throughput inference speeds.

By Carlos Bain, Max Bain
arXiv Computation and Language
Sep 1

Distributional Validity and Calibration of a Korean Synthetic Persona Panel for Digital and AI Service Use: A Secondary-Data Validation Against the Korea Media Panel Survey

The study evaluates a Korean synthetic persona panel (NVIDIA Nemotron‑Personas‑Korea) conditioned on Gemini 3.5 Flash and EXAONE against the KISDI Korea Media Panel Survey. Across eight digital‑AI service‑use indicators and eight innovativeness constructs, the panels achieved mean absolute errors of 15–19 pp, with segment‑level errors up to 52.4 pp and correlation coefficients between 0.69 and 0.90. Holdout calibration reduced sex‑by‑age cell errors but still lagged behind direct real‑data estimation, indicating that synthetic panels are useful diagnostically but not as survey substitutes.

By Howard Kim, Keun Tae Cho
arXiv Computation and Language
Sep 1

The Fragility of Jailbreak Robustness Across Operational States

The study shows that jailbreak robustness in language models is highly sensitive to operational-state changes. Even minor alterations to system prompts, not intended to affect safety, can dramatically shift attack success rates across seven aligned models and three jailbreak methods. The authors link these variations to changes in hidden representations along a refusal-related axis, which can predict jailbreak outcomes.

By Yuna Park, Hwang Youn Kim, Yujin Kim, Won Woo Ro, Suhyun Kim, Jae-In Hwang
arXiv Computation and Language
Sep 1

Privacy-Preserving Generation of Clinical Narratives from Medical Terminologies

The paper introduces Term2Note, a method for generating full-length clinical notes under differential privacy constraints. It separates content and form, conditioning note sections on medical terms and applying distinct DP protections to terms and notes, followed by a DP quality maximizer. Experiments show that the synthetic notes closely match real clinical notes in statistical properties, and models trained on them perform comparably to those trained on real data, outperforming existing DP text generation baselines.

By Yuping Wu, Viktor Schlegel, Warren Del-Pinto, Srinivasan Nandakumar, Iqra Zahid, Yidan Sun, Hai Li, Usama Farghaly Omar, Amirah Jasmine, Arun-Kumar Kaliya-Perumal, Chun Shen Tham, Gabriel Connors, Anil A Bharath, Goran Nenadic
arXiv Computation and Language
Sep 1

Identifying and Mitigating Bottlenecks in Role-Playing Agents: A Systematic Study of Disentangling Character Profile Axes

The paper introduces a diagnostic framework to disentangle the effects of character profile axes—Familiarity, Structure, and Disposition—on large language model role‑playing agents. Experiments on 211 personas and five LLMs show that Familiarity and Structure have little impact, whereas Disposition, particularly immoral traits, consistently degrades performance. The authors propose Field‑Aware Contrastive Decoding (FACD), a training‑free method that mitigates this performance gap without harming moral‑character performance.

By Yonghyun Jun, Junhyuk Choi, Jeonghyun Park, Jihyeong Park, Liu Nicole Geumheon, Hwanhee Lee
arXiv Computation and Language
Sep 1

Different Demographic Cues Yield Inconsistent Conclusions About LLM Personalization and Bias

The paper examines how large language models (LLMs) respond to different demographic cues—such as names—when users seek advice, focusing on race and gender in a U.S. context. It finds that using different cues for the same group leads to only partially overlapping changes in model responses, producing inconsistent conclusions about personalization and unstable bias metrics. The authors argue that LLMs react to linguistic signals tied to cues rather than to stable demographic categories, and they call for evaluations that use multiple cues and consider underlying mechanisms.

By Manuel Tonneau, Neil K. R. Sehgal, Niyati Malhotra, Sharif Kazemi, Victor Orozco-Olvera, Ana Mar\'ia Mu\~noz Boudet, Lakshmi Subramanian, Samuel P. Fraiberger, Sharath Chandra Guntuku, Valentin Hofmann
arXiv Computation and Language
Sep 1

Investigating Social Bias Changes in Quantized Language Models

Post‑training quantization of large language models reduces memory usage but can alter social biases in ways that aggregate metrics miss. In a large‑scale study of 50 quantized models on PostTrainingBiasBench, the authors discovered a phenomenon called quantization‑induced bias flipping, where up to 21% of responses switch from biased to unbiased or vice versa, especially for uncertain predictions and stronger quantization (4‑bit vs 8‑bit). These flips lead to asymmetric impacts across demographic groups, with some groups experiencing up to an 18.6% worsening of bias while others improve by 14.1%, resulting in misleadingly neutral overall scores.

By Stanley Z. Hua, Sanae Lotfi, Irene Y. Chen
arXiv Computation and Language
Sep 1

UReason: Benchmarking Reasoning-to-Generation Alignment in Unified Multimodal Models

UReason is a benchmark that evaluates how well unified multimodal models (UMMs) align textual reasoning with image generation. It contains 2,000 human‑curated instances across five reasoning‑intensive tasks—Code, Arithmetic, Spatial, Attribute, and Text—and compares direct generation, reasoning‑guided generation, and decontextualized generation. The study finds that while reasoning‑guided generation improves over direct generation, decontextualized generation consistently outperforms it, indicating that the visual semantics in textual reasoning are not reliably reflected in the generated images.

By Cheng Yang, Chufan Shi, Bo Shui, Yaokang Wu, Muzi Tao, Huijuan Wang, Ivan Yee Lee, Yong Liu, Xuezhe Ma, Taylor Berg-Kirkpatrick
arXiv Computation and Language
Sep 1

Learning Concepts, Not Tokens: Self-Supervised Semantic Alignment for Language Models

The paper proposes a self‑supervised framework that trains language models to predict concepts—sets of semantically equivalent tokens—rather than single tokens. This approach improves alignment with human similarity judgments, boosts performance on classification, clustering, and reranking tasks, and yields comparable or stronger downstream reasoning while lowering perplexity on semantically meaningful words and only slightly increasing overall perplexity.

By Christine Zhang, Dan Jurafsky, Chen Shani
arXiv Computation and Language
Sep 1

Graph2Counsel: Clinically Grounded Synthetic Counseling Dialogue Generation from Client Psychological Graphs

Graph2Counsel is a framework that generates synthetic counseling dialogues by leveraging Client Psychological Graphs (CPGs) to encode the relationships among a client’s thoughts, emotions, and behaviors. The system uses a structured prompting pipeline guided by counselor strategies and explores techniques such as Chain‑of‑Thought and Multi‑Agent Feedback to produce 760 realistic sessions from 76 CPGs. Expert evaluation shows the dataset surpasses previous ones in specificity, counselor competence, authenticity, conversational flow, and safety, and fine‑tuning an open‑source model on it improves performance on several counseling benchmarks.

By Aishik Mandal, Hiba Arnaout, Clarissa W. Ong, Juliet Bockhorst, Kate Sheehan, Rachael Moldow, Tanmoy Chakraborty, Iryna Gurevych
arXiv Computation and Language
Sep 1

TxSum: User-Centered Ethereum Transaction Understanding with Micro-Level Semantic Grounding

TxSum introduces a user-centered approach to understanding Ethereum transactions by providing structured, risk-aware explanations grounded at the token‑flow level. The authors built a dataset of 187 complex transactions with 2,375 token‑flow annotations and transaction‑level summaries, and developed MATEX, a multi‑agent framework that retrieves external knowledge and audits explanations for factual consistency. MATEX outperforms existing baselines, improving user comprehension from 52.9% to 76.5% and increasing malicious‑transaction rejection from 36.0% to 88.0% while keeping false‑rejection rates low.

By Zifan Peng, Jingyi Zheng, Yule Liu, Huaiyu Jia, Qiming Ye, Jingyu Liu, Xufeng Yang, Mingchen Li, Qingyuan Gong, Xuechao Wang, Xinlei He
arXiv Computation and Language
Sep 1

Overreliance on AI in Information-seeking from Video Content

The study examines how generative AI influences accuracy, efficiency, and confidence in retrieving information from videos. Across 900 participants and 8,000+ tasks, AI assistance improved accuracy by 3‑7% when the relevant video segment was viewed and by 27‑35% when it was not, while also boosting efficiency by 10‑25% depending on video length. However, participants over‑relied on AI, leading to accuracy drops of up to 32% with a deceiving assistant, and their confidence remained unchanged across conditions.

By Anders Giovanni M{\o}ller, Elisa Bassignana, Francesco Pierri, Luca Maria Aiello
arXiv Computer Vision
Sep 1

OptiGeo: Efficient Monocular Geometry for Embodied Perception in Optically Challenging Scenes

arXiv:2608.29881v1 Announce Type: new Abstract: Monocular depth estimation has achieved strong open-domain generalization, yet reliable robotic deployment remains difficult in transparent, reflective...

By Muxin Liu, Tianbo Liu, Jing Xia, Xiaoyang Lyu, Xiaoshan Wu, Bo Wang, Peng Dai, Zhongrui Wang, Shaoshuai Shi, Xiaojuan Qi
arXiv Computer Vision
Sep 1

RoboGesture: Real-Time Semantic-aligned Co-Speech Gestures Generation for Humanoid Interaction

arXiv:2608.28693v1 Announce Type: cross Abstract: Enabling humanoid robots to respond to human speech with synchronized and semantically meaningful gestures is fundamental to natural human-robot inte...

By Zifan Wang, Ziang Ren, Pengyang Shi, Zirui Wang, Chenghuai Lin, Tianze Wang, Zekun Qi, Liangliang Zhao, He Wang, Li Yi