AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

9,195 stories · RSS feed

arXiv AI

Jul 24

Diagnosing Pathological Chain-of-Thought in Reasoning Models

arXiv:2602. 13904v2 Announce Type: replace Abstract: Chain-of-thought (CoT) reasoning is fundamental to modern LLM architectures and represents a critical intervention point for AI safety.

By Manqing Liu, David Williams-King, Ida Caspary, Linh Le, Hannes Whittingham, Puria Radmard, Cameron Tice, Edward James Young

llms safety

arXiv AI

Jul 24

DINOde: Continuous Vision-Text Alignment for Open-Vocabulary Semantic Segmentation

arXiv:2607. 21371v1 Announce Type: cross Abstract: Open-vocabulary semantic segmentation (OVSS) leverages textual semantics to segment objects beyond predefined categories.

By Sung-Hoon Yoon, Hoyong Kwon, Changgyoon Oh, Kuk-Jin Yoon

rag computer-vision multimodal benchmarks safety

arXiv AI

Jul 24

TOUR: A Trajectory-Level Unlearning Benchmark for Offline Reinforcement Learning

arXiv:2607. 21111v1 Announce Type: cross Abstract: Offline Reinforcement Learning (RL) agents are trained on fixed behavioral trajectories, which makes trajectory-level deletion important when selected data must be removed after training.

By Chaofan Pan, Lingfei Ren, Xiangyu Jiang, Yanhua Li, Xuemei Cao, Xiangkun Wang, Hao Yu, Wei Wei, Xin Yang

agents reinforcement-learning robotics fine-tuning benchmarks safety

arXiv Machine Learning

Jul 24

Conflict Resolution under Degraded Surveillance in Air Corridors Using Multi-Agent Reinforcement Learning

arXiv:2607. 20547v1 Announce Type: new Abstract: Safe Advanced Air Mobility operations require aircraft to maintain separation when surveillance information is noisy, delayed, incomplete, or temporarily unavailable.

By Esrat Farhana Dulia, Syed Arbab Mohd Shihab, Caleb Adams, Ruben Del Rosario

agents reinforcement-learning safety

arXiv AI

Jul 24

Training Large Language Models for Self-Explanation Faithfulness

arXiv:2607. 21090v1 Announce Type: cross Abstract: We propose a Reinforcement Learning (RL) method to directly optimize the faithfulness of self-explanations - the extent to which a model's generated reasoning accurately reflects its internal decision-making process.

By Yeoktatt Cheah, Mar\'ia P\'erez-Ortiz, Noah Y. Siegel, Oana-Maria Camburu

llms reinforcement-learning fine-tuning safety

arXiv AI

Jul 24

Routing Subspaces: Auditing Evaluation-to-Deployment Mismatch in Fine-Tuned Language Models

arXiv:2607. 20436v1 Announce Type: cross Abstract: Safety evaluations often assume that behavior observed during testing reflects behavior in ordinary use, but fine-tuning can break this assumption.

By Phongsakon Mark Konrad, Toygar Tanyel, Serkan Ayvaz

llms fine-tuning safety

arXiv AI

Jul 24

Beyond Sycophancy: Structured Resistance and Compliance in LLM Moral Reasoning

arXiv:2607. 21558v1 Announce Type: new Abstract: Building socially calibrated large language models, which can learn from others without simply yielding to them, requires more than reducing sycophancy as a one-dimensional failure mode.

By Baihui Wang, Bernard Koch

llms safety

arXiv AI

Jul 24

Reexamining zero-shot summarization: Empirical investigation of trustworthiness of LLM-summarizers

arXiv:2607. 21010v1 Announce Type: new Abstract: Zero-shot summarization using Large Language Models (LLMs) has significantly advanced the abstractive summarization task by producing coherent and fluent summaries.

By Vasudha Bhatnagar, Purnima Bindal, Vikas Kumar, Raj Kumari Bahl

llms nlp benchmarks safety

arXiv Machine Learning

Jul 24

CASC: Causal Adversarial Subspace Clustering for Multivariate Spatiotemporal Data

arXiv:2607. 21088v1 Announce Type: new Abstract: Deep subspace clustering plays a critical role in applications involving multivariate spatiotemporal data, such as sea ice monitoring, disease spread analysis, and tracking neuro-degeneration over time.

By Francis Ndikum Nji, Vandana Janeja, Jianwu Wang

llms safety

arXiv AI

Jul 24

Understanding and Accelerating the Training of Masked Diffusion Language Models

arXiv:2605. 13026v2 Announce Type: replace-cross Abstract: Masked diffusion models (MDMs) have emerged as a promising alternative to autoregressive models (ARMs) for language modeling.

By Chunsan Hong, Sanghyun Lee, Chieh-Hsin Lai, Satoshi Hayakawa, Yuhta Takida, Yuki Mitsufuji, Seungryong Kim, Jong Chul Ye

llms diffusion benchmarks safety

arXiv AI

Jul 24

Autonomous Topology Mutation: Safe Runtime Restructuring for Multi-Agent LLM Systems with Capability, State, and Shadow Invariants

arXiv:2607. 20488v1 Announce Type: new Abstract: Multi-agent LLM frameworks typically fix their team topology at boot time.

By Bronislav Sidik, Chaya Levi, Nizzan Kimhi

llms agents efficiency benchmarks safety

arXiv AI

Jul 24

Marking the Wrong Symptoms: Evaluating LLM Watermarks in Medical Texts

arXiv:2607. 20462v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly integrated into clinical workflows, stressing the need for reliable traceability of model-generated output with watermarking.

By Melanie Rieff, Robin Staab, Thibaud Gloaguen, Stefan Hegselmann, Martin Vechev

llms multimodal benchmarks safety

arXiv AI

Jul 24

Representing Entity Importance in AI Knowledge Systems: A Dual-Signal Framework of Audience Evaluation and Structural Authority

arXiv:2607. 20925v1 Announce Type: new Abstract: AI knowledge systems require representations of entity importance for retrieval, recommendation, evidence selection, and knowledge-intensive reasoning.

By Shen Xu

rag safety

arXiv AI

Jul 24

KeySI: An Interaction Framework for Tuning Text Embeddings Based on Human Feedback

arXiv:2607. 20556v1 Announce Type: new Abstract: In large-scale text analysis tasks, pre-trained language models are often used to embed text corpora for downstream analysis.

By Yan Zhu, Y. Chen, Rebecca Faust

llms rag safety

arXiv AI

Jul 24

Attention Degradation, Function Token Anchoring, and the Limits of Attention-Based Intervention in Large Language Models

arXiv:2607. 20524v1 Announce Type: new Abstract: Mean cross-positional attention degradation is widely reported in transformer interpretability, yet whether it causally limits contextual retrieval remains untested.

By Sagar Dangal, Manoj Shakya

llms safety

arXiv Machine Learning

Jul 24

Environment-Aware Channel Inference via Cross-Modal Flow: From Multimodal Sensing to Wireless Channels

arXiv:2512. 04966v2 Announce Type: replace-cross Abstract: Accurate channel state information (CSI) underpins reliable and efficient wireless communication.

By Guangming Liang, Mingjie Yang, Dongzhu Liu, Paul Henderson, Lajos Hanzo

diffusion multimodal benchmarks safety

arXiv Machine Learning

Jul 24

Information-Theoretically Secure Aggregation for Lightweight Federated Learning: Resilient to Dropouts and Adversaries

arXiv:2607. 20890v1 Announce Type: new Abstract: On-device federated learning (FL) enables privacy-preserving and personalized model training on resource-constrained devices such as smartphones and IoT nodes.

By Hyeong-Gun Joo, Songnam Hong, Dong-Joon Shin

safety

arXiv AI

Jul 24

HiMe: Real-Time Self-Hosted Personal Agent Platform for Health Insights with Wearable Devices

arXiv:2607. 21019v1 Announce Type: new Abstract: Traditional approaches to wearable health signal analysis, such as smartwatches, are constrained by rigid analytical frameworks and limited personalisation.

By Wei Liu, Siya Qi, Linhai Zhang, Lorainne Tudor Car, Yulan He

llms agents safety

arXiv AI

Jul 24

Phonetic forced alignment for low-resource language varieties: Model training and evaluation on Chengdu Mandarin

arXiv:2607. 21332v1 Announce Type: cross Abstract: Phonetic forced alignment is a key technique in phonetic research, yet existing alignment systems lack specialized models for low-resource language varieties.

By Zhiheng Qian, Aini Li, Hai Hu, Liang Zhao

fine-tuning safety

arXiv AI

Jul 24

Scaling Interpretable Transformers with Parity Bottleneck Layers

arXiv:2607. 20652v1 Announce Type: cross Abstract: Language models are thought to exhibit the phenomenon of superposition, representing many more features than dimensions in their residual streams.

By Andrew Mack, Kraig Yuheng Tou, Mark Henry, Zhengxun Wu, Lauren Greenspan

llms efficiency safety

Get the daily briefing