AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

9,333 stories · RSS feed

arXiv AI

Jul 24

Knowledge-Guided Time-Varying Causal Inference for Arctic Sea Ice Dynamics

arXiv:2601. 17647v3 Announce Type: replace-cross Abstract: Quantifying the causal relationship between sea ice thickness and sea surface height (SSH) is essential for understanding the mechanisms driving polar climate dynamics.

By Akila Sampath, Vandana Janeja, Jianwu Wang

diffusion benchmarks safety

arXiv AI

Jul 24

PILD: Physics-Informed Learning via Diffusion

arXiv:2601. 21284v2 Announce Type: replace-cross Abstract: Diffusion models have emerged as powerful generative tools for modeling complex data distributions, yet their purely data-driven nature limits applicability in engineering and scientific problems where physical laws must be respected.

By Tianyi Zeng, Tianyi Wang, Jiaru Zhang, Zimo Zeng, Feiyang Zhang, Yiming Xu, Sikai Chen, Junfeng Jiao, Christian Claudel, Xinbo Chen

diffusion safety

arXiv AI

Jul 24

Playing Along: Learning a Double-Agent Defender for Belief Steering via Theory of Mind

arXiv:2604. 11666v2 Announce Type: replace-cross Abstract: As large language models (LLMs) become the engine behind conversational systems, their ability to reason about the intentions and states of their dialogue partners (i.

By Hanqi Xiao, Vaidehi Patil, Zaid Khan, Hyunji Lee, Elias Stengel-Eskin, Mohit Bansal

llms agents reinforcement-learning safety

arXiv AI

Jul 24

AI Security Policy Should Assess Systems, Not Only Models

arXiv:2605. 09504v2 Announce Type: replace-cross Abstract: We present swarm-attack, an open-source adversarial testing framework in which multiple lightweight LLM agents coordinate through shared memory, parallel exploration, and evolutionary optimization.

By Michael A. Riegler, Inga Str\"umke

llms agents safety

arXiv AI

Jul 24

WildTrace: Benchmarking Natural Evidence Trails in Long-Context Reasoning

arXiv:2607. 09328v2 Announce Type: replace-cross Abstract: Answering complex questions over long documents frequently requires integrating evidence that the source itself disperses naturally across distant passages.

By Zixin Chen, Peng Liu, Haobo Li, Rui Sheng, Jianhong Tu, Xiaodong Deng, Fei Huang, Kashun Shum, Dayiheng Liu, Huamin Qu

benchmarks safety

arXiv Machine Learning

Jul 24

Multimodal CoLRAG-TF: Triple-Filtered Retrieval for Complex PDFs

arXiv:2607. 20517v1 Announce Type: new Abstract: Retrieval-augmented generation (RAG) over heterogeneous PDF collections remains challenging due to multimodal content, domain-specific terminology, and the need for multi-hop reasoning across dispersed evidence.

By Takato Yasuno

llms rag computer-vision multimodal benchmarks safety

arXiv Machine Learning

Jul 24

Cardinality-Decomposed Loss: Matching Training Objectives to Relation Structure in Heterogeneous Recommendation Graphs

arXiv:2607. 20737v1 Announce Type: new Abstract: Graph Neural Networks trained on heterogenous bipartite graphs form a common basis in recommendation systems.

By Parul Maheshwari, Amulya Paruchuri, Yiqing Zou, Alireza Sahami Shirazi, Farhad Farahani, Prakhar Mehrotra

rag computer-vision safety

arXiv Machine Learning

Jul 24

Toward Mechanistic Interpretability of an AI Foundation Model Fine-Tuned for Atmospheric Chemistry

arXiv:2607. 20778v1 Announce Type: new Abstract: Weather forecasting foundation models (FMs) are increasingly fine-tuned to predict air quality, offering fast global pollution forecasts at lower computational cost than conventional chemical transport models.

By Jason Y. Hu, Ivan Higuera-Mendieta, Patrick Obin Sturm, Makoto M. Kelp

fine-tuning benchmarks safety

arXiv Machine Learning

Jul 24

Three-Pronged Spectral Control for Federated Parameter Efficient Fine Tuning

arXiv:2607. 20914v1 Announce Type: new Abstract: Federated parameter-efficient fine-tuning (PEFT) enables communication-efficient adaptation of large pretrained models on decentralized edge data, but it remains fragile under non-IID client heterogeneity.

By Shiva Raj Pokhrel, Dipsan Bhattarai, Anwar Walid

fine-tuning benchmarks safety

arXiv Machine Learning

Jul 24

From Evaluation to Optimisation: Hierarchy-Aware Training Signals for CWE Prediction in Python

arXiv:2607. 21069v1 Announce Type: new Abstract: The original ALPHA benchmark introduced a taxonomy-aware penalty for evaluating CWE-level vulnerability prediction in Python and proposed that the penalty could theoretically also serve as a training signal.

By Muntasir Adnan, Manile Srun, Carlos C. N. Kuhn

reinforcement-learning fine-tuning benchmarks safety

arXiv Machine Learning

Jul 24

Automated Synthesis and Adversarial Validation of Executable Causal Research Pipelines

arXiv:2607. 21173v1 Announce Type: new Abstract: While automated research systems promise to accelerate empirical analysis, they are prone to silent failures: instances in which analysis code executes successfully yet relies on invalid causal assumptions.

By Irena Girshovitz, Dan Zeltzer, Ran Gilad-Bachrach

llms benchmarks safety

arXiv Machine Learning

Jul 24

HERMES: Heterogeneous Edge-Relational Multi-Head Embedded SSM Attention for Traffic Conflict Prediction at Signalized Intersections

arXiv:2607. 20505v1 Announce Type: cross Abstract: Surrogate safety measures (SSMs) enable proactive traffic safety assessment, but many existing methods evaluate pairwise interactions independently or flatten multi-agent scenes into fixed feature vectors, limiting their ability to represent heterogeneous interaction structure and evolving scene-level risk.

By Md Monzurul Islam, Subasish Das

llms agents safety

arXiv Machine Learning

Jul 24

SPECTRA: State-Space Exogenous Context and Temporal-Frequency Resolution Architecture for Probabilistic Energy Forecasting

arXiv:2607. 20587v1 Announce Type: cross Abstract: Modern power systems increasingly require probabilistic forecasts amid interacting uncertainties from renewable intermittency, flexible demand, market volatility, and weather-dependent generation.

By Hang Ye, Xinyan Jiang, Yuedong Shi, Yangxin Zhu, Jianming Wei, Tian Zheng, Xiaoying Zheng, Yongxin Zhu

safety

arXiv Machine Learning

Jul 24

Safety-oriented sidewalk and road segmentation for smartphone-based assistive navigation

arXiv:2607. 21137v1 Announce Type: cross Abstract: Independent sidewalk mobility is essential for blind and visually impaired pedestrians (BVIPs), yet smartphone-based assistive navigation requires perception models that distinguish walkable sidewalks from adjacent unsafe regions.

By Hakan Calim, Anamaria Dumitrescu, Adarsh Bhandary Panambur, Huzaifa Asif, Andreas Maier

computer-vision fine-tuning benchmarks safety

arXiv Machine Learning

Jul 24

Climate-resilient electric vehicle charging infrastructure for sustainable cities: An interpretable causal-ensemble framework for preventive maintenance and low-carbon mobility

arXiv:2607. 21444v1 Announce Type: cross Abstract: Reliable electric vehicle (EV) charging infrastructure is a cornerstone of sustainable, low-carbon cities, yet urban climate stress such as extreme heat, heavy precipitation, and humidity increasingly raises equipment fault risk and undermines the resilience of urban energy and mobility services.

By Cande Lian (School of Management, Foshan University, Foshan, China), Wentao Zeng (School of Management, Foshan University, Foshan, China), Jiabin Wu (School of Management, Foshan University, Foshan, China), Yiming Bie (School of Transportation, Jilin University, Changchun, China), Wei Zhou (Department of Civil and Environmental Engineering, National University of Singapore)

safety

arXiv Machine Learning

Jul 24

Concept Concentration for Faithful Representation Intervention

arXiv:2505. 18672v2 Announce Type: replace Abstract: Representation intervention aims to localize and modify the representations that encode the underlying concepts in large language models (LLMs) to elicit the aligned and expected behaviors.

By Hongzheng Yang, Yongqiang Chen, Zeyu Qin, Tongliang Liu, Chaowei Xiao, Kun Zhang, Bo Han

llms safety

arXiv Machine Learning

Jul 24

Tackling Heterogeneity in Federated Learning via Variance-Reduced Boltzmann Sampling within Homogeneous Social Coalitions

arXiv:2506. 02897v3 Announce Type: replace Abstract: Federated Learning (FL) enables privacy-preserving collaborative model training, but its effectiveness is often limited by client data heterogeneity.

By Alessandro Licciardi, Roberta Raineri, Anton Proskurnikov, Lamberto Rondoni, Lorenzo Zino

benchmarks safety

arXiv Machine Learning

Jul 24

Counterfactual Explainability Framework With CycleGAN And Counterfactual-Classifier Alignnment Score for Retinal Disease Classification

arXiv:2607. 21068v1 Announce Type: new Abstract: Automated detection of vision impairing retina-based ocular conditions from fundus images is important for early screening, timely referral and reducing dependency on specialist-only assessment, for which neural network-based deep learning (DL) models have been widely utilized.

By Kritanu Chattopadhyay, Sayanjit Singha Roy, Soumya Chatterjee

rag benchmarks safety

arXiv Machine Learning

Jul 24

Position Bias is Hidden Behind Ceiling Effects: A Permutation Diagnostic for LLM Benchmarks

arXiv:2607. 20864v1 Announce Type: new Abstract: Position bias in multiple-choice LLM evaluation is widely cited as a confound in capability comparisons, but published measurements rely on single answer-order shuffles whose results confound the bias signal with content-level noise and sampling stochasticity.

By Hiroki Tamba

llms benchmarks safety

arXiv AI

Jul 24

Robust Critics: Defending LLMs Against Multi-Turn Attacks

arXiv:2607. 20472v1 Announce Type: new Abstract: When a user asks a language model something harmful, is it a genuine attack or a misunderstood but well-meaning question?

By Roman Belaire, Arunesh Sinha, Pradeep Varakantham

llms reinforcement-learning fine-tuning safety

Get the daily briefing