AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,896 stories · RSS feed

arXiv Computer Vision

Aug 21

Reading Between the Pixels: An Inscriptive Jailbreak Attack on Text-to-Image Models

arXiv:2604. 05853v3 Announce Type: replace Abstract: Modern text-to-image (T2I) models can now render legible, paragraph-length text, enabling a fundamentally new class of misuse.

By Zonghao Ying, Haowen Dai, Lianyu Hu, Zonglei Jing, Quanchen Zou, Yaodong Yang, Aishan Liu, Xianglong Liu

llms diffusion robotics multimodal benchmarks safety

arXiv Machine Learning

Aug 21

Inadvertent Context Leakage in Language Models

arXiv:2608. 19857v1 Announce Type: new Abstract: For AI agents to be useful beyond simple chat, they must hold sensitive user context such as calendars, credentials, health records, and financial data.

By Jaiden Fairoze, Neal Mangaokar, Kamalika Chaudhuri, Sanjam Garg, Saeed Mahloujifar

llms agents reinforcement-learning safety

arXiv Computation and Language

Aug 21

Multi-Agent Orchestration with the Common-Sense Reasoning Capabilities of LLMs for Autonomous Driving

arXiv:2608. 20129v1 Announce Type: cross Abstract: Autonomous vehicles require robust perception and decision-making capabilities to operate in diverse and unseen scenarios.

By Mehdi Azarafza, Faezeh Pasandideh, Ali Ehteshami Bejnordi, Stefan Henkler, Achim Rettberg

llms agents reinforcement-learning robotics multimodal safety

arXiv Computer Vision

Aug 21

DIFFCZSL: Compositional Zero-Shot Learning Regularized by Diffusion Representations

arXiv:2608. 19871v1 Announce Type: new Abstract: Compositional Zero-Shot Learning (CZSL) aims to recognize unseen attribute-object compositions by leveraging knowledge of primitive concepts learned from seen compositions.

By Hangyu Tian, Zhenqi He, Yanghao Wang, Long Chen

llms rag diffusion multimodal benchmarks safety

arXiv AI

Aug 21

Adaptive Probabilistic Shielding by Learning MDPs for Safe Reinforcement Learning

arXiv:2608. 19836v1 Announce Type: cross Abstract: Probabilistic shielding is a technique for safe reinforcement learning (RL).

By Astrid Horn Brorholt (Aalborg University, Aalborg, Denmark), Maris F. L. Galesloot (Radboud University, Nijmegen, Netherlands), Nils Jansen (Radboud University, Nijmegen, Netherlands), Kim Guldstrand Larsen (Aalborg University, Aalborg, Denmark), Christian Schilling (Aalborg University, Aalborg, Denmark)

agents reinforcement-learning safety

arXiv AI

Aug 21

A knowledge-guided agentic framework for mitigating patient-context ambiguity in health queries

arXiv:2608. 19875v1 Announce Type: cross Abstract: Patients often submit short, underspecified queries to healthcare chatbots that lack the patient-specific information needed to determine an appropriate response.

By Mahyar Abbasian, Saba A. Farahani, Arshia Ilaty, Hung Cao, Ramesh Jain, Amir M. Rahmani

llms agents benchmarks safety

arXiv Computer Vision

Aug 21

MOSAIC: Modality-agnostic Spectral Alignment for Federated Image-level Weakly Supervised Tumor Segmentation under Client-specific Missing Modalities

arXiv:2608. 19788v1 Announce Type: cross Abstract: Trustworthy multimodal fusion in clinical settings requires handling incomplete and heterogeneous modality subsets across institutions, where privacy constraints prohibit centralized data sharing.

By Tarun Kumar Garg, Vaanathi Sundaresan

diffusion computer-vision multimodal benchmarks safety

arXiv AI

Aug 21

Does Marginal Coverage Guarantee Class-Conditional Safety for Zero-Shot VLMs Under Shift?

arXiv:2608. 19376v1 Announce Type: cross Abstract: Split-conformal prediction provides marginal coverage under exchangeability and is increasingly used as an abstention layer for zero-shot vision-language models (VLMs).

By Jai Kumar Sharma, Amartya Dutta

llms multimodal safety

arXiv AI

Aug 21

An Inclusive and Lightweight Approach to Federated Continual Learning for Cultural Heritage

arXiv:2608. 20038v1 Announce Type: cross Abstract: Artificial intelligence can support cultural heritage and digital humanities through large-scale retrieval and analysis of digitized collections.

By Ioannis Theologitis, Debin Meng, Stylianos Eleftheriadis, Vasileios Lolis, Konstantinos Votis

safety

arXiv AI

Aug 21

SoftVTBench: A Safety-Aware Visuo-Tactile Benchmark for Physically Constrained Robotic Manipulation of Deformable Objects (Early Version)

arXiv:2607. 04234v2 Announce Type: replace-cross Abstract: Deformable object manipulation poses challenges beyond task completion: successful execution must also maintain safe physical interaction, holding the object stably without slip or drop while avoiding excessive deformation.

By Bowen Jing, Mingxin Wang, Ruiyang Hao, Chenchen Ge, Hanwen Shen, Junjie He, Yang Cui, Yiming Hou, Weitao Zhou, Jiawei Wang, Minglei Li, Dandan Zhang, Ding Zhao, Houde Liu, Xiaofan Li, Si Liu, Ping Luo, Haibao Yu

robotics benchmarks safety

arXiv AI

Aug 21

Enforcing LLM Safety through DMD-based Classification of Prompt-Response Embedding Dynamics

arXiv:2608. 19579v1 Announce Type: new Abstract: Large Language Models (LLMs) are increasingly deployed in high-stakes applications, yet their tendency to generate toxic, harmful, or policy-violating content poses significant risks.

By Mohamed Akrout, Olivera Kotevska, Dan Wilson

llms rag benchmarks safety

arXiv AI

Aug 21

Distilling Aggregate Mobility Statistics into a Language Model Policy for Post-Event Crowd Simulation

arXiv:2608. 19778v1 Announce Type: cross Abstract: Pedestrian simulators need a behaviour rule for every agent, but privacy usually limits the data for setting one to aggregate statistics, namely zone-level device counts and origin-to-destination (OD) flows, with no individual trajectories.

By Tatsuya Amano, Hirozumi Yamaguchi

llms agents fine-tuning safety

arXiv Computer Vision

Aug 21

FermatSyn: SAM2-Enhanced Bidirectional Mamba with Isotropic Spiral Scanning for Multi-Modal Medical Image Synthesis

arXiv:2505. 07687v4 Announce Type: replace-cross Abstract: Multi-modal medical image synthesis is pivotal for alleviating clinical data scarcity, yet existing methods fail to reconcile global anatomical consistency with high-fidelity local detail.

By Feng Yuan, Yifan Gao, Haoyue Li, Xin Gao

llms computer-vision fine-tuning multimodal benchmarks safety

arXiv AI

Aug 21

TestifAI: Tomography-Based Testing for Deep Learning Systems

arXiv:2608. 18900v2 Announce Type: replace Abstract: As AI systems are increasingly deployed in safety-critical application domains (e.

By Arooj Arif, Tobias Hartung, Elena Botoeva, Alexandros Koliousis

agents robotics safety

arXiv Machine Learning

Aug 21

Auditing Cross-Lingual Fairness in Language Model Watermarking

arXiv:2608. 20047v1 Announce Type: cross Abstract: Watermarking schemes for large language model output are evaluated almost exclusively on English text using each scheme's detection threshold and a narrow set of quality measurements.

By Alexander Nemecek, Osama Zafar, Debargha Ganguly, Vikash Singh, Vipin Chaudhary, Erman Ayday

llms safety

arXiv Machine Learning

Aug 21

Gravitational-wave parameter estimation with machine-learning generated surrogate waveforms

arXiv:2608. 20222v1 Announce Type: cross Abstract: The worldwide network of gravitational-wave detectors have detected more than 350 binary coalescence events till date.

By Suyog Garg, Kipp Cannon

safety

arXiv AI

Aug 21

Electronic Navigational Chart Change Classification

arXiv:2608. 20218v1 Announce Type: new Abstract: Electronic Navigational Charts (ENCs) are geospatial vector datasets used in maritime navigation systems that represent hydrographic and navigational information such as depths, navigational aids, traffic schemes, and hazards.

By Jacob Arndt, Abhishek Potnis, Alexandre Sorokine

rag safety

arXiv AI

Aug 21

Beyond Recall: Behavioral Specification as an Interpretive Layer for AI Personalization

arXiv:2605. 28969v2 Announce Type: replace-cross Abstract: If an AI agent makes decisions on a person's behalf, those decisions must align with its user.

By Aarik Gulaya

llms agents benchmarks safety

arXiv Machine Learning

Aug 21

Holtercare-Bench: A Multimodal Benchmark for Evaluating Long-Term Dynamic ECG Analysis

arXiv:2608. 19297v1 Announce Type: new Abstract: While multimodal large language models (MLLMs) excel in medical applications, most of them favor static images or short-term signals.

By Yihan Xie, Hanwen Cui, Runze Ye, Juekai Lin, Haoyang Wang, Jinhao Mao, Bo Zhang, Wenqiao Zhang, Xiaogang Guo, Jun Xiao, Lei Zhang

llms nlp fine-tuning multimodal benchmarks safety

arXiv Machine Learning

Aug 21

Merge Now, Regret Later: The Hidden Cost of Model Merging Is Adversarial Transferability

arXiv:2509. 23689v2 Announce Type: replace Abstract: Model Merging (MM) has proven to be an effective alternative to multi-task learning, where several fine-tuned models are merged, without access to the tasks' training data, into one model that retains performance across different tasks.

By Mauro Conti, Ankit Gangwal, Aaryan Ajay Sharma

fine-tuning safety

Get the daily briefing