AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,798 stories · RSS feed

arXiv AI

Oct 5

Bounded Reachability & Jailbreak Detection via Contraction-Constrained State Space Models

arXiv:2610.02853v1 Announce Type: new Abstract: Safety heads are lightweight classifiers attached to pretrained language models for flagging harmful inputs before generation. Their empirical detectio...

By Omanshu Thapliyal

llms rag safety

arXiv AI

Oct 5

HASTE: Evolving Agent Harnesses Against Emerging Attacks Using Sparse Evidence

arXiv:2610.02920v1 Announce Type: new Abstract: Agent harnesses play a critical role in defenses by enforcing safety constraints to prevent unsafe actions. However, rapidly emerging attacks outpace m...

By Xiqiao Xiong, Moxin Li, Zhixin Ma, Ouxiang Li, Wenjie Wang, Fuli Feng, Xiangnan He

agents safety

arXiv AI

Oct 5

DuetMoE: Coupling Inter- and Intra-Subgroup Robustness for Fair Medical Image Analysis

arXiv:2605.10521v2 Announce Type: replace-cross Abstract: As medical AI expands across diverse healthcare settings worldwide, equitable performance across patient populations is becoming essential to...

By Yiqi Tian, Jinwoong Park, Sangjoon Park, Bo Zeng, Pengfei Jin, Yujin Oh, Quanzheng Li

safety

arXiv Computation and Language

Oct 5

Output Language Confusion under Multilingual Prompt Contamination

arXiv:2610.02926v1 Announce Type: new Abstract: Standard factual benchmarks assume clean monolingual prompts and exact-match scoring, two assumptions that break simultaneously in real-world multiling...

By Riju Marwah, Ritvik Garimella, Khusham Bansal, Atishay Jain, Amit Sheth

llms rag benchmarks safety

arXiv AI

Oct 5

Coding Agents with Harness for Safe Robot Control

arXiv:2609.20822v2 Announce Type: replace-cross Abstract: Coding agents have emerged as a promising paradigm for robot manipulation: a language model writes the robot controller as a program, and age...

By Bingxin Xu, Yuzhang Shang, Zhen Dong, Emilio Ferrara

llms agents robotics benchmarks safety

arXiv Computation and Language

Oct 5

EpiWorld: Grounding LLM Policy Agents in Epidemiological World Models

arXiv:2610.02744v1 Announce Type: new Abstract: Epidemic intervention policies are textual artefacts that human decision-makers interpret, justify, and revise through natural language, making large l...

By Zeeshan Memon, Yiqi Su, Kai Shu, Naren Ramakrishnan, Liang Zhao

llms agents safety

arXiv AI

Oct 5

Relevant Evidence Decoding for Audio-Visual Hallucination Mitigation

arXiv:2610.02976v1 Announce Type: new Abstract: Audio-Visual Large Language Models (AV-LLMs) remain prone to cross-modal hallucinations, where one modality incorrectly affects predictions about anoth...

By Hyunjae Ra, Aecheon Jung, Jungin Park, Sungeun Hong

llms multimodal benchmarks safety

arXiv Machine Learning

Oct 5

ArrivalBench: Agent-Generated Data Pipelines Are Correct Once and Wrong Under Time

arXiv:2610.02363v1 Announce Type: new Abstract: Benchmarks for agent-generated data work grade a pipeline by running it once against a fixed snapshot. ArrivalBench instead re-executes the pipeline an...

By Pranay Kothari

agents benchmarks safety

arXiv AI

Oct 5

Right Order, Wrong Scale: Auditing LLM Judges for Occupational AI Measurement

arXiv:2610.02492v1 Announce Type: new Abstract: LLM judges are increasingly used to assess whether AI outputs meet workplace requirements, but agreement on response rankings does not establish agreem...

By Harry Lyu, Neil Thompson

llms fine-tuning safety

arXiv AI

Oct 5

Frequency Is Not Sensitivity Identifying Safety-Sensitive Experts in Sparse MoE LLM

arXiv:2610.02910v1 Announce Type: new Abstract: Suppressing a small set of routed experts can weaken the safety behavior of a sparse Mixture-of-Experts (MoE) language model without retraining. Which...

By Md Nurul Absar Siddiky, Liuwan Zhu, Yingfei Dong

llms safety

arXiv AI

Oct 5

Labels Override Definitions in Jev-Style Typed Decision Models

arXiv:2610.02586v1 Announce Type: new Abstract: A typed decision model answers a fixed question about an input by returning a probability for each of several caller-defined options. Each option carri...

By Seyedarmin Azizi, Erfan Baghaei Potraghloo, Massoud Pedram

llms safety

arXiv AI

Oct 5

Revisiting Visual Representation Enhancement of VLMs via Kernel Canonical Correlation Analysis

arXiv:2610.02718v1 Announce Type: cross Abstract: Vision-language models such as CLIP exhibit strong semantic generalization, but remain limited in fine-grained visual perception. A recent work named...

By Peilin Yang, Xiaoyu Liu, Jian Sun, Qinghua Tao

llms rag fine-tuning multimodal safety

arXiv Machine Learning

Oct 5

State-Space Unlearning for Non-Stationary Bias in Land Surface Forecasting

arXiv:2610.02248v1 Announce Type: new Abstract: Operational land surface forecasting systems built on Mamba-family Structured State Space Models absorb non-stationary confounding events (unrecorded i...

By Anidipta Pal

benchmarks safety

arXiv Machine Learning

Oct 5

Hybrid Machine Learning-Assisted Raman Spectroscopy with Generative Feature Augmentation for Pharmaceutical Identification

arXiv:2610.02224v1 Announce Type: new Abstract: Rapid and reliable identification of pharmaceutical residues is important for safeguarding public health, ensuring food safety, and enabling practical...

By Quach Thi Thai Binh, Ton Nu Quynh Trang, Thang B. Phan, Vu Thi Hanh Thu, Nguyen Tuan Hung

rag diffusion benchmarks safety

arXiv AI

Oct 5

Safe and Robust Neural Policy Learning with Statistical Verification for Sim-to-Real Deployment in Robotics

arXiv:2608.06481v2 Announce Type: replace-cross Abstract: Synthesizing safe and robust neural controllers in simulation for reliable sim-to-real deployment remains a critical challenge in robotics. E...

By Riccardo Curcio, Hongpeng Cao, Marco Caccamo

robotics benchmarks safety

arXiv AI

Oct 5

Evaluation format, not model capability, drives measured triage failure in the assessment of consumer health AI

arXiv:2603.11413v4 Announce Type: replace-cross Abstract: A recent Nature Medicine study reported that ChatGPT Health under-triages 51.6% of emergencies and concluded that consumer-facing AI triage p...

By David Fraile Navarro, Jialei Sheng, Farah Magrabi, Enrico Coiera

llms benchmarks safety

arXiv Machine Learning

Oct 5

From Patching to Pruning Visual Computation in Vision Language Models

arXiv:2610.03389v1 Announce Type: cross Abstract: Vision language models (VLMs) incur substantial inference cost because every visual token is processed by the attention and MLP projections of every...

By Rahul Chowdhury, Timothy A Rupprecht, Xuan Shen, Shaoyi Huang, Pu Zhao, Yanzhi Wang

llms efficiency multimodal benchmarks safety

arXiv Machine Learning

Oct 5

RMCW: A Deletion-Robust Watermark Based on Reed--Muller Codes for Language Models

arXiv:2610.02817v1 Announce Type: cross Abstract: Large Language Model (LLM) watermarking provides a lightweight mechanism for identifying text generated by a specific model, but its robustness remai...

By Yi Wang, Baicheng Chen, Yu Wang, Jian Zhao, Yilei Chen, Tianxing He

llms safety

arXiv Machine Learning

Oct 5

Localized Conformal Safety Monitoring with Vision-Language Models for Autonomous Driving

arXiv:2610.02765v1 Announce Type: cross Abstract: Monitoring planned driving trajectories requires accurately estimating the collision likelihood with actors whose motion is itself impacted by the eg...

By Lu\'is Marques, Rong Fang, Disha Kamale, Dmitry Berenson

llms agents robotics multimodal safety

arXiv Machine Learning

Oct 5

Metropolis-Hastings Dominates Importance Resampling for Policy Composition

arXiv:2610.03480v1 Announce Type: new Abstract: Post-training a large language model (LLM) often requires exploring trade-offs between multiple rewards, but retraining for each trade-off is expensive...

By Alexey Kurennoy, Ramil Yarullin, Fergal Reid

llms safety

Get the daily briefing