AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,788 stories · RSS feed

arXiv AI
Oct 27

BoardSafety: AI-Ready Data from Manufacturer Documentation for Development-Board Safety

The paper introduces BoardSafety, a benchmark that transforms manufacturer datasheets and hardware guides into a board‑specific safety rule base for evaluating large language models (LLMs) on hardware safety judgments. Each data item pairs a development board and a proposed operation with a safety rule and a ground‑truth label (safe, unsafe, or insufficient‑info). Experiments show that providing safety rules improves model accuracy, while structured‑rationale prompting yields mixed results, indicating that current models cannot reliably assess operation safety.

By Linjian Yang, Xinyan Wang, Kunpeng Liu
arXiv AI
Oct 27

Jointly Modeling Preferences and Imperfect Decision-Making in Large Language Models

The paper introduces the Preference-Behavior Model (PBM), which jointly infers Large Language Model (LLM) preferences and imperfect decision-making mechanisms from observed choices. By applying PBM to three social dilemma games—Public Goods, Collective-Risk, and Common-Pool Resource—the authors show that preferences derived from one dilemma can predict behavior in others, whereas traditional behavioral economic elicitation instruments fail to do so consistently. The study also evaluates how well inferred preferences align with persona instructions, revealing both broad agreement and specific divergences across dimensions, LLMs, and games.

By Axel Abels, Elias Fern\'andez Domingos, Tom Lenaerts
arXiv AI
Oct 27

InferPriv: Evaluating Inferential Privacy Leakage in LLM Assistants

InferPriv is a new benchmark that evaluates inferential privacy leakage in large language model (LLM) assistants. It measures whether an honest-but-curious recipient can deduce a user’s sensitive attributes from an LLM’s reply during routine tasks, a type of leakage not captured by existing direct‑leakage benchmarks. Testing five frontier models, the study finds that inferential leakage rates (16.2–25.2%) far exceed direct leakage, even when direct leakage is near zero under privacy‑aware prompts.

By Pengrun Huang, Yiwei Fu, Luoxi Meng, Kamalika Chaudhuri
arXiv AI
Oct 27

Responsibility Boundaries for Agentic Systems: A Comparative Evaluation of Probabilistic Agentic Programming

The paper introduces Probabilistic Agentic Programming (PAP), a framework that explicitly defines responsibility boundaries for language‑model agents by linking mandate, authority, effects, and evidence. It compares PAP to various programming paradigms and agent systems across seven dimensions, and presents a mediation kernel that enforces checks before tool execution, operation‑bound approval, and resource limits. An exploratory study with seven business cases and multiple models demonstrates that PAP’s native Touba implementation achieves the highest or tied observation counts, supporting its effectiveness in managing agentic behavior.

By Khadim Lo, Soukaina Alaoui Ismaili, Jean-S\'{e}bastien Dessureault
arXiv AI
Oct 27

SEDATE: Do Large Language Models Share an Aggression Circuit Across Languages?

The paper introduces SEDATE, a training‑free method that detects, edits, and verifies responsible neurons in large language models to improve safety without extra training. It evaluates this approach across seven languages and four model families, showing superior safety scores compared to the best baseline while preserving general capabilities and adding minimal overhead. The method also demonstrates robustness on out‑of‑distribution languages and datasets.

By Sagnik Basu, Sayan Das, Somnath Banerjee, Animesh Mukherjee
arXiv AI
Oct 27

Joint Dynamic Lane Reversal and Variable Speed Limit Control: A Graphormer-Based Approach

The paper proposes a graph-based method for jointly controlling dynamic lane reversal and variable speed limits in traffic networks, ensuring the network remains strongly connected during lane changes. It introduces a transformer model with Global Endpoint Relation-Aware Spatio-Temporal Self-Attention (GERASTSA) that is pretrained on traffic forecasting and fine-tuned with proximal policy optimization. Experiments on a 6×6 grid show the learned policy can make two-thirds of roads one-way during peak hours, reducing daily delay by up to 90% compared to a fixed configuration.

By Hamed Araab, Abbas Ahmadi, Mohammad Hossein Basiri
arXiv AI
Oct 27

Joint Latent Reasoning and Observation for Agentic Video Understanding

The paper introduces LARVA, a framework that distills multi‑round video agents by jointly learning latent reasoning and visual observations. Instead of using textual captions, an observer sends compact continuous representations to a reasoner, which then selects the next action or produces an answer. The approach incorporates action and answer supervision through this interface and uses observation reconstruction and reasoning‑distribution alignment as auxiliary signals, achieving strong results on five video‑understanding benchmarks with 4B and 7B models.

By Junzhang Liu, Hao Yuan, Qi Zhang, Yiming Wu, Long Chen, Zhecan Wang
arXiv AI
Oct 27

OmniShowcase: Omnidirectional Subject-Consistent Video Generation from Multi-View Images

OmniShowcase introduces a method for generating omnidirectional videos of a specific subject from a handful of multi‑view images. It combines an automated data‑synthesis pipeline that creates training videos with diverse camera trajectories driven by the subject’s geometry, and a geometry‑conditioning technique that supplies the model with surface‑normal maps reconstructed from the reference views. The authors release Showcase‑4K, a large corpus of 3D‑asset‑based videos with 12 camera motions and 88 trajectory configurations, and ShowcaseBench, a held‑out evaluation set with source meshes and camera parameters.

By Junyeong Ahn, Geonwoo Kim, Kinam Kim, Gihwi Kim, Hoonjin Jung, Hyojin Jang, Jaegul Choo
arXiv AI
Oct 27

Foundation Models in Medical Image Analysis: A Systematic Review and Meta-Analysis

The article reviews foundation models (FMs) in medical image analysis, highlighting their shift from task‑specific AI to generalized representations learned from large datasets. It categorizes studies into vision‑only foundation models (VFMs) and vision‑language foundation models (VLFMs), mapping their architectures, training strategies, and clinical applications. The review includes quantitative analyses of dataset use, performance trends, and discusses challenges such as generalization, scalability, evaluation, fairness, and deployment, while proposing future research directions for robustness, interpretability, and clinical integration.

By Praveenbalaji Rajendran, Mojtaba Safari, Wenfeng He, Mingzhe Hu, Shansong Wang, Jun Zhou, Xiaofeng Yang
arXiv AI
Oct 27

What Can One Sentence Fix? A Failure Taxonomy and a Minimal Language Correction for Frozen VLA Policies

The paper investigates whether a single descriptive sentence can correct failures in frozen vision‑language‑action (VLA) policies. By labeling failures in the LIBERO‑90 benchmark, the authors identify four failure modes, finding that only execution misalignment—where the policy reaches the correct target but misjudges distance or force—can be remedied with a concise human‑generated sentence. This sentence, parsed by an LLM into a bias on the policy’s actions, successfully restores performance on both simulated and real‑world tasks, while automated language models fail to achieve similar gains.

By Owen Kwon, Pablo Ortega-Kral, Arthur Bucker, Jean Oh
arXiv AI
Oct 27

The Future Is Finer: Multivariate Time Series Super-Resolution Forecasting

The paper introduces Multivariate Time Series Super-Resolution Forecasting (MTS‑SRF), a method that predicts fine‑grained future values from coarse historical data. It proposes the TimeMagnifier framework, which includes Fourier Anchor to handle data sparsity, De‑noise Only to refine predictions along the resolution axis, and banded DTW Alignment to correct cross‑resolution misalignment. On a new benchmark of ten datasets across five domains, TimeMagnifier outperforms 15 baselines, reducing mean‑squared error by an average of 12.3% and improving existing forecasters by 5.1% to 9.0% when its components are used as plug‑ins.

By Shuhao Li, Weidong Yang, Yue Cui, Fan Zhang, Lu Qin
arXiv AI
Oct 27

Learning When to Collaborate: Selective Multi-Agent Medical Reasoning via Uncertainty-Aware Routing

The paper introduces SMART‑Med, a local multi‑agent system for medical reasoning that learns when to collaborate instead of always engaging all agents. By using an uncertainty‑driven routing mechanism, it selectively recruits expert agents only for uncertain queries, thereby converting dense collaboration into query‑adaptive coordination. The authors also present a new multimodal dataset, MedVR‑3K, and demonstrate that SMART‑Med matches or surpasses existing multi‑agent approaches while cutting inference time by over 3.8× and token usage by 72%, with a 6.7% average accuracy gain across four VQA benchmarks.

By Jiacheng Hou, Xueliang Cui, Dan Lu, Ruxin Wang
arXiv AI
Oct 27

From Outranking Flow Scores to Linear Programs: Group Decision Making in PROMETHEE II

The paper presents a linear reparametrization of PROMETHEE II net flow scores that incorporates both criteria weights and preference functions, enabling a clearer understanding of their interaction. Using this representation, the authors formulate linear optimization models for three inverse‑analysis scenarios: (1) minimizing weight deviations while allowing preference functions to vary, (2) minimizing deviations over all preference parameters, and (3) achieving consensus among multiple decision makers with objectives based on total or maximal individual deviation to promote fairness. An illustrative example demonstrates that permitting preference functions to vary can substantially alter inverse‑analysis conclusions compared to weight‑only models, and that the group formulations can distinguish efficient consensuses from fairer ones.

By Alexandre Flachs, Yves De Smet
arXiv AI
Oct 27

Scope-Gap Injection: Exploiting Evaluator Channel Blind Spots in the OpenClaw Agent Security Competition

Scope-Gap Injection (SGI) is an indirect prompt injection technique that exploits a mismatch between what an LLM judge inspects and what an agent can access during a security competition. In the IJCAI 2026 OpenClaw Agent Security Attack Competition, SGI hides malicious instructions in an attachment that the judge’s safety check (S2) ignores, while the agent reads the attachment during live execution (S3). The method successfully bypassed all three evaluation stages for 5 of 19 targets, achieving a top score of $F = 0.385$, and demonstrated that placing malicious content in attachments yields higher acceptance rates than embedding it directly in the skill definition.

By Weihang Niu, Mengxing Ren, Che Qu
arXiv AI
Oct 27

SAP-ViT: A Self-Attributing Patch Vision Transformer for Medical Image Classification

SAP-ViT is a Vision Transformer that replaces the traditional global classification token with direct aggregation of last‑layer patch embeddings, enabling class predictions to be decomposed into patch‑level contributions during the forward pass. The model incorporates sparsity regularization to produce concise, readable evidence and introduces an input‑space back‑mapping mechanism to identify which input patches influence the output. Experiments on diverse medical imaging datasets show that SAP‑ViT retains strong classification performance while delivering consistent, interpretable patch‑level explanations, with ablation studies confirming the benefits of both sparsity and back‑mapping.

By Wei Zhang, Wolfgang Nejdl, Ming Tang
arXiv AI
Oct 27

Repairing Broken Access Control in Web Applications with LLMs: An Empirical Benchmark Study

The paper introduces BAC-Bench, a benchmark comprising 46 real-world broken access control vulnerabilities from six popular web applications across four programming languages. It evaluates eight base large language models (LLMs) and two commercial LLM agents, showing that single-turn LLMs repair about 38% of the vulnerabilities, while minimal scaffolds can raise this to 66% and the best LLM achieves 75%. Some LLM-generated patches even outperform official fixes and one patch closed a dormant zero‑day vulnerability.

By Mingrui Liu, Liang He, Xinchu Wang, Yeting Li, Dongxu Han, Yunpeng Li, Yuling Liu
arXiv AI
Oct 27

FOCUS: Benchmarking Retinal Model Generalization from Foundation Vision Encoders to Multimodal LLMs

The paper introduces FOCUS, a cross‑dataset benchmark for retinal fundus models that evaluates vision‑only encoders, vision‑language dual encoders, and multimodal large language models across ten public datasets. It measures ranking performance, calibration, subgroup disparities, and image‑quality robustness for tasks such as diabetic retinopathy and glaucomatous optic neuropathy. Results show no single model family dominates, with vision‑only encoders ranking highest on average, while fine‑tuning improves in‑domain performance but varies in external transfer.

By David Restrepo, Chenwei Wu, Luis Filipe Nakayama, Miguel L. Martins, Stergios Christodoulidis, Maria Vakalopoulou, Enzo Ferrante