AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,788 stories · RSS feed

arXiv Computation and Language
Oct 23

Meaning Contracts: Local Semantic Geometry in Large Language Models

The paper investigates how semantic meaning is arranged locally within large language models. By using two controlled probes—one cross‑lingual and one refinement probe—on four decoder‑only LLMs, it finds that after removing dominant variance directions, sentence pairs rank by semantic relatedness, semantic factors cluster around an invariant core with structured variation, refinement tightens neighborhoods, and sentence‑level versus token‑local readouts reveal complementary geometric patterns. These results suggest that LLM semantics form local anisotropic neighborhoods that contract when meaning is refined.

By Andr\'es Daniel Godoy Ortiz
arXiv Computation and Language
Oct 23

Once a Tree, Always a Tree? Cross-lingual Transfer of Hierarchical Generalization in Language Models

The paper investigates whether language models can transfer hierarchical syntactic knowledge from a source language (L1) to a target language (L2) using a novel cross‑lingual poverty‑of‑stimulus (CPOS) experimental design. Results show that models can partially transfer L1 structural evidence to L2, producing a small but detectable hierarchical bias that varies across language pairs. The study further demonstrates that vocabulary overlap is crucial for successful transfer, indicating that cross‑lingual learning of hierarchical generalization is conditionally achievable.

By Shin Yamaguchi, Masaya Taniguchi, Tatsuki Kuribayashi, Taro Watanabe, Yoshinobu Kano
arXiv Computation and Language
Oct 23

RetiJev: Fast and Source-Grounded Ophthalmic Decision Models

RetiJev is a family of 0.8B‑9B decision models designed for ophthalmic clinical decision support, offering fast, accurate, and privacy‑preserving inference without autoregressive decoding. The authors created RetiJev‑Data, a source‑grounded corpus of 78,158 clinical‑decision and diagnosis questions, 50,074 guideline‑grading questions, and a 24,662‑question evaluation benchmark. RetiJev outperforms a 27B LLM and the commercial Jev 1.13 model on guideline grading and clinical decisions, achieving the highest accuracy while answering a case in 158 ms on a single GPU—3.5× faster than LLM generation and 470× faster than reasoning.

By Bo Liu, Lianyu Wang, Meng Wang, Ching-Yu Cheng, Huazhu Fu
arXiv Computation and Language
Oct 23

IndoNutriQA: An Indonesian Clinical Nutrition Question Answering Benchmark

IndoNutriQA is a newly released Indonesian clinical nutrition question‑answer benchmark comprising 911 expert‑curated question‑answer pairs drawn from real clinical‑nutrition reports. Each pair is grounded in a clinically relevant context and enriched with structured metadata such as anthropometric assessment, medical diagnosis, and document provenance. The dataset supports generative, extractive, and retrieval‑augmented QA, and includes fine‑tuned baselines and zero‑shot evaluations of contemporary large language models.

By Tirana Noor Fatyanosa, Ummu Ditya Erliana, Mahendra Data, Muhammad Nurwegiono, Eleazar Tadeo Eman, Alham Fikri Aji
arXiv Computation and Language
Oct 23

Protection Is Not One Event: A Survey of RAG Protection Research by Measured Endpoint

The paper surveys 118 retrieval‑augmented generation (RAG) defense studies, categorizing them by the specific protection endpoint rather than the claimed goal or mechanism. It introduces a seven‑family taxonomy covering content disclosure, privacy proxies, access control, corpus‑use detection, and pipeline integrity, and reports high inter‑annotator agreement (kappa = 0.867). The survey highlights gaps: most studies target non‑primary endpoints, rely on generic QA datasets and non‑adaptive attacks, and fail to prevent private‑fact recovery even when verbatim copying is blocked.

By Peiyang Liu, Xi Wang, Di Liang, Youru Li, Zhi Cai, Wei Ye
arXiv Computation and Language
Oct 23

SCOPE: Compiling Natural-Language Policies into Neurosymbolic Predicate Programs

SCOPE compiles natural‑language policies into neurosymbolic predicate programs that combine small neural models, deterministic extractors, and logical composition. The system is jointly optimized with adversarially generated counterexamples, achieving higher accuracy than prompt‑optimization, task‑decomposition, and workflow‑generation baselines on real‑world policy benchmarks. Programs trained with the weakest executor transfer to other models, enabling deployment on inexpensive models with stable, interpretable predictions.

By Arnon Mazza, Elad Levi
arXiv Computer Vision
Oct 23

Visual Grounding First, Multimodal In-context Learning Follows

The paper introduces ProCoRe, a contrastive reinforcement learning approach that improves visual grounding in multimodal large language models (MLLMs) for in-context learning (ICL). By generating captions for contrastive image pairs, decomposing them into propositions, and optimizing contrastive rewards, ProCoRe enhances alignment with relevant visual evidence without requiring ICL-formatted training data. The method achieves significant gains in multimodal ICL classification accuracy and captioning ROUGE-L over the Qwen-VL-3-8B baseline and outperforms state‑of‑the‑art ICL‑data‑dependent methods across multiple benchmarks. Additionally, the authors present PICL, a personalized multimodal ICL captioning benchmark designed to test genuine understanding of visual demonstrations.

By Minhyuk Seo, Minjae Lee, Chaeeun Lee, Wei Lin, Muhammad Jehanzeb Mirza, Tinne Tuytelaars, Jonghyun Choi
arXiv Computer Vision
Oct 23

One-shot federated unsupervised domain adaptation with Smoothed Knowledge Distillation and teacher refinement

The paper introduces SKD-ETR, a one‑shot federated unsupervised domain adaptation framework that uses Smoothed Knowledge Distillation and Exponential Moving Average Teacher Refinement. It trains a student model on an unlabeled target domain with smoothed pseudo labels from an ensemble of source models, then refines each source model by moving its parameters toward the student. This bidirectional refinement improves pseudo‑label quality and student generalization while eliminating complex aggregation and multiple communication rounds, achieving competitive performance on OfficeHome, Office‑Caltech, and DomainNet with low communication and computation overhead.

By Ali Abedi, Q. M. Jonathan Wu, Ning Zhang, Farhad Pourpanah
arXiv Computer Vision
Oct 23

Where Did the Light Come From? Diagnosing Cast-Shadow Reasoning in VLMs and Borrowing the Missing Shadow Prior from Image Generation Models

The paper introduces ShadowBench, a benchmark built from real photographs that tests vision‑language models (VLMs) on three shadow‑related tasks: reading a visible shadow’s direction, inferring where a shadow should fall when absent, and judging physical consistency of shadows. Evaluations of 19 open VLMs and proprietary models show that while they can read shadow directions well, they perform poorly on inferring absent shadows and on consistency checks, with only Gemini‑3.1‑pro marginally above chance. The authors demonstrate that routing VLMs through image‑generation models (IGMs) significantly improves inference, suggesting that IGMs’ latent knowledge of illumination can bridge the gap between VLMs and physical reasoning.

By Ryugo Morita, Shunsuke Kitada, Stanislav Frolov, Brian Bernhard Moser, Ko Watanabe, Riku Takahashi, Jinjia Zhou, Andreas Dengel
arXiv Computer Vision
Oct 23

WovenAnchor Matcher: Specialized Intra- and Inter-Image Context Modeling for Feature Matching

The paper introduces WovenAnchor Matcher (WAM), a semi‑dense feature matching framework that separates intra‑image and inter‑image context aggregation. WAM uses WovenMamba for horizontal‑then‑vertical state‑space propagation to preserve structured 2D dependencies within each image, and AnchorCrossAttention to gather context from plausible corresponding regions across image pairs via cross‑attention over local anchors. Experiments on MegaDepth show WAM achieving 66.1 pose AUC@5°, a 2.0‑point improvement over the JamMa baseline, and ablations confirm the necessity of both specialized components.

By Zhiyang Li, Ruijiang Jin, Thibaut Klenke, Yusuke Sekikawa, Nakamasa Inoue
arXiv Computer Vision
Oct 23

CAVC: Convergence-Adaptive Audio-Visual Cache for Modality-Asymmetric Diffusion Generation

The paper introduces CAVC, a training‑free, modality‑asymmetric caching framework that speeds up diffusion‑based audio‑visual generation by selectively caching audio denoiser outputs while continuing full video refinement. By detecting when audio denoiser outputs become cache‑tolerant, CAVC bypasses audio denoising and cross‑modal attention in the middle of the sampling trajectory, then restores a few joint steps at the end to correct drift and improve synchronization. Experiments on public benchmarks show 1.10–1.35× wall‑clock acceleration, better visual dynamics and temporal statistics, and competitive audio fidelity and alignment.

By Lifan Jiang, Yibo Zhao, Senlin cheng, Bolei Xu, Xiaobo Li, Can Ye, Dehai Zhao, Deng Cai, Boxi Wu
arXiv Computer Vision
Oct 23

Direct Forcing: A Single-Stage Training Recipe for Few-Step Autoregressive Text-to-Video Generation

Direct Forcing is a single‑stage training framework for few‑step autoregressive text‑to‑video generation that eliminates the need for separate causal adaptation and distribution matching stages. It uses reference videos both as paired examples for causal continuation alignment and as a target distribution for rollout distribution matching, with dynamic gradient balancing to adjust their contributions during training. Starting from a bidirectional Wan2.1‑1.3B model, Direct Forcing trains a four‑step generator in only 250 updates, achieving state‑of‑the‑art VBench scores without additional initialization or diffusion‑score supervision.

By Haojun Xu, Jie Huang, Xin Lu, Mingchen Zhong, Zihao Fan, Linjiang Huang, Si Liu
arXiv Computer Vision
Oct 23

I$^2$-KAN: Interpretable Intra- and Inter-Modal Multi-Pathway Representation Learning for Multimodal Sentiment Analysis

I$^2$-KAN is a new multimodal sentiment analysis framework that separates intra‑modal and pairwise inter‑modal information into distinct pathways. Each pathway is refined by a Task‑Decoupled Linear‑Gated Denoising Bottleneck and fused using a structured Kolmogorov‑Arnold Network, enabling explicit pathway associations and multi‑level interpretation. Experiments on CMU‑MOSI, CMU‑MOSEI, and CH‑SIMS v2 show strong performance and reveal that a modality’s standalone predictive power does not always reflect its interaction‑conditioned utility.

By Xiaoting Xie, Feng Qiao, Xianwei Zhang, Hailong Chen, Kaiwen Zheng, Joemon M. Jose, Xuri Ge, Yujun Li
arXiv Computer Vision
Oct 23

WildDepth-v2: A Synchronized RGB--LiDAR--Radar Dataset and Benchmark for Metric Depth, Motion, and Tracking of Wildlife

WildDepth‑v2 expands the original RGB‑LiDAR WildDepth dataset by adding 37 RGB‑LiDAR‑4D‑radar sequences, a frozen hidden test set for RGB‑only depth estimation and RGB‑plus‑sparse‑LiDAR depth completion, and metric 3D trajectories derived from 2D tracks using LiDAR, radar, or monocular depth. The dataset demonstrates that object‑level radar calibration improves monocular depth accuracy (AbsRel 0.120 vs. 0.198) and that once‑per‑track radar scaling reduces 3D position error four‑fold, matching the performance of LiDAR calibration. Additionally, transferring radar Doppler onto LiDAR yields denser motion masks, lowers LiDAR accumulation error, and provides radial speed estimates. whyItMatters":"The dataset supplies the first imaging‑radar wildlife recordings and a benchmark that enables metric depth, motion, and tracking research in ecological contexts, filling a gap where monocular video lacks scale and existing wildlife datasets lack metric ground truth."

By Muhammad Aamir, Sangyun Shin, Naoya Muramatsu, Matthew Wijers, Amir Patel, Andrew Loveridge, Andrew Markham
arXiv Computer Vision
Oct 23

Pos4CHD: A Positive-Free Mask-Guided Counterfactual Synthesis Framework for Fetal Congenital Heart Defect Analysis

Pos4CHD is a positive‑free, mask‑guided counterfactual synthesis framework designed to generate realistic fetal heart ultrasound images with congenital heart defects (CHDs) from only normal samples. It introduces FE‑Gen, a mask‑guided generative model that improves image‑mask alignment, and uses manipulated semantic masks to render anatomically consistent pseudo‑positive images. Experiments on in‑house datasets and expert evaluations show high visual fidelity (−12.97 FID) and anatomical consistency (+3.96% DSC), and the synthesized data boosts performance in zero‑shot classification, lesion detection, and anatomy segmentation tasks.

By Wenjie Xuan, Yapeng Li, Hang Su, Chen Cheng, Juhua Liu, Sheng Zhao, Ziming Huang, Bo Du
arXiv Computer Vision
Oct 23

Navigating human-robot interaction challenges in private and residential aged care homes

The chapter examines how robotic systems and ambient intelligence can improve care for older adults in both residential aged care homes and private homes, highlighting the extreme and challenging nature of these environments. It reviews embodied and non‑embodied solutions for daily living, safety monitoring, and social engagement, emphasizing hybrid deployments that combine robots with fixed and wearable sensors. Design considerations focus on accessibility, privacy, trust, and practical evaluation metrics such as false alarms, fault recovery time, staff workload, and long‑term reliability, while outlining future research challenges for scalable, regulation‑ready deployments.

By David Ahmedt-Aristizabal, Mohammad Ali Armin, Akila Pemasiri, Harshala Gammulle, Tharindu Fernando, David Silvera-Tawil, Simon Denman
arXiv Statistics ML
Oct 23

Multiply Robust Conformal Risk Control with Coarsened Data

The paper introduces two new methods for conformal prediction when training data are coarsened or partially missing. Robust Split Prediction (RSP) extends doubly robust estimating equations to general coarsening patterns, producing multiply robust prediction sets. Robust Split Conformal Prediction (RSCP) combines semiparametric theory with conformal risk control by fixing a preliminary threshold estimate to restore monotonicity, enabling higher‑order bias‑controlled coverage guarantees. The authors provide theoretical results for various missingness scenarios and demonstrate the methods with simulations and a rheumatoid‑arthritis study.

By Manit Paul, Arun Kumar Kuchibhotla, Eric J. Tchetgen Tchetgen
arXiv AI
Oct 23

SLoSa: Small, Low-Power, and Safe Foundation Models for Robotic Task Planning

SLoSa is a neuro‑symbolic framework that compresses and fine‑tunes foundation models to create small, low‑power, and safety‑aware robotic task planners. It builds a safety‑aware dataset using classical planners, a teacher LLM, and formal verifiers, then applies supervised and reinforcement‑learning fine‑tuning guided by a verification‑based reward machine. Evaluations on PDDL domains, household simulators, and a desktop robotic arm show that verifier‑guided post‑training boosts success rates to 70.5–76.0% and that INT4 deployment on a Jetson Orin consumes only 17.85–26.41 W of power.

By Weizhe Xu, Jialiang Fan, Essien Taylor, Zijun Wang, Russell Joseph, Fanxin Kong
arXiv AI
Oct 23

Correlated Failures of AI Approval Monitors

The paper investigates how AI approval monitors can fail together when they share a base model or poisoned input, using concepts from distributed computing to identify vulnerable monitor sets. Experiments on twelve open‑weight monitors and several closed models show that fooling a monitor into approving fraud is limited to its own evidence source, while blocking correct payments requires overcoming injection resistance. A rule that counts distinct evidence sources rather than votes reduces false rejections, and adding a rule engine that cannot be persuaded restores safety and liveness against the measured attacks.

By Luca Zanolini