AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,884 stories · RSS feed

Hugging Face Trending Papers
Aug 27

DPA-I2P: Depth-Guided Projective Alignment for Image-to-Point-Cloud Registration in Autonomous Driving

The paper introduces DPA-I2P, a depth‑guided projective alignment method for image‑to‑point‑cloud registration in autonomous driving. It employs Ray‑Conditioned Metric Depth Encoding and Projection‑Consistent Vision Lifting to align depth and visual cues geometrically, and uses Cross‑Modal Query Pruning to filter unreliable matches during refinement. Experiments on KITTI and nuScenes show significant improvements, reducing rotation and translation errors by up to 55.6% compared to existing implicit baselines.

arXiv Machine Learning
Aug 27

Adversarial Training of Linear Models under Stealthy Attacks

The paper introduces a detector‑based switched model to defend linear predictive models against stealthy false data injection attacks. It derives a convex formulation of the adversarial risk that incorporates protected features and a hyperparameter for attack probability, allowing an explicit trade‑off between clean and attacked data performance. Numerical experiments on real and synthetic datasets demonstrate improved performance on partially attacked data, even when the attack probability is misspecified.

By Lovisa Eriksson, Dave Zachariah, Andr\'e M. H. Teixeira
arXiv Machine Learning
Aug 27

Learning Mixtures of Plackett-Luce Models for Multi-Objective Alignment

The paper introduces MoPLEx, an expectation‑maximization algorithm for learning mixtures of Plackett‑Luce models from multi‑way ranking data. It augments rankings with synthetic responses from a base language model and uses a gradient‑based estimation to reduce inference cost, enabling efficient fitting of large‑scale models. Experiments show the method achieves low probability estimation error and improves clustering and ranking accuracy by 43.7% and 15.2% over baselines.

By Dongyue Li, Ziniu Zhang, Lu Wang, Hongyang R. Zhang
arXiv Computer Vision
Aug 27

Automatic weld seam segmentation for industrial quality control: a comparison of RGB and polarimetric imaging with CNN and transformer architectures

The study evaluates automatic weld seam segmentation using RGB and polarimetric images, comparing convolutional neural networks (CNNs) and transformer architectures. In controlled RGB settings, CNNs achieve a mean mask mAP50 of up to 0.87, but performance drops significantly under uncontrolled conditions. Polarimetric imaging, combined with geometric augmentation, reaches a mean mask mAP50 of up to 093 even in uncontrolled settings, and transformer models, especially RF‑DETR, maintain high accuracy under viewpoint shifts while CNNs fail.

By Simone Garbin, Leonardo Venturoso, Marco Todescato
arXiv Computation and Language
Aug 27

Think-Probe-Respond: Improving Large Language Models as Judges of Research Idea Novelty

The paper introduces Think‑Probe‑Respond (TPR), a lightweight method to improve large language models’ ability to judge the novelty of research ideas. It identifies a systematic bias where models tend to label ideas as "medium novel" despite generating human‑like rationales, and shows that probing hidden states during reasoning and conditioning the final response on these probes boosts novelty judgment accuracy by 22.30%. TPR effectively reduces the medium‑novelty bias across strong baseline models.

By Tim Schopf, Tobias Schreieder, Akiko Aizawa
arXiv Computation and Language
Aug 27

EgoArgus: Benchmarking VLMs as Situational Assistants for Modality-Grounded User Supports

EgoArgus is a new, human‑annotated dataset that tests visual‑language models (VLMs) as situational assistants in five everyday dialogue‑video scenarios. It evaluates how well VLMs understand and decide when to intervene, especially when visual and textual cues are helpful, irrelevant, or conflicting. The study finds that current VLMs still struggle to reliably act as egocentric assistants and that existing modality‑bias mitigation methods offer limited improvement.

By Yu-Chien Tang, Yu-Hsiang Liu, An-Zi Yen
arXiv Computation and Language
Aug 27

When RAG Fails to Equalize: Geo-bias in Factual Question Answering over Public Companies

The paper investigates whether retrieval‑augmented generation (RAG) can uniformly correct factual errors in large language models (LLMs) by testing six LLMs on a benchmark of about 2,000 public companies. In a controlled factual QA setting, the authors evaluate the models under four conditions—no‑context, perfect context, misleading context, and distraction context—across four atomic attributes. Results show significant geographic disparities in baseline accuracy, and while perfect context improves performance, it does not eliminate these gaps; misleading context often leads models to copy incorrect information, and larger models only marginally reduce structural biases.

By Abhinav Havaldar, Enrico Santus
arXiv Computation and Language
Aug 27

Query-Side Attacks on GNN-Based KGQA: Tracing Failures from Entity Linking to Answer Generation

The paper investigates where failures occur in GNN‑based Knowledge Graph Question Answering pipelines when faced with adversarial question perturbations. By isolating stages—entity linking, subgraph retrieval, GNN reasoning, and answer generation—and applying two answer‑preserving attacks (Compositional Restructuring and Relation Synonym Swap), the authors find that subgraph construction is responsible for over 99% of end‑to‑end failures, even though the correct answer is often present in the retrieved subgraph. This challenges the assumption that reasoning models are the weak link and highlights subgraph construction as the critical mitigation target.

By Pankaj Kumar, Subhankar Mishra
arXiv Computation and Language
Aug 27

Anchoring Bias in LLM-as-a-Judge Systems: Prior Scores Compromise Evaluation Independence

The study investigates how prior scores influence large language model (LLM) judgments in the LLM-as-a-Judge paradigm. By testing three prompt conditions—no metadata, revision framing, and anchored metadata containing prior scores—the authors find that prior scores systematically bias evaluations, shifting ratings toward those scores across 192,000 attempts. The bias also affects categorical decisions, blocking 48% of error corrections and flipping 10.18% of correct judgments, and is not mitigated by Chain-of-Thought or a warning, underscoring the need for careful context engineering.

By Ante Kapetanovic, Kemal Altwlkany, Andro Mercep, Tomislav Duricic, Emanuel Lacic
arXiv Computation and Language
Aug 27

Adaptive Triggering for Bias Correction in LLM Reasoning

The paper introduces an adaptive triggering mechanism for bias correction in large language model (LLM) reasoning. By framing bias intervention as an online change‑point detection problem, the authors update a CUSUM statistic at each step using either a white‑box next‑token probability signal or a black‑box LLM judge signal, and inject corrective prompts only when the accumulated evidence exceeds a calibrated threshold. Experiments on gpt‑4o‑mini and six open‑weight models show that adaptive black‑box triggering restores most of the accuracy lost by fixed‑interval interventions while reducing the number of corrections, whereas the white‑box signal improves ambiguous‑item accuracy but can hurt disambiguated‑item accuracy due to difficulty distinguishing stereotype reliance from correct evidence.

By Nayoung Kim, Mickey Mancenido, Huan Liu
arXiv Computer Vision
Aug 27

Bidirectional Temporal Dynamics Modeling for EEG-based Driving Fatigue Recognition

The paper introduces DeltaGateNet, a framework that models bidirectional temporal dynamics in EEG signals for driving fatigue recognition. It uses a Bidirectional Delta module to separate positive and negative temporal differences, and a Gated Temporal Convolution module to capture long‑term dependencies while preserving channel specificity. Experiments on SEED‑VIG and SADT datasets show that DeltaGateNet outperforms existing methods, achieving high intra‑subject and inter‑subject accuracies across balanced and unbalanced data.

By Yip Tin Po, Jianming Wang, Yutao Miao, Jiayan Zhang, Yunxu Zhao, Xiaomin Ouyang, Zhihong Li, Nevin L. Zhang
arXiv Statistics ML
Aug 27

Barycentric Weak Inner-Product Gromov-Wasserstein

The paper introduces a weak Gromov-Wasserstein (wGW) framework that compares source relations with relations between target conditional laws, focusing on inner-product relations and preserving conditional means. It defines the barycentric weak inner-product GW (wIGW) distance, proves existence of minimizers under finite second moments, and presents a ridge-regularized dual formulation leading to an iterative algorithm for finitely supported measures. Experiments on point clouds, graphs, and a PBMC multiome study demonstrate that mean-preserving target refinements can incur zero cost and improve atlas-based cell type transfer.

By Youssef Mroueh
arXiv Statistics ML
Aug 27

Schr\"odinger Bridges over Kinetic Swarming Models

The paper studies finite‑horizon minimum‑energy steering of inertial swarms under stochastic disturbances, focusing on mean‑field models with Cucker–Smale alignment or Morse attraction–repulsion interactions. It formulates the problem as a Schr"odinger bridge, deriving nonlinear, time‑symmetric optimality systems and proposing nested fixed‑point schemes for numerical solution. Numerical experiments demonstrate that the optimal corrective drift can either exploit or counteract the natural interaction forces, depending on their alignment with the steering objective.

By Asmaa Eldesoukey, Md Zulfiqur Haider, Italo Napolitano, Yongxin Chen, Abhishek Halder
arXiv Computer Vision
Aug 27

PAGS: Autofocusing Photoacoustic Tomography via Speed-of-Sound-Adaptive Gaussian Splatting

PAGS is a new differentiable framework for blind autofocusing in photoacoustic computed tomography (PACT). It represents the initial pressure field with sparse Gaussian photoacoustic sources and models the speed‑of‑sound heterogeneity using a compact anisotropic path‑averaged field parameterized by spherical harmonics. By jointly optimizing the Gaussian source parameters and the speed‑of‑sound field in a closed‑loop signal‑domain approach, PAGS achieves sharper reconstructions in heterogeneous media, remains robust to sparse‑view sampling, and offers computational advantages through an analytic Gaussian acoustic projection.

By Jiarui Ge, Jintao Ma, Bangxu Fan, Jinyan Zhang, Xiaokang Yang, Shuai Na, Xiaoyun Yuan
arXiv Computation and Language
Aug 27

From Passive Response to Proactive Correction: Enhancing LLM Robustness Against Input Fact Perturbations

The paper introduces DEDUCE, a three‑stage framework that turns large language models into proactive error correctors by detecting input fact errors, devising correction strategies, and delivering reliable answers. It also presents MisFactQA, a dataset of factual errors, and new metrics for robustness evaluation. Experiments on TruthfulQA, FalseQA, and MisFactQA show significant gains in accuracy and error correction across Qwen, LLaMA, and Gemma models.

By Ping Wang, Xiangguo Sun, Bingbing Xu, Guocong Li, Xiaofeng Meng
arXiv Machine Learning
Aug 27

Does Fine-Tuning Undo Activation Steering? Behavioural Recovery Without Weight-Edit Reversal

The paper investigates whether fine‑tuning a language model erases previously embedded activation steering interventions that suppress refusals and encourage brevity. Across five instruction‑tuned models (3B–14B) subjected to non‑adversarial supervised fine‑tuning (SFT) and reinforcement learning from human feedback (RLHF), the authors find that the steering’s behavioural effect degrades when the fine‑tuning objective conflicts with the targeted behaviour, yet the underlying weight edits remain largely unchanged. Mechanistically, the steering vectors survive with minimal alteration, but functionally the steering is vulnerable and must be re‑validated after downstream training.

By Philipp E. Glass, Allan Tucker, Yongmin Li, Alina Miron
arXiv Computer Vision
Aug 27

GraftSR: Grafting Authentic Textures for Real-World Image Super-Resolution via Identical-Instance Guidance

GraftSR is a diffusion-based super‑resolution framework that uses reference images of the same object to guide texture restoration, mitigating hallucination. It introduces a dual‑mask reference guidance mechanism to decouple texture extraction from application, avoiding reliance on spatial alignment. The authors also release TexRefSR‑141K, a large dataset of reference pairs with spatial masks, and show that GraftSR outperforms existing methods on the TexRefSR‑Eval benchmark, reducing LPIPS by 20.2%.

By Qifan Yu, Haoran Bai, Zongyao He, Weijie He, Sibin Deng, Honggang Qi, Ying Chen
arXiv Computation and Language
Aug 27

Overview of SHROOM-Visions 2026: A Shared Task on Hallucination Detection in Large Vision-Language Models

In 2026, the SHROOM-Visions shared task was launched at the UncertaiNLP Workshop co‑located with EMNLP to address hallucinations in large vision‑language models. The task builds on the SHEEP dataset and asks participants to detect and classify fine‑grained hallucination spans in image‑conditioned text generation across four languages (Chinese, English, French, Italian) using a five‑class taxonomy. The competition attracted 27 teams and over 600 system submissions, with top systems achieving character‑level, label‑conditioned, and IoU scores of 0.58, 0.46, and 0.51 respectively, surpassing baselines by 30‑40 points.

By Ra\'ul V\'azquez, Aman Sinha, Chuyuan Li, Claudio Savelli, Eduardo Cal\`o, Emilio Raimond, Stella Frank, Hengyu Luo, Flavio Giobergia, Vincent Segonne, Lorenzo Vaiani, J\"org Tiedemann, Timothee Mickus
arXiv Machine Learning
Aug 27

Provable Privacy Attacks on Trained Shallow Neural Networks

The paper investigates provable privacy attacks on trained 2‑layer ReLU neural networks, specifically membership inference and data reconstruction. It demonstrates that the implicit bias of such networks can be leveraged to identify, with high probability, whether a given point was part of the training set in high‑dimensional, nearly orthogonal settings, and to construct a finite set containing a constant fraction of training points in a univariate setting. This work claims to be the first to reveal provable vulnerabilities arising from implicit bias in shallow neural networks.

By Guy Smorodinsky, Gal Vardi, Itay Safran
arXiv Computation and Language
Aug 27

Retrieval-Augmented Agentic Rubric Generation for Reliable Medical Response Evaluation

The paper introduces a retrieval‑augmented multi‑agent framework that automatically generates instance‑specific evaluation rubrics for medical language models. By retrieving authoritative medical evidence, decomposing it into atomic facts, and combining these with user interaction constraints, the system produces fine‑grained criteria that outperform GPT‑4o on HealthBench and LLMEval‑Med. The generated rubrics also guide response refinement, improving medical LLM output quality by 9.2%.

By Yinzhu Chen, Abdine Maiga, Hossein A. Rahmani, Emine Yilmaz