AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,808 stories · RSS feed

arXiv Computer Vision
Sep 4

TetraSDF: Analytic Isosurface Extraction with Multi-resolution Tetrahedral Grid

TetraSDF is an analytic isosurface extraction framework that represents a neural signed distance function (SDF) using a ReLU MLP combined with a multi-resolution tetrahedral positional encoder. This design preserves a global continuous piecewise affine (CPWA) structure, enabling exact extraction of the zero-level set as a triangle mesh while maintaining the expressiveness of grid-based encoders. The method also introduces a fixed analytic input preconditioner to reduce directional bias and stabilize training, achieving SDF reconstruction accuracy that matches or surpasses existing grid-based encoders across multiple benchmarks.

By Seonghun Oh, Youngjung Uh, Jin-Hwa Kim
arXiv Computer Vision
Sep 4

Quantum Implicit Neural Representations for Novel View Synthesis

Quantum Implicit Neural Representations for Novel View Synthesis introduces 3D Quantum Implicit Scene Representation (3D-QISR), a hybrid quantum‑classical radiance‑field model that replaces the classical NeRF backbone with parameterised quantum circuits. Two architectures are proposed: Full 3D-QISR, which uses a unified quantum state, and Dual‑Branch 3D-QISR, which separates spatial and view‑dependent embeddings to reduce complexity and improve scalability. Experiments on moderate‑resolution novel‑view synthesis benchmarks show that 3D‑QISR on simulated quantum hardware matches or outperforms classical baselines while using fewer than half the trainable parameters.

By Daniele Lizzio Bosco, Shuteng Wang, Giuseppe Serra, Vladislav Golyanik
arXiv Computer Vision
Sep 4

Thermal Imaging for Contactless Cardiorespiratory and Sudomotor Response Monitoring

The paper investigates the use of thermal infrared video to non‑invasively monitor cardiorespiratory and sudomotor activity in industrial human‑machine interfaces. It presents a signal‑processing pipeline that tracks facial regions, aggregates thermal signals, and separates slow sudomotor trends from faster heart‑rate and breathing‑rate components. Experiments on 31 driver‑monitoring sessions show that thermal imaging can estimate heart rate, breathing rate, and electrodermal activity with reasonable accuracy, while highlighting challenges such as ROI selection, polarity changes, latency, and subject variability.

By Constantino \'Alvarez Casado, Mohammad Rahman, Sasan Sharifipour, Nhi Nguyen, Manuel Lage Ca\~nellas, Xiaoting Wu, Miguel Bordallo L\'opez
arXiv Computer Vision
Sep 4

Subgroup performance analysis of adaptation strategies for chest X-ray foundation models

The study examines how three parameter‑efficient adaptation methods—linear heads on the raw CLS token, an MLP, and an attention‑pooling module—affect pathology classification accuracy and subgroup fairness when applied to a frozen Rad‑DINO chest X‑ray encoder. Using the MIMIC‑CXR dataset, the authors evaluate eight pathologies across race, sex, and imaging‑view subgroups, finding that attention pooling yields the best overall performance and encodes protected attributes most strongly, yet higher performance does not consistently reduce subgroup disparities. The results show that attribute encoding strength and layer choice do not reliably predict fairness outcomes, indicating that fairness must be assessed directly for each task.

By Dhruv Gupta, Emma A. M. Stanley, Fabio De Sousa Ribeiro, Sujal R. Desai, Ben Glocker
arXiv AI
Sep 4

HalluPeer: A Taxonomy-driven Benchmark for Detecting Hallucinations in Scientific Peer Reviews

HalluPeer is a new benchmark designed to detect hallucinations in scientific peer reviews. It provides aligned triples of paper content, human-written reviews, and hallucination-injected reviews, annotated for detection, classification, and localization. Experiments on 12K papers and 38K reviews show that existing detectors struggle to separate hallucinations from legitimate critique, and that HalluPeer-defined hallucination patterns occur in real peer reviews.

By Tzu-Ling Lin, Dong-Ting Yao, Teng-Fang Hsiao, Wei-Chih Chen, Hong-Han Shuai
arXiv Computation and Language
Sep 4

Jina-OCR-v1: Efficient Document Parsing with Speculative Decoding and Dense Verifiable Rewards

Jina-OCR-v1 is an end‑to‑end document parsing model designed for low‑budget GPUs, combining a compressed‑vision encoder with a 3B mixture‑of‑experts decoder that activates about 570 M parameters per token. It uses a FastMTP speculative decoding head that shares a single draft block across three prediction steps, with greedy verification ensuring lossless decoding. Post‑training includes instruction alignment, robustness fine‑tuning on difficult documents, and GRPO with dense verifiable rewards, achieving 91.14 on OmniDocBench v1.6 and 83.4 on olmOCR‑Bench while delivering the highest page throughput at 2.57 pages per second on an NVIDIA L4 GPU.

By Alejandro Bar\'on Garc\'ia, Feng Wang, Emilia Garcia Casademont, Han Xiao
arXiv AI
Sep 4

The Attention Triangle in Audio-Video Models

The paper investigates audio‑video diffusion models by examining the "attention triangle"—the cross‑attention links among text, audio, and video. It finds that the audio‑video edge is bidirectional and heavily influenced by model biases, leading to semantic leakage when prompts conflict with learned priors. The authors develop attention‑derived diagnostics and inference‑time interventions that improve semantic grounding without sacrificing generation quality.

By Sagi Polaczek, Noa Kraicer, Gal Metzer, Zhuo Ning, Ali Mahdavi-Amiri, Daniel Cohen-Or, Raja Giryes
arXiv Machine Learning
Sep 4

Beyond Straightness: Non-Crossing Flow Matching via Quantile AlignTree Coupling

The paper introduces Quantile AlignTree Flow Matching (QAT‑FM), a structured coupling method that builds a hierarchical, quantile‑aligned tree to connect a Gaussian prior with a target distribution. QAT‑FM achieves efficient coupling construction in ≠ Nd log N time and allows per‑pair source sampling in ≠ d time, enabling scalable training for high‑dimensional generative tasks. The authors prove that the coupling preserves marginal consistency, produces non‑crossing interpolation paths, and improves path separation compared to independent coupling, while also extending naturally to conditional generation.

By Junyi Lin, Mengyu Li, Jingxuan Hu, Kejun He, Cheng Meng
arXiv Machine Learning
Sep 4

High-Dimensional Learning Dynamics of Attention-Indexed Models

The paper investigates the training dynamics of attention mechanisms in high-dimensional settings, focusing on attention-indexed models that encompass multi-layer and multi-head architectures. It shows that while the loss landscape can be described by a finite set of trace order parameters, the online stochastic gradient descent dynamics involve an infinite hierarchy of matrix moments that can be accurately approximated by a finite truncated system. The study further reveals that the choice of attention parameterization acts as an implicit bias: untied attention can get trapped in uninformative states, whereas tied attention induces symmetry breaking and enables weak recovery with θ(d² log d) samples, and untied attention exhibits a fast-slow dynamic leading to weak recovery when symmetry is broken.

By Yizhou Xu, Margarita Sagitova, Lenka Zdeborov\'a, Florent Krzakala
arXiv Machine Learning
Sep 4

Mind the Gap: Robustness Risks in PII Detection Systems

The paper "Mind the Gap: Robustness Risks in PII Detection Systems" examines how modern PII detection methods—encoder-based NER, rule-based hybrid detection, and generative LLM extraction—fail when faced with realistic distribution shifts such as noisy, unstructured, and informal inputs. A stress‑test benchmark covering seven categories of natural shift reveals that each architecture degrades in distinct ways: encoder models struggle with unseen surface forms and boundary detection, rule‑based systems falter on non‑standard formats, and LLMs suffer from entity‑type confusion and generation instability. The authors argue that aggregate benchmark scores hide these deployment‑critical weaknesses and propose a hybrid detection pipeline with a QA‑driven feedback loop to iteratively mitigate risks, also releasing the benchmark for OOD‑aware evaluation.

By Adeel Zafar, Slawomir Nowaczyk
arXiv Machine Learning
Sep 4

BharatGather: A Culturally-Informed Benchmark Dataset for Misinformation and Fake News Detection in Indian Public Events

BharatGather is a curated, multi-source dataset designed for binary misinformation classification in Indian public events such as religious festivals, political rallies, and cultural gatherings. The corpus contains 14,646 records assembled through systematic web scraping of fact‑checking platforms, multimedia transcript extraction, and LLM‑mediated synthetic augmentation to capture narrative diversity. It serves as a culturally informed benchmark to evaluate and develop fake‑news detection systems tailored to the socio‑cultural nuances of India’s mass‑gathering context.

By Parth Bramhecha, Smit Deshmukh, Sairaj Bodhale, Adwait Borate, Raviraj Joshi
arXiv AI
Sep 4

Seeing Before Synthesizing: VLM-Guided Transition Event Discovery for Weakly-Supervised Dense Video Captioning

The paper introduces Seeing Before Synthesizing (SBS), a weakly-supervised dense video captioning framework that uses a vision‑language model to generate frame‑level narratives for gaps between events and detect transitions based on semantic changes. SBS refines temporal masks by aligning transition points with vision‑language cues, rather than relying on rigidly placed synthetic captions. Experiments on ActivityNet Captions and YouCook2 show that SBS achieves state‑of‑the‑art results in both captioning and localization tasks.

By Ye-Chan Kim, Seunghee Choi, SeungJu Cha, Si-Woo Kim, Hwiseon Kim, Hyungee Kim, Dong-Jin Kim
arXiv Statistics ML
Sep 4

Markov Chain Monte Carlo with Diffusion Paths

The paper introduces a new Markov chain Monte Carlo method that samples from multimodal distributions by interpolating along the diffusion path of a noising diffusion process, preserving mode weights and improving mixing. It proposes a Metropolis-adjusted diffusion path (MAD-Path) sampler that corrects for bias from approximate score estimates and discretization errors, ensuring the target distribution remains invariant. Experiments on Bayesian posteriors demonstrate that MAD-Path outperforms tempering-based MCMC and unadjusted diffusion samplers in global exploration and accurate mode-weight estimation.

By Han Chen, Sifan Liu, Jun Yang
arXiv AI
Sep 4

ScoreMix: Synthetic Data Generation by Score Composition in Diffusion Models Improves Recognition

ScoreMix is a self‑contained synthetic data generation method that improves recognition tasks by mixing class‑conditioned scores along reverse diffusion trajectories, thereby creating hard synthetic samples without external resources. The approach shows that selecting classes far apart in the discriminator’s embedding space yields larger performance gains, up to 3% more improvement than proximity‑based selection. Across eight public face recognition benchmarks, ScoreMix boosts accuracy by up to 7 percentage points, demonstrating robustness and practicality without hyperparameter tuning.

By Parsa Rahimi, Sebastien Marcel
arXiv Machine Learning
Sep 4

Beyond Blur: A Semantic Tri-view Pipeline for Teledermatology Gradability via Skin Micro-relief

The paper introduces the Semantic Tri-view Pipeline, an interpretable system that automatically screens teledermatology photographs for gradability by analyzing epidermal micro-relief across up to three smartphone views. It uses a lightweight DeepLabV3+ model to segment micro-relief fidelity and aggregates the resulting spatial masks with logistic regression, leveraging viewpoint redundancy to improve robustness. Evaluated on the SCIN dataset, the approach raises the AUC from 0.81 to 0.96 on optically clear cases, offering real‑time, privacy‑by‑design feedback to filter ungradable photo sets before clinician review.

By Robert Engel
arXiv AI
Sep 4

Decentralized Vision-Based Autonomous Aerial Wildlife Monitoring

The paper introduces a decentralized, vision-based system using multiple quadrotors equipped with a single RGB camera for monitoring wildlife. It emphasizes scalability, low bandwidth, and minimal sensor requirements, enabling robust identification and tracking of large species in natural habitats. The authors present novel coordination and tracking algorithms that operate without centralized communication, and validate the approach with real-world field experiments.

By Makram Chahine, William Yang, Alaa Maalouf, Justin Siriska, Ninad Jadhav, Daniel Vogt, Stephanie Gil, Robert Wood, Daniela Rus
arXiv AI
Sep 4

Proactive Service Agents: A Unified Decision Framework, Methods, and Evaluation

The paper presents a unified framework for proactive service agents, defining proactivity as an agent’s ability to infer service opportunities from incomplete signals and decide whether to remain silent, ask, assist, or act. It models this as a partially observable sequential decision process constrained by authorization and risk, integrating timing, content, and delivery into a single structured action. The authors categorize existing methods along a decision pipeline—state and need estimation, intervention gating, action construction, and feedback adaptation—and propose standardized metrics for evaluating triggering, timing, calibration, user burden, safety, and policy value across diverse interaction modalities.

By Yan Tang, Tingyu Cao, Yuanbo Tang, Huaze Tang, Keer Hu
arXiv AI
Sep 4

SVG-Score: Human-Aligned Evaluation of Text-to-SVG Generation

SVG-Score introduces a human‑aligned evaluation framework for text‑to‑SVG generation, addressing the shortcomings of existing image‑based metrics like CLIPScore that poorly capture SVG‑specific errors such as color, count, and spatial inaccuracies. The authors first demonstrate that CLIP‑based scores are largely insensitive to these errors and that generic Vision‑Language Models respond inconsistently across error types and styles. They then present a human‑annotated Semantic Alignment dataset and develop two complementary evaluators: a CLIP‑based scorer adapted to vector graphics and a VLM judge refined through supervised fine‑tuning and reward‑shaped reinforcement learning, enabling both fast large‑scale and expressive, interpretable assessment of SVG generators.

By Marco Cipriano, Leonardo Zini, Alexandra Schild, Valentin Teutschbein, Afsana Mimi, Marcella Cornia, Lorenzo Baraldi, Gerard de Melo
arXiv Computer Vision
Sep 4

FlashRender: Few-Step Generative Rendering via Camera-Controlled Video MeanFlow

FlashRender is a few-step generative rendering framework that quickly retakes a source video along a target camera trajectory. It addresses discretization error by introducing Representation Transformation and Alignment (RETA) to align source-video representations with target-video features, reducing denoising trajectory curvature. The model is further refined with a MeanFlow objective and on-policy flow map distillation, achieving video quality and geometric consistency comparable to multi-step baselines at a 25× lower sampling cost while improving camera controllability.

By Byeongjun Park, Byung-Hoon Kim, Hyungjin Chung
arXiv AI
Sep 4

FLY-EVAL++: An Evidence-Driven Evaluation Protocol for Safety-Constrained Flight Prediction with Large Language Models

FLY-EVAL++ is an evidence-driven evaluation protocol designed for safety-constrained flight prediction with large language models. It combines deterministic verification of protocol compliance, physical feasibility, and safety constraints with rubric-guided aggregation into interpretable multi-dimensional scores. Applied to Flight Trajectory and Attitude Prediction, the protocol revealed that safety compliance is the most discriminative dimension among 66 LLMs, with models showing up to 28-point differences in safety scores and recurrent failures such as safety violations under physically plausible predictions and instability in multi-step rollouts.

By Yalun Wu, Junfeng Fang, Jiawei Wang, Haotian Liu, Qijun Yang, Minghan Yang, Hongcheng Guo, Zhoujun Li, Boyang Wang