AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,798 stories · RSS feed

arXiv Machine Learning
Sep 25

Edge AI on Constrained Devices for Binary Sleep-Wake Classification in Dynamic Environments

This paper introduces an Edge AI system that classifies sleep and wake states on constrained devices using a multimodal pipeline on an ESP32‑S3 microcontroller. It fuses inertial head‑movement sensing with visual pose classification, running in parallel under FreeRTOS to meet real‑time constraints. The two‑stage detection achieves 96.5 % accuracy for motion‑based detection and 89 % for pose classification, proving robust binary sleep‑wake classification in mobile scenarios.

By Stefan Reitmann, Lena Oden
arXiv Machine Learning
Sep 25

Diverse Geometries, Frozen Weights: Robust Heterogeneous Treatment-Effect Estimation via Causal Expert Ensembles

The paper introduces GeoACE, a five‑expert framework for estimating heterogeneous treatment effects that blends a common anchor‑correction estimator with overlap‑aware and outcome‑guided geometries. The ensemble’s task‑level weights are learned from internal validation predictions, frozen before test evaluation, and applied to experts refitted on the full development data. Adding the outcome‑free, overlap‑aware expert O‑Phi‑ACE consistently improves performance across seven benchmarks, achieving the lowest average rank among 11 comparators.

By Ali Haghpanah Jahromi, Mohammad Taheri
arXiv Machine Learning
Sep 25

Don't Read the Log: Execution Traces Contaminate Verifiers in Video-Generation Agents

The paper investigates how providing execution traces to multimodal judges in agentic video‑generation systems can bias their verdicts. On a benchmark of 109 two‑event clips, traces that falsely report successful tool calls cause large‑language‑model judges to incorrectly accept 78–90 % of failures, while contradictory traces lead to 100 % rejection of correct clips. The effect persists even when judges are instructed to consider only the video frames, indicating that the vulnerability stems from the judges’ learned trust in tool logs rather than the visual content itself.

By Jian Xu
arXiv Machine Learning
Sep 25

ELF-REG: Scaling Continuous Diffusion Language Models to Reasoning Tasks

The paper introduces ELF-REG, a scaling of continuous diffusion language models (dLMs) for reasoning tasks. By aligning representations and using a frozen autoregressive teacher to supervise intermediate denoiser features, ELF-REG achieves higher pass@1 rates on GSM8K, MATH-500, HumanEval, and MBPP compared to prior dLMs. The method also enables strong low‑NFE performance through early‑stop decoding of intermediate predictions.

By Zeyu Michael Li, William Xingxu Chen, Bingshuo Qian, Jiayin Liu, Xiang Cheng
arXiv Machine Learning
Sep 25

AERIAL: Adversarial Evaluation of Robustness in Accuracy-Preserving Low-Precision EEG Decoders

The study evaluates how low‑precision compression affects adversarial robustness in EEG decoders used for brain‑computer interfaces. Using BCI Competition IV‑2a data, the authors compare 32‑bit floating‑point models (EEGNet and ShallowConvNet) with models pruned to 50 % and quantized to INT8 via post‑training quantization (PTQ) or quantization‑aware training (QAT). Results show that accuracy‑preserving compression does not improve direct robustness—PGD attack accuracy remains 22–24 % across all variants—yet pruning reduces bidirectional transfer efficiency more than PTQ, indicating that robustness, transferability, and deployment efficiency are distinct properties of compressed EEG decoders.

By Saim Rehman, Muhammad Shafique
arXiv Machine Learning
Sep 25

Agentic Detection of Online Conspiracies

The paper presents an agentic framework for detecting conspiratorial content in social media by inferring the speaker’s intent rather than merely identifying explicit claims. It leverages social context and adaptive tool use, demonstrating superior performance over text-only and non-agentic models on a large Hebrew tweet dataset spanning election cycles and the COVID pandemic. The study highlights the importance of context-aware, reasoning-driven approaches for accurate conspiracy detection.

By Lior Biton, Oren Tsur
arXiv Machine Learning
Sep 25

Capturing Unseen Spatial Heat Extremes Through Dependence-Aware Generative Modeling

The paper introduces DeepX-GAN, a deep generative model that captures spatial dependence in rare climate extremes. It can simulate statistically plausible unseen heat extremes—both direct-hit and near-miss events—beyond the observed record. Applied to the Middle East and North Africa, the model shows that unseen heat extremes disproportionately affect vulnerable countries and that future warming could create new persistent hotspots, underscoring the need for spatially adaptive resilience planning.

By Xinyue Liu, Xiao Peng, Shuyue Yan, Yuntian Chen, Dongxiao Zhang, Zhixiao Niu, Hui-Min Wang, Xiaogang He
arXiv Machine Learning
Sep 25

Pointwise Generalization in Deep Neural Networks

The paper introduces a pointwise generalization theory for fully connected deep neural networks, using a pointwise Riemannian Dimension derived from eigenvalues of learned feature representations across layers. This framework provides hypothesis-dependent, representation-aware generalization bounds that are significantly tighter than traditional size- or norm-based approaches, both theoretically and experimentally. The authors analytically identify structural properties that explain deep networks’ tractability and empirically show that the pointwise Riemannian Dimension captures feature compression, over‑parameterization effects, and optimizer bias.

By Shaojie Li, Yunbei Xu
arXiv Machine Learning
Sep 25

Calpric: Inclusive and Fine-grain Labeling of Privacy Policies with Crowdsourcing and Active Learning

Calpric is a system that combines automatic text selection, segmentation, active learning, and crowdsourced annotation to create a large, balanced training set for privacy policy classification. By simplifying the labeling task, it enables untrained crowd workers to match the performance of trained annotators and reduces inter‑annotator disagreement, cutting labeling costs. The approach yields a dataset of 16,000 policy text segments across nine data categories and produces models that deliver accurate, fine‑grained labels at a cost of roughly $0.92–$1.71 per segment.

By Wenjun Qiu, David Lie, Lisa Austin
arXiv Computation and Language
Sep 25

PTC-Bias: Phoneme-Level Temporal Competition for Bias Retrieval and Post-Decoding Correction in Speech LLMs

PTC-Bias is a two-stage framework that improves contextual biasing in speech large language models by using phoneme-level temporal competition. In the first stage, PTC Retrieval performs frame-synchronous phoneme decoding to generate a compact shortlist of bias words and their speech intervals. The second stage, PTC Correction, applies a local competition between retrieved candidates and mismatched transcript spans within those intervals, reducing near-homophone and word-segmentation errors without extra SpeechLLM passes. Experiments on LibriSpeech demonstrate consistent gains across two SpeechLLMs, with PTC-Bias reducing B-WER by up to 23.9% relative to CTC-Filter while keeping U-WER nearly unchanged.

By Zhiqi Ai, Han Cheng, Shiyi Mu, Yongjin Zhou, Shugong Xu
arXiv Computation and Language
Sep 25

Baseline Shape Decides the Verdict: A Controlled Re-Examination of Ternary Language Models at 60K Parameters

The study re‑examines a reported advantage of a routed ternary (1.58‑bit) language model over a full‑precision transformer at 60K parameters. By running controlled experiments with multiple seeds and a fixed training recipe, the authors find that the apparent benefit largely stems from the choice of baseline model shape rather than the ternary architecture itself. While the routed model does outperform other shapes at a larger 130M‑byte budget, its advantage diminishes when a plain gated diagonal‑SSM block is used, and the ternary penalty varies with architecture and quantization details.

By Gautam Veldanda
arXiv Computation and Language
Sep 25

StepCOPS: Closed-Testing Lower-Tail Certificates for Language-Model Policy Selection

StepCOPS is a new method for selecting a language‑model policy from many checkpoints, prompts, and decoding rules by providing closed‑testing lower‑tail certificates. It uses an independent proposal split to nominate a lower‑tail floor for each candidate, applies exact binomial tests on a fresh certification split, and employs Holm’s step‑down procedure to certify a set of floors. In experiments across 24 configurations and 11 benchmarks, StepCOPS achieves 96.4% selected‑policy coverage, raises the certified floor by 1.5 points over prior methods, stays 0.6 points below a large‑reference jury oracle, and abstains in 2.4% of trials.

By Ibne Farabi Shihab, Sanjeda Akter, Anuj Sharma
arXiv Computation and Language
Sep 25

A Training Criterion with Token-Level Tolerance to Transcription Ambiguity for Automatic Speech Recognition

The paper introduces a token‑level extension of Omni‑Temporal Classification (OTC) for automatic speech recognition, allowing unsupported tokens to be bypassed while preserving supervision for the rest of the word. Across 19 languages and three corpora, this token‑level OTC consistently outperforms standard CTC, achieving the lowest mean word error rate on every dataset and a 9.45% average relative WER reduction. A predictive‑entropy‑indexed schedule replaces epoch‑based relaxation, reducing training‑length dependence while maintaining performance.

By Saurabh Kumar, Diptiman Mohanta, Prasanta Kumar Ghosh
arXiv Computation and Language
Sep 25

ModularSQL: A Runtime Guardrail for the Multiplicity Blind Spot in Text-to-SQL

The paper introduces ModularSQL, a lightweight runtime guardrail designed to detect and correct multiplicity errors—such as missing DISTINCT clauses, inflated aggregates, and Cartesian join explosions—in Text-to-SQL systems. It highlights the Multiplicity Blind Spot (MBS), where standard set-based accuracy metrics fail to capture these errors, and proposes Multiset-EX as a more comprehensive evaluation criterion. Experiments on several models show that ModularSQL can improve multiplicity-aware accuracy while adding minimal computational overhead.

By Tianxin Zhou, Ruixi Lin
arXiv Computer Vision
Sep 25

Interpretable Similarity of Synthetic Image Utility

The paper introduces Interpretable Utility Similarity (IUS), a novel metric for quantifying how closely a synthetic image set matches a real image set in terms of usefulness for deep‑learning clinical decision support systems. IUS is interpretable, leveraging generalized neural additive models to explain why one synthetic dataset may outperform another based on clinically relevant image features. Experiments on color medical imaging modalities—endoscopic, dermoscopic, and fundus—show that selecting synthetic images with high IUS can boost classification performance by up to 54.6%, and the method also generalizes to grayscale X‑ray and ultrasound data.

By Panagiota Gatoula, George Dimas, Dimitris K. Iakovidis
arXiv Computer Vision
Sep 25

BARRIER: Bounded Activation Regions for Robust Information Erasure

BARRIER (Bounded Activation Regions for Robust Information Erasure) is a method for machine unlearning that confines parameter updates to a controlled activation space, allowing stronger erasure of targeted concepts while limiting collateral damage to other representations. By employing interval arithmetic, it derives a closed‑form bound on worst‑case representation changes in protected regions, which serves as a knowledge‑preservation objective. The approach is architecture‑agnostic, compatible with existing erasure objectives, and empirically shows competitive performance in both classification and generative tasks, with enhanced robustness against adversarial recovery attacks.

By Jan Miksa, Patryk Krukowski, Przemys{\l}aw Spurek, Dawid Damian Rymarczyk, Marcin Sendera
arXiv Computer Vision
Sep 25

OptiSAR-Net++: A Large-Scale Benchmark and Transformer-Free Framework for Cross-Domain Remote Sensing Visual Grounding

OptiSAR-Net++ introduces a new cross‑domain remote sensing visual grounding task (CD‑RSVG) and the first large‑scale benchmark dataset, OptSAR‑RSVG. The framework replaces Transformer decoding with a CLIP‑based contrastive approach, employing a patch‑level Low‑Rank Adaptation Mixture of Experts for efficient cross‑domain feature decoupling and a text‑guided dual‑gate fusion module for improved semantic‑visual alignment. Experiments show state‑of‑the‑art performance on OptSAR‑RSVG and DIOR‑RSVG, with notable gains in localization accuracy and computational efficiency.

By Xiaoyu Tang, Jun Dong, Jintao Cheng, Rui Fan
arXiv AI
Sep 25

Near-Oracle KV Selection via Pre-hoc Sparsity for Long-Context Inference

The paper introduces Pre-hoc Sparsity (PrHS), a method that selects key-value (KV) cache entries before attention scoring to avoid posterior bias in large language model inference. By bounding mutual‑information loss through the dropped attention mass, PrHS offers explicit accuracy control and implements three orthogonal selectors across time, depth, and layer. Experiments on LLaMA and Mistral models show that PrHS cuts retrieval overhead by over 90%, achieves higher sparsity than HShare, and delivers significant speedups and reduced FLOPs on NVIDIA A100 GPUs while maintaining near‑dense accuracy.

By Yifei Gao, Lei Wang, Rong-Cheng Tu, Qixin Zhang, Jun Cheng, Dacheng Tao
arXiv AI
Sep 25

A Multimodal 3D Foundation Model for Light Sheet Fluorescence Microscopy Enables Few-Shot Segmentation, Classification, and Deblurring

The paper presents a 3D foundation model for light sheet fluorescence microscopy (LSM) that is pretrained on a large curated set of 3D images from various organisms, stains, and imaging protocols. By jointly optimizing for masked reconstruction and image‑text alignment, the model learns transferable volumetric representations that dramatically reduce the need for annotated data. The pretrained backbone enables efficient few‑shot adaptation to downstream tasks such as segmentation, classification, and deblurring, consistently outperforming baselines according to standard metrics and expert evaluation.

By Adina Scheinfeld, Haotan Zhang, Shang Mu, Rudolf L. M. van Herten, Lucas Stoffl, Ali Erturk, Zhuhao Wu, Johannes C. Paetzold
arXiv AI
Sep 25

J-Zero: Unified Challenger--Solver--Judge Self-Evolution from Zero Data

J-Zero introduces a unified Challenger–Solver–Judge self‑evolution framework that operates without any initial data. The Challenger and Solver co‑evolve through adversarial task generation and response improvement, while the Judge adapts using known preference pairs derived from the Solver’s outputs rather than its own scores. Experiments show J‑Zero surpasses baselines by 4.2 points on verifiable tasks and 8.0 points on unverifiable tasks, maintaining improvement over ten iterations versus baseline degradation after two.

By Gyouk Chu, Myeongho Jeon, Teresa Yeo, Eunho Yang