AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,808 stories · RSS feed

arXiv AI
Sep 17

When AI Generates Covariates: Causal Typing and Estimand Drift in Sequential Experiments

The paper introduces a causal type discipline for sequential experiments that use AI-generated covariates. It defines a framework—including a versioned representation map, causal role classifier, claim-status filter, and estimand lock—to ensure that generated features are correctly classified as treatments, mediators, outcomes, or other roles, thereby preserving the intended causal estimand. The authors apply this framework to analyze compression bias, mediator adjustment, leakage, and other issues, demonstrating through simulations that careful refinement of generated covariates can reduce bias while design erasure or improper selection can lead to bias or undercoverage.

By Takes Fujita (VRI), Nobutaka Hattori (Department of Neurology, Juntendo University School of Medicine)
arXiv AI
Sep 17

CPR: Combining global composing, local performing and full-sequence refining in piano rendering with continuous autoregressive modelling

The paper introduces CPR, a piano rendering framework that combines continuous autoregressive modeling with local flow matching and full‑sequence refinement. It predicts continuous hidden states, generates 24 kHz acoustic latents, and upsamples to 48 kHz, while new techniques BREPA and MT‑RoPE enhance musical semantics and cross‑modal alignment.

By Chong Jing, Junan Zhang, Zhizheng Wu
arXiv Machine Learning
Sep 17

LightSleepX: A Lightweight, Inception-Based Dual-Modal Network for Sleep Staging

LightSleepX is a lightweight, inception‑based dual‑modal network designed for sleep staging in resource‑constrained environments. It uses depthwise separable convolutions, multi‑scale enhanced attention for efficient EEG/EOG feature extraction, and a Mamba encoder for long‑range temporal modeling. On public benchmarks, it achieves 85.9% accuracy on Sleep‑EDF‑20 and 81.8% on ISRUC‑S3 with only 0.049M parameters and 195.9 MFLOPs.

By Yi Wang
arXiv Machine Learning
Sep 17

Uncertainty measurement for complex event prediction in safety-critical systems

The paper presents a machine‑learning approach (ML_CP) that automatically learns patterns and rules for complex event prediction, reducing reliance on manual rule creation. It incorporates sensitivity analysis to assess how output varies with each input and uses conformal prediction to generate uncertainty‑aware prediction intervals. Experiments on binary, multi‑level classification, and regression tasks show promising results for safety‑critical embedded systems.

By Maria J. P. Peixoto, Akramul Azim
arXiv Machine Learning
Sep 17

Consensus-based optimization for closed-box adversarial attacks and a connection to evolution strategies

The paper investigates Consensus-based Optimization (CBO) as a gradient‑free method for closed‑box adversarial attacks, where imperceptible perturbations fool a classifier without gradient access. It establishes a theoretical link between CBO’s consensus hopping and natural evolution strategies (NES), and relates both to gradient‑based optimization. Experiments demonstrate that CBO can outperform NES and other evolutionary strategies in some scenarios.

By Tim Roith, Leon Bungert, Philipp Wacker
arXiv Computation and Language
Sep 17

Register Bias in Complexity-Based Large Language Model Routing

The paper examines how large language model (LLM) services route queries to models of varying size based on a cheap complexity estimate. It finds that this routing is not register neutral: queries written in non‑standard English registers (e.g., African American English or second‑language English) are systematically assigned to lower‑capacity models because they appear shorter due to omitted function words. Experiments on 37,704 learner sentence pairs and a controlled corpus show that this bias leads to significantly lower accuracy across all model tiers, including the highest‑capacity cloud models, while the routing decision itself adds little marginal cost.

By Simran Koul
arXiv Computer Vision
Sep 17

DualCount: Structurally Consistent Density and Point Modeling for Zero-Shot Object Counting

DualCount introduces an instance-aware dual-decoder framework that couples density and point representations for zero‑shot object counting. By treating density estimation as a structured mass allocation over latent object instances, it applies two geometric constraints—per‑instance mass conservation and center‑of‑mass alignment—to enforce instance‑level consistency. Experiments on FSC‑147, PUCPR+, and CARPK demonstrate that this approach consistently reduces counting error and achieves new state‑of‑the‑art performance.

By Xuan Cuong Ngo
arXiv Computation and Language
Sep 17

Divide and Conquer: A Hybrid Strategy Defeats Multimodal Large Language Models

The paper introduces JMLLM, a multimodal jailbreaking approach that targets text, visual, and auditory inputs to expose vulnerabilities in large language models. It also presents TriJail, a new dataset containing jailbreak prompts across all three modalities. Experiments on TriJail and AdvBench show higher attack success rates and lower time overhead compared to existing methods.

By Yanxu Mao, Peipei Liu, Tiehan Cui, Zhaoteng Yan, Congying Liu, Datao You
arXiv Computation and Language
Sep 17

Modelling Adjectival Modification Effects on Semantic Plausibility

The paper investigates how adjectival modifiers affect the semantic plausibility of events, using the Adept benchmark of 16,000 English sentence pairs that differ by a single adjective. Experiments show that sentence transformers, despite being conceptually suited to the task, underperform compared to models like RoBERTa. The authors provide an error analysis and discuss the implications of their findings for future work on balancing training and test data.

By Anna Golub, Beate Zywietz, Annerose Eichel
arXiv Computer Vision
Sep 17

Generalizable Face Forgery Detection via Separable Prompt Learning

The paper introduces Separable Prompt Learning (SePL), a method that enhances face forgery detection by leveraging CLIP’s textual encoder through two separate learnable prompts. SePL incorporates a cross-modality alignment strategy and specific objectives to distill forgery knowledge from CLIP. Experiments show that SePL outperforms existing approaches in cross-dataset and cross-method evaluations.

By Enrui Yang, Baoyuan Wu, Yuezun Li
arXiv Machine Learning
Sep 17

VLA-ULAP: Interleaving Cloud VLA Calls with Ultra-Lightweight Local Action Prediction at the Edge

The paper introduces VLA-ULAP, a lightweight local action predictor that interleaves remote vision–language–action (VLA) calls with on‑edge inference. ULAP, with only 7.4 M parameters, predicts action chunks in a single pass using current views, proprioception, and action history, eliminating the need for VLA hidden states or server round‑trips. Experiments on Jetson Orin Nano and simulated benchmarks show that VLA-ULAP can remove 48.8–76.7 % of VLA calls while preserving 95–97.5 % of baseline success, and it outperforms local VLA‑acceleration alternatives in both inference time and energy consumption.

By Deyu Cao, Ryuji Oi, Kosuke Matsushima, Yuxuan Pan, Ziheng Wang, Daichi Fujiki, Atsutake Kosuge
arXiv Machine Learning
Sep 17

Sparse Bayesian Modeling of EEG Channel Interactions Improves P300 Brain-Computer Interface Performance

The paper introduces a sparse Bayesian time‑varying regression framework that models pairwise EEG channel interactions and performs temporal feature selection for P300 brain‑computer interfaces. Using a relaxed‑thresholded Gaussian process prior, the method achieves a median character‑level accuracy of 96.4% on a public P300 speller dataset and outperforms both statistical and deep learning baselines. It also yields subgroup‑specific gains, notably for participants who abstain from alcohol, and improves median BCI‑Utility by over 10%, reaching peak throughput after only six sequence repetitions.

By Guoxuan Ma, Yuan Zhong, Moyan Li, Yuxiao Nie, Jian Kang
arXiv Computation and Language
Sep 17

A Benchmark Suite and Ground-Truth Methodology for Formal Verification of IEC 61131-3 Ladder Diagram Programs

The paper introduces the first benchmark suite for formal verification of PLC programs, covering both Structured Text and Ladder Diagram encodings of IEC 61131‑3. It contains 50 programs in 83 variants across ten industrial domains, each paired with a formal property, a machine‑checkable expected verdict, and a violation witness in SV‑COMP format. The suite’s ground‑truth methodology uses construction, fault injection, and cross‑tool consensus to ensure reliable verdicts, and the authors validate it with ESBMC and nuXmv, revealing format and semantics fragmentation in existing tools.

By Pierre Dantas, Lucas Cordeiro, Waldir Junior
arXiv Computer Vision
Sep 17

Lumen: Parameter-Efficient Alignment of Pretrained Vision and Language Encoders for Zero-Shot Computational Pathology

Lumen is a pathology vision‑language model that aligns frozen unimodal foundation models (Virchow2 and BioMedBERT) using rank‑4 adapters and projection heads, training only 0.40% of the total parameters on the QUILT‑1M corpus. It achieves the highest mean chance‑corrected balanced accuracy (0.546) across nine zero‑shot patch benchmarks and demonstrates strong performance on lymph‑node metastasis detection, with AUROC scores of 0.964 internally and 0.955 externally. While it ranks third in cross‑modal retrieval, Lumen’s low‑parameter training yields competitive results at both patch and slide levels.

By Kiarash Tajbakhsh, Abdelrahman Faqieh, Michael Jopiti, Javier Garcia-Baroja, Philipp Zens, Branislav Zagrapan, Yuri Tolkach, Martin D. Berger, Aurel Perren, Bastian Dislich, Inti Zlobec, Amjad Khan
arXiv Computer Vision
Sep 17

Sim-to-Real Traffic Scene Understanding by Decoupling Semantics from Caption Generation with V-JEPA

The paper presents a decoupled framework for sim-to-real traffic scene understanding, separating semantic fact extraction from caption generation. It uses a frozen V-JEPA encoder for predictive scene representations and a lightweight Llama-based predictor for VQA, followed by a training-free structured refinement that leverages statistical priors, inter-question relationships, and temporal consistency. The refined facts are then fed to Qwen3-VL-8B to produce pedestrian and vehicle descriptions, achieving top performance on the 2026 AI City Challenge Track 2 benchmark with 87.09% VQA accuracy and an overall S2 score of 60.0853.

By Nguyen Hoai Thuong Bui, Thanh Nguyen Vo, Trinh Tra Giang Nguyen, Ha Duc Bui
arXiv AI
Sep 17

GVD: Governed Versioning and Deduplication for Document Repositories

GVD (Governed Versioning and Deduplication) is a framework that unifies cross‑document version linking with rule‑level conflict resolution under an auditable update policy. It assigns incoming documents to version families via bidirectional rule alignment, then compares their rules against family memory to detect duplicates, contradictions, asymmetric refinements, and new knowledge, using Counterfactual Span Probing (CSP) to correct neutral misclassifications. In a test on 120 enterprise documents across 59 version families, GVD achieved an F1 of 0.97 for version‑family construction and 0.94 for rule‑level consistency, with CSP improving rule consistency from 0.90 to 0.94.

By Mohammadreza Sediqin, Shivali Dalmia, Sumukha Thoppanahalli, Abhishek Mukherji
arXiv AI
Sep 17

FairCompressAgent: An Agentic Framework for Fairness-Aware Model Compression for FPGA Deployment

FairCompressAgent (FCA) is an agentic framework that unifies fairness-aware pruning, incremental quantization, and sparse low‑rank factorization for FPGA deployment. A language‑model planner selects compression configurations based on model profiles and measured outcomes, while an execution layer handles compression, fine‑tuning, evaluation, and constraint‑based selection. Experiments on Fitzpatrick‑17k with VGG‑11 show FCA can reduce inference tensor storage by 59.54% under accuracy constraints, improve validation average precision, and lower equalized opportunity, achieving similar results to one‑shot planning with fewer candidate evaluations.

By Yuanbo Guo, Yiyu Shi
arXiv AI
Sep 17

Where Grokking Happens: Distributed Utility and Fourier Recoding Without a Module Switch

The paper investigates where the transition from memorization to generalization occurs within a Transformer. Using Transition Games—activation games with paired non-generalizing controls—it finds that distributed utility gain is driven by a block‑0 attention bias, with degree‑two modes explaining 67–92% of the contrast across replacement games. A detailed path study shows that block‑1 MLP mediates more of this effect than other downstream paths, contradicting the common "MLP memorizes, attention generalizes" hypothesis and suggesting that grokking is a spectral recoding of an existing distributed circuit rather than a module switch.

By Dekun Yang
arXiv AI
Sep 17

Lexara-RF: Reference-Free Metrics for Evaluating Conversational Visual Analytics Agents

Lexara-RF introduces reference‑free metrics for evaluating conversational visual analytics agents that generate visualizations and natural‑language explanations. The framework uses only the prompt, data, and model response to score outputs, applying 13 metrics derived from visualization design theory and Gricean principles as consistency, intent‑alignment, and design validity checks. In tests against a human‑rated corpus, Lexara‑RF matches reference‑based methods, outperforms surface‑similarity NLG baselines, and accurately identifies structurally grounded failures.

By Srishti Palani, Vidya Setlur