AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,788 stories · RSS feed

arXiv Computer Vision
Oct 9

LLM-powered Query Expansion for Enhancing Boundary Prediction in Language-driven Action Localization

The paper introduces a method that uses large language models (LLMs) to expand language queries with detailed textual descriptions of action start and end boundaries, providing richer cues for video action localization. It also proposes a training strategy that models boundary probability scores by combining semantic similarity between frames and the expanded query with temporal distance to annotated boundaries, thereby offering more consistent supervision. The approach is model‑agnostic and can be integrated into existing language‑driven action localization systems, with experiments on multiple datasets showing its effectiveness.

By Zirui Shang, Xinxiao Wu, Shuo Yang
arXiv Machine Learning
Oct 9

Spectrally Targeted Muon

The paper introduces Spectrally Targeted Muon, an optimizer that orthogonalizes only singular values of update matrices above or below a threshold τ, thereby interpolating between normalized SGD and the full Muon optimizer. It uses Newton‑Schulz iteration on a shifted Gram matrix to avoid SVD, and evaluates the method on CIFAR‑10 and NanoGPT, tracking effective rank and a new alignment metric. The study finds that orthogonalizing small singular values is crucial, that shrinking large singular values keeps the parameter spectrum flat, and that AdamW differs mainly in the speed of structure formation compared to Muon.

By Vishrut Goyal, Rohan Ramkumar
arXiv AI
Oct 9

Prior or Feedback? What an LLM Uses When Adapting Neural Operators

The study investigates whether large language models (LLMs) rely solely on their initial task context or adapt based on experimental feedback when selecting fine‑tuning configurations for neural operators. Experiments across partial differential equation families show that LLMs outperform random search and Bayesian optimisation, with early configurations already near the top of a random‑search pool and subsequent proposals changing in response to validation scores. Controlled interventions confirm that the LLM combines a task‑dependent prior with sensitivity to observed outcomes.

By Julian Chan, Javier Mora Jimenez
arXiv Computer Vision
Oct 9

Connected Self Forcing: Beyond Local Learning in Video Autoregression

The paper introduces Connected Self Forcing, a training framework that reconnects gradient paths across autoregressive video chunks to improve long‑horizon visual quality and temporal consistency. It allows feedback from later predictions to influence earlier context generation, using shortcut gradient replay to keep memory usage manageable. The method is combined with distribution‑matching distillation and shows experimental gains without altering the inference procedure.

By Dongbin Zhang, Chaoda Zheng, Kangjie Chen, Xiangyu Li, Shijia Chen, Jinhao Deng, Yuqi Zhang, Guangfeng Jiang, Hongbin Lin, Choo Sin Wai, Minqi Wang, Puyi Wang, Jingye Zhang, Yu Zhang, Xianming Liu, Boyang Wang
arXiv Computation and Language
Oct 9

Black-Box Forensics for Conversational LLM Agents

The paper presents methods for black-box forensic analysis of conversational large language model (LLM) agents, focusing on two tasks: attribution—identifying the base model powering a chatbot—and fingerprinting—detecting identical system prompts across different endpoints. Attribution classifiers achieve 98% accuracy using only a few non‑adversarial conversation turns, while a cross‑encoder fingerprinting approach attains an AUC of 0.768 and an F1 of 0.703 on unseen prompts, improving to 0.943 AUC when aggregating 50 interactions. These techniques enable investigators to trace AI‑enabled scams back to their underlying models and link related scams through shared system prompts.

By Isadora White, Yasaman Jafari, Taylor Berg-Kirkpatrick
arXiv Computation and Language
Oct 9

A Formative Study of Brief Affective Text as a Complement to Wearable Sensing for Longitudinal Student Health Monitoring

The study examined whether ultra‑brief naturalistic concern text could complement wearable sensor data for monitoring student health. Over a year, 458 university students provided three‑word responses to a bi‑monthly prompt while wearing Oura rings, and the researchers compared different NLP methods to link these brief texts to nine sleep and physical activity outcomes. Findings indicated that weeks with academic‑concern framing correlated with lower physical activity, while emotional exhaustion language correlated with poorer sleep quality and lower heart rate variability, suggesting that affective content may add value to passive physiological monitoring.

By Tamunotonye Harry, Johanna Hidalgo, Matthew Price, Yuanyuan Feng, Kathryn Stanton, Connie Tompkins, Peter Sheridan Dodds, Mikaela Irene Fudolig, Laura Bloomfield, Christopher Danforth
arXiv Computation and Language
Oct 9

FA-Bench: A Benchmark for Phone- and Word-Level Timestamp Accuracy in Forced Alignment and ASR on Clean and Noisy Speech

FA-Bench is an open framework that standardizes forced alignment evaluation by fixing transcript normalization, data splits, phone mapping, and scoring procedures. It offers two tracks: one with reference transcripts and another with recognizer outputs, evaluated on clean and four degraded audio conditions using 21 open models and 9 commercial APIs. The benchmark scores every utterance boundary with a tolerance‑based F1 metric, revealing systematic timing biases such as Whisper’s 150 ms early alignment and commercial APIs’ 50 ms late alignment.

By Wei Chu, Yuanzhe Dong, Ke Tan, Dong Han, Yichao Zhou, Ruchao Fan, Bingshen Mu, Jingbei Li, Vishwas Shetty, Sarthak Bisht, Ziyue Qiu, Massa Baali, Rita Singh, Bhisha Raj
arXiv Computation and Language
Oct 9

PlurVA-LLM-2026 Shared Task Track-1: Pluralistic Value Alignment in LLMs via Multilingual Fine-Tuning and Threshold Calibration

The paper describes a system for the PlurVA-LLM 2026 Shared Task Track‑1, which aims at pluralistic value alignment in China, Indonesia, and Sri Lanka. The authors fine‑tuned Llama 3.1 8B Instruct with 4‑bit QLoRA and applied data‑specific techniques: option‑permutation augmentation for Chinese data, annotator vote expansion for Indonesian data, and binary reformulation with SinhalaMMLU augmentation plus conditional threshold calibration for Sri Lankan data. The resulting model achieved macro‑average accuracy of 0.805, with per‑language accuracies of 0.785 (Chinese), 0.715 (Indonesian), and 0.916 (Sri Lankan).

By Vihindi Kotalawala, Nevidu Jayatilleke
arXiv Computation and Language
Oct 9

Beyond Speech Captions: Speech-Rewarded Style Planning for Conversational Text-to-Speech

The paper introduces Speech-Rewarded Style Planning (SRSP), a method that trains a text-based style planner using a frozen downstream TTS model. SRSP generates candidate style instructions from dialogue history and response text, optimizing them with group-relative policy optimization (GRPO) where the reward is the teacher‑forced likelihood of target speech tokens. Experiments on an English subset of the ISCSLP 2026 CoT‑TTS corpus show that SRSP outperforms baseline LLM and target‑audio‑informed captioning approaches in speech‑style and emotion similarity, mel‑cepstral distortion, and contextual appropriateness. "whyItMatters":"The study demonstrates a more effective way to align natural‑language style descriptions with actual acoustic output, improving controllability and fidelity in conversational text‑to‑speech systems."

By Shiao Zhu, Lianbo Liu, Sizhen Lyu, Yuzhe Wang, Sheng Li, Takahiro Shinozaki
arXiv Computation and Language
Oct 9

SAGE: Sink-Aware Guided Emphasis for Visual Grounding in Vision-Language Decoders

The paper introduces SAGE, a lightweight method that guides decoder attention in vision-language models away from prompt‑invariant attention sinks (PIS) toward query‑dependent regions of interest (ROIs). By using token‑aligned ROI masks from vision backbones like CLIP, ViT, and DINOv3, SAGE improves visual grounding and reduces hallucinations across various VLM families. Experiments show consistent gains on public vision‑language benchmarks, especially in fine‑grained visual discrimination tasks where localized evidence is essential.

By Jeonghyo Song, YoungJoon Yoo
arXiv Computation and Language
Oct 9

Quantifying Retriever-Generator Alignment in RAG with Local Explanations

The paper introduces RAG-E, an explainability framework for Retrieval-Augmented Generation (RAG) systems that quantifies how well a language model’s document usage aligns with the retriever’s rankings. It adapts Integrated Gradients for retriever analysis, uses a Monte Carlo‑stabilized Shapley Value approximation for generator attribution, and proposes the Weighted Alignment between Retriever and Generator (WARG) metric. Experiments on PopQA, QAMPARI, and TREC CAST show that generators often ignore top‑ranked documents, and WARG outperforms Pearson and Spearman correlations in measuring alignment, offering a practical tool for auditing RAG performance.

By Korbinian Randl, Guido Rocchietti, Aron Henriksson, Ziawasch Abedjan, Tony Lindgren, John Pavlopoulos
arXiv Computation and Language
Oct 9

Limited Stereotype Control Through Routing Reweighting in MoE Language Models

The paper introduces Fairness-Aware Routing Equilibrium (FARE), a diagnostic framework that examines how demographic prompts are routed differently from neutral prompts in Mixture-of-Experts (MoE) language models and applies fixed inference-time reweighting to attempt stereotype control. Five MoE architectures were evaluated in English, and the reweighting procedure produced only marginal changes in stereotype preference—at most 1.3 percentage points—and did not consistently affect toxicity, lexical, or reference-overlap metrics across four generation protocols. The results indicate that the tested reweighting method offers limited effectiveness in controlling stereotypes within these models.

By Junhyeok Lee, Han Jang, Kyu Sung Choi
arXiv Computation and Language
Oct 9

CiteVQA: Benchmarking Evidence Attribution for Trustworthy Document Intelligence

CiteVQA is a benchmark that evaluates multimodal large language models on both answering document questions and providing element‑level bounding‑box citations for the evidence used. It contains 1,897 questions across 711 PDFs in seven domains and two languages, with ground‑truth citations generated via an automated pipeline and strict quality control. The evaluation metric, Strict Attributed Accuracy (SAA), rewards a prediction only when both the answer and the cited region are correct, revealing widespread attribution hallucination in current models.

By Dongsheng Ma, Jiayu Li, Zhengren Wang, Yijie Wang, Jiahao Kong, Weijun Zeng, Jutao Xiao, Jie Yang, Bangrui Xu, Yuhan Wang, Bin Wang, Conghui He
arXiv Computer Vision
Oct 9

FearCaut-Qwen: Affective Steering in a Vision-Language Model Shifts the Decision Criterion for Hazard Assessment

The paper investigates why vision‑language models (VLMs) often miss hazards, attributing this to an overly conservative decision criterion rather than poor perception. By identifying and steering an affective ‘fear’ circuit within the model, the authors shift the decision threshold, dramatically increasing hazard recall from 27.0 % to 75.7 % without retraining. The study demonstrates that affect‑based inference-time manipulation can correct VLM decision biases in practical applications.

By Xiaoshan Zhou
arXiv Computer Vision
Oct 9

Fluid-Gen-Zero: Grounding Pretrained Video Generators in Physics without Training

Fluid-Gen-Zero is a training-free framework that generates physics-aware fluid-object interaction videos by separating physical reasoning from appearance synthesis. It uses a physics simulator for motion dynamics and a pretrained video generator for visual quality, coordinated through a two-level agentic workflow involving a vision-language model for planning and latent-space guidance for simulation signals. The method improves simulation alignment and fluid flow accuracy across multiple backbones while maintaining perceptual quality, as shown by quantitative metrics and human preference studies.

By Hong Huang, Yuqiu Liu, Chenyu You, Daniel Martin, Chuhang Zou, Wuyang Chen
arXiv AI
Oct 9

Beyond Visual Enhancement: Adaptive Multi-Context Steering to Mitigate LVLM Hallucinations

The paper introduces AIMS (Adaptive Information Multi-source Steering), a lightweight, training‑free framework that dynamically balances visual, prefilled textual, and generated contexts during decoding to reduce hallucinations in Large Vision‑Language Models (LVLMs). By constructing compact prototypes for each context domain and estimating their affinities with the current query, AIMS assigns head‑wise steering weights that guide adaptive context integration without extra training or forward passes. Experiments across multiple LVLMs and decoding strategies show that AIMS effectively mitigates object hallucination while preserving general multimodal performance.

By Shuran Ma, JiaLe Li, Yuxin Dong, Shan Zheng, Qingyun Jiang, Xiang Chen, Qi Zhu, Deyi Ji, Yifan Yang, Jianfeng Pan, Yu Tian, Xue Yang
arXiv AI
Oct 9

From Surface to Depth: Towards Cognitive Appraisal Reasoning in Multimodal Emotion Understanding

The paper introduces a new paradigm for multimodal emotion understanding that moves beyond surface-level cue-label associations by incorporating cognitive appraisal reasoning. It presents CogEmo-40K, a large instruction‑tuning dataset that elicits evidence‑grounded reasoning across six appraisal dimensions; CogEmo‑MoE, a compact sparse multimodal large language model with interleaved MoE blocks for appraisal‑specific adaptation; and CogEmo‑Bench, a benchmark featuring an Appraisal Evidence Quality Score (AEQS) to evaluate why emotions arise. Experiments demonstrate that this perception‑to‑appraisal approach improves cross‑domain generalization and aligns model reasoning more closely with human cognitive processes.

By Jia Li, Yichao He, Yangchen Yu, Qiankun Li, Xinyi Li, Baiyi Ye, Zhenzhen Hu, Richang Hong, Erik Cambria
arXiv AI
Oct 9

Neural Network Verification for Deep Joint Source-Channel Coding

The paper introduces the first bound‑propagation framework for verifying the decoder of Deep Joint Source‑Channel Coding (DeepJSCC), providing worst‑case reconstruction error guarantees over a specified wireless channel noise region. It extends existing DNN verification techniques to handle PReLU activations, transposed convolutions, and Rayleigh fading, and incorporates Lipschitz‑regularized global robustness training (GloRo) to tighten certification bounds. Experiments on image transmission models show significant improvements in certified safety and real‑world OFDM validation confirms the theoretical guarantees.

By Thanh Le, Hai Duong, Takeshi Matsumura, ThanhVu Nguyen
arXiv AI
Oct 9

The Polytopal Neural Network

The paper introduces Polytopal Neural Networks (PNNs), a framework that enforces a polytope-based structure on neural network representations to extract distinct layer-wise aspects. By using learned corpus representations and an amortized simplex inference procedure, PNNs preserve meaningful latent structures with minimal performance loss and yield compressed representations comparable to vector‑quantized (VQ) methods. The authors demonstrate that PNNs provide a new, interpretable approach to VQ deep learning training while maintaining competitive performance.

By A. Emilie J. Wedenborg, Anders V. N{\o}rskov, Teresa Dorszewski, Kristoffer Wickstr{\o}m, Morten M{\o}rup
arXiv Computer Vision
Oct 9

Boosting the Local Invariance for Better Adversarial Transferability

The paper introduces LI-Boost, a technique that enhances the local invariance of adversarial perturbations to improve their transferability across models. By demonstrating a positive correlation between local invariance and transferability, the authors show that LI-Boost significantly boosts five categories of transfer-based attacks on ImageNet, including CNNs, ViTs, defense mechanisms, commercial APIs, and vision-language models. Extensive experiments confirm the method’s effectiveness across diverse architectures and attack types.

By Bohan Liu, Xiaosen Wang