AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,798 stories · RSS feed

arXiv Machine Learning
Oct 5

DIVINE: Simple Cross-Market Stock Pretraining via Diverse Indicator Reconstruction

DIVINE is a cross‑market pretraining framework for financial time‑series that reconstructs 77 targets derived from 16 standard technical indicators using raw OHLCV data. Trained jointly on six equity markets, its lightweight 0.05M‑parameter encoder outperforms larger pretraining baselines and matches or exceeds larger financial foundation models in downstream stock ranking tasks. The study demonstrates that diverse indicator supervision and cross‑market training yield complementary gains, highlighting that supervision design and market diversity, rather than model size, drive strong transferable financial representations.

By Kuan-Yu Chen, Shu-Cheng Zheng, Yu-Chen Den, Wei-Cheng Liao, Tien-Hao Chang
arXiv Machine Learning
Oct 5

Trade-off Functions for DP-SGD with Subsampling based on Random Allocation: Tight Upper and Lower Bounds

arXiv:2605. 06259v3 Announce Type: replace Abstract: Within the $f$-DP framework, we derive a tight analysis of the trade-off function for Differentially Private Stochastic Gradient Descent (DP-SGD) with subsampling based on random allocation in which each sample is independently assigned to exactly one of $M$ minibatches per epoch, each minibatch corresponding to one of the $M$ SGD rounds within a single epoch.

By Marten van Dijk, Murat Bilgehan Ertan
arXiv Machine Learning
Oct 5

Uncovering EEG Patterns Consistently Associated with Cybersickness Discomfort Using Deep Learning Interpretability Maps

The study identifies specific EEG spatio‑temporal features that reliably indicate cybersickness discomfort. Using deep learning interpretability maps on two auditory ERP datasets, the authors found consistent importance of left frontal, midline central, and right parietal scalp locations, particularly between 80–260 ms after stimulus. These findings suggest that such EEG markers can be used for future discomfort classification in VR settings.

By Jacqueline Yau, Evan G. Center, Pawel Augustynowicz, Steven M. LaValle, Timo Ojala, Wenzhen Yuan, Nancy M. Amato, Pawel Strozak, Minje Kim, Kara D. Federmeier, Katherine J. Mimnaugh
arXiv Computation and Language
Oct 5

Collective Bias Mitigation via Model Routing and Collaboration

The paper introduces Collective Bias Mitigation (CBM), a framework that reduces bias in large language models by learning fine‑grained model behavior and enabling knowledge sharing among diverse LLMs. CBM systematically explores how to select and organize distinct models to produce fairer responses, outperforming standalone baselines. Experiments demonstrate significant bias reduction, with the Committee topology balancing effectiveness and inference cost.

By Mingzhe Du, Luu Anh Tuan, Xiaobao Wu, Yichong Huang, Yue Liu, Dong Huang, Huijun Liu, Bin Ji, Jie M. Zhang, See-Kiong Ng
arXiv Computation and Language
Oct 5

Single-Pass Uncertainty Heads for Claim-Level Hallucination Detection in Persian Medical Language Models

The paper introduces a single‑pass uncertainty head for detecting hallucinations in Persian medical language models. By adapting the LLM Uncertainty Head framework to Aya‑Expanse‑8B and two Gaokerena backbones, the authors build claim‑level hallucination datasets and train lightweight heads that achieve PR‑AUCs of 0.4820 and 0.4652 and ROC‑AUCs of 0.7852 and 0.7810. These heads operate without retrieval or repeated sampling during inference, offering a cost‑effective approach to hallucination detection in Persian medical LLMs.

By Mehrdad Ghassabi, Pedram Rostami, Hamidreza Baradaran Kashani, Sadra Hakim, Audrina Ebrahimi
arXiv Computer Vision
Oct 5

Parasitic Co-Denoising: Unlocking 3D Human Motion Generation in a Frozen Video Diffusion Model

The paper demonstrates that large text-to-video diffusion models contain implicit 3D motion information in their intermediate denoising states. By introducing parasitic co-denoising, the authors extract motion directly from the frozen model using a Parasitic Motion Decoder (PMD) that shares the noise schedule and fuses intermediate features. This approach yields efficient, aligned 3D motion generation without training a separate motion model, achieving comparable results to dedicated generators with far fewer parameters.

By Yunjiao Zhou, Junlang Qian, Lihua Xie, Jianfei Yang
arXiv Computer Vision
Oct 5

A Benchmark for Spatially Grounded Gesture Generation

The paper introduces a benchmark for spatially grounded gesture generation, featuring about 2,000 pointing-annotated clips from naturalistic VR dialogue with ground-truth 3D referents. It defines a task where systems must decide when, how, and where to point within conversational speech, and proposes a protocol that separates temporal alignment, spatial grounding, and perceived naturalness. A flow‑matching baseline (MM‑Conv‑Flow) and a retrieval‑based system are evaluated against captured human motion, revealing that geometric grounding can surpass human pointing quality without improving perceived naturalness, underscoring the need to measure referential gesture quality along distinct dimensions.

By Anna Deichler, Rishabh Dabral, Fethiye Irmak Dogan, Anindita Ghosh, Jonas Beskow
arXiv Computer Vision
Oct 5

VDOT++: Unified Few-Step Video Generation via Unbalanced Optimal Transport Distillation

The paper introduces VDOT++, a unified distillation framework for few-step video generation across text-to-video, image-to-video, and condition-based tasks. It extends the previous VDOT method by incorporating an asymmetric unbalanced optimal transport distillation that allows student tokens to carry less mass, and an α1 ground cost that uses a weighted median to preserve modes. The approach combines distribution matching with adversarial refinement and cross-scale distillation, achieving competitive performance on multiple video benchmarks with only four sampling steps.

By Yutong Wang, Xingtong Ge, Enhuai Liu, Yunke Wang, Tianfan Xue, Yu Qiao, Yaohui Wang, Xinyuan Chen, Chang Xu
arXiv Computer Vision
Oct 5

DuoMatching: Joint-Marginal Distribution Matching for Few-Step Video Generation

DuoMatching is a new framework for few-step video generation that extends distribution matching distillation by adding a joint‑marginal formulation. It introduces a marginal matching objective that uses an image generator to provide frame‑level supervision, and employs LatentBridge to align latent representations between the video student and image teacher. Latent Variation Sampling distributes this supervision across temporal segments, improving visual quality, composition, and semantic alignment while preserving motion dynamics, with human evaluations favoring it over baselines.

By Jiahao Zhan, Yan Wang, Yongrui Ma, Qunliang Xing, Ruchang Yao, Runtao Liu, Shijie Zhao, Tianfan Xue
arXiv AI
Oct 5

Lightweight, Rubric-Guided Trajectory Evaluation for Production AI Agents

LiteTrajEval is a lightweight architecture designed for budget‑bounded trajectory evaluation of large‑language‑model agents. It creates compact domain‑specific rule profiles offline, preprocesses trajectories online to flag heuristic failure signals, and then uses a single rubric‑guided LLM judge to generate structured diagnostic reports. On public Magnetic‑One‑style and τ‑bench‑style datasets, LiteTrajEval improves failure‑localization alignment with human annotations by 20–35 percentage points and reduces cost and evaluation time by 6× and 8× respectively, and has been deployed in an enterprise agentic platform.

By Linh-An Phan, MingXue Wang, Guangyu Wu, Feng Pan, Zhaoyu Pang, Yanbin Zhang
arXiv Machine Learning
Oct 5

Evaluating and Improving the Robustness of Large Language Models to Input Sequence Variations

This thesis introduces methods for assessing and enhancing the robustness of large language models (LLMs) against adversarial input variations. It proposes a generative robustness metric, R_stab(f), based on Jensen‑Shannon divergence, and proves bounds for localized attacks. The work also presents adaptive attacks, Trojan detection techniques, defense strategies for multi‑layer systems, and lightweight attestation for Model Context Protocol (MCP) agentic systems, all implemented in the JudgeGuard, TrojanArmor, and MCPSec suites.

By Narek Maloyan
arXiv Machine Learning
Oct 5

High-Dimensional Asymptotics and Dataset Selection for Private Transfer Learning

The paper addresses the challenge of selecting external datasets for private transfer learning by modeling high‑dimensional regression with heterogeneous sources and a weighted ridge estimator. It relies solely on aggregated statistics and offers privacy guarantees under $ ho$‑zero‑concentrated differential privacy for labels or both features and labels. A deterministic equivalent of test error is derived, enabling optimization of hyperparameters and decision‑making about the utility of private external data without accessing individual records.

By Filip Kova\v{c}evi\'c, Edwige Cyffers, Stefano Sarao Mannelli, Marco Mondelli
arXiv AI
Oct 5

Jumping the Line: Exploiting Length Predictions in LLM Scheduling

The paper introduces JIL, an attack that manipulates length predictions used by large language model (LLM) schedulers to gain higher priority and reduce completion time. By optimizing an adversarial suffix, JIL causes lightweight output-length probes to underestimate request lengths, leading to up to an 83.4 % reduction in predicted length and an average 1.53× faster completion in end‑to‑end serving experiments. The study also evaluates defenses, showing that grouping predictions into coarse intervals can mitigate JIL’s advantage and protect benign requests.

By Yuyang Dai, Rana Shahout, Mahmood Sharif