AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,884 stories · RSS feed

arXiv AI
Aug 28

J-Zero: Unified Challenger--Solver--Judge Co-Evolution from Zero Data

J-Zero introduces a unified Challenger–Solver–Judge co‑evolution framework that enables self‑improvement of language models without requiring external supervision. The Challenger generates increasingly difficult tasks, the Solver learns to produce better responses, and the Judge adapts using preference pairs derived from the Solver’s own outputs rather than from its own scores. Experiments show J‑Zero outperforms baselines by an average of 4.2 points on verifiable tasks and 8.0 points on unverifiable tasks, maintaining improvement over ten iterations while baselines degrade after two.

By Gyouk Chu, Myeongho Jeon, Eunho Yang
arXiv Machine Learning
Aug 28

LLM Analysis of 150+ years of German Parliamentary Debates on Migration Reveals Shift from Post-War Solidarity to Anti-Solidarity in the Last Decade

The paper evaluates large language models (LLMs) for annotating German parliamentary debates on migration, achieving macro‑F1 scores comparable to human agreement, particularly with GPT‑5 and gpt‑oss‑120B. It combines soft‑label outputs with Design‑based Supervised Learning to mitigate systematic bias and applies the method to a 150‑plus‑year corpus, revealing high solidarity post‑war and a sharp rise in anti‑solidarity since 2015. The study demonstrates that LLMs can enable large‑scale social‑scientific analysis while highlighting the need for rigorous validation and bias correction.

By Aida Kostikova, Ole P\"utz, Steffen Eger, Olga Sabelfeld, Benjamin Paassen
arXiv Computation and Language
Aug 28

STAR : Sentence Translation Alignment Rate for Document-to-Document Machine Translation

The paper introduces STAR, a metric that measures sentence-level alignment between source and target documents in document-to-document machine translation. Using STAR, the authors develop StarPO, a preference‑optimization framework that ranks translation hypotheses by structural quality and applies a dynamic alignment mask to focus learning on misaligned segments. Experiments on news and literary data show that StarPO improves both translation quality and structural integrity, enabling small models to outperform large proprietary systems such as GPT‑4o while remaining more token‑efficient.

By Yichen Dong, Hao Wang, Junhui Li, Linlong Xu, Longyue Wang, Weihua Luo
arXiv AI
Aug 28

Do LLMs Understand Personality? Rethinking Persona Fidelity Evaluation through Structured Behavioral Inference

The paper introduces PRISM, a new framework for evaluating how well large language models (LLMs) maintain persona fidelity in dynamic dialogue. PRISM reframes the task as a structured inverse inference problem grounded in Systemic Functional Linguistics, breaking persona fidelity into Task Framing, Interpersonal Stance, and Linguistic Style dimensions. Experiments demonstrate that PRISM produces more accurate and stable judgments than existing holistic or static psychometric methods, offering a more reliable and auditable evaluation process.

By Mengfan Li, Zesheng Wei, Xuanhua Shi, Yang Deng
arXiv Computation and Language
Aug 28

Reward-Informed Sparse Autoencoders and the Solution-Completeness Confound

The paper introduces Reward‑Informed Sparse Autoencoders (RI‑SAEs), which use reinforcement‑learning rewards to curate data for training sparse autoencoders on language‑model activations. On Llama‑3.1‑8B, a sparse subset of features separates high‑reward from low‑reward reasoning continuations, but control experiments show this separation largely reflects solution completeness rather than true reasoning quality. The authors conclude that reward filtering can cheaply reuse RL signals for interpretability, though most of the discovered features capture completion form rather than deep reasoning.

By Tanvi Nagilla, Alexander Jameson, Daniel Manta, Shayaan Uddin
arXiv AI
Aug 28

Training-Time Explainability for Multilingual Hate Speech Detection: Aligning Model Reasoning with Human Rationales

The paper proposes a training‑time explainability framework that aligns model reasoning with human‑annotated rationales to improve both classification performance and interpretability for multilingual hate speech detection. It is evaluated on HateXplain (English) and BullySent (Hinglish), datasets that capture anti‑Muslim hate in culturally coded, multilingual forms. Using methods such as LIME, Integrated Gradients, Grad‑X‑Input, and attention, the study shows that gradient‑ and attention‑based regularization boosts F‑scores, enhances plausibility and faithfulness, and captures culturally specific cues for detecting implicit anti‑Muslim hate.

By Muhammad Deedahwar Mazhar Qureshi, Sannaan Khan, Muhammad Atif Qureshi, Wael Rashwan
arXiv AI
Aug 28

ADeptS-Bench: Measuring the Trustworthiness of Computer Use Agents Across Devices

ADeptS-Bench is a new benchmark designed to assess the trustworthiness of Computer Use Agents (CUAs) across mobile and desktop devices. It consists of two streams: a Safety stream with paired benign and malicious tasks that embed visual threats, and a Disambiguation stream that tests whether agents seek clarification when instructions are ambiguous. Evaluation of seven models shows none consistently achieves high task success while keeping attack success low, and all models exhibit problematic behaviors such as unhesitant checkout on a $25K order and failure to detect a mislabeled factory reset button.

By Joy Chen, Alejandro Castillejo Munoz, Pierluca D'Oro, Yuxuan Sun, Chloe Evans, Joseph Tighe
arXiv Statistics ML
Aug 28

An Accurate and Single-Communication Federated Inference Algorithm

The paper introduces a federated inference algorithm that requires only a single communication round between participating centers and a coordinating server. By extending previous second‑order Taylor expansion methods to third‑order expansions, the algorithm more accurately approximates local log‑likelihood functions, especially when local sample sizes are small. Simulation studies based on real data show that this higher‑order approach improves inference accuracy while maintaining privacy, communication efficiency, and scalability for collaborative biomedical and epidemiological research.

By Laura Montagnani, Anthony CC Coolen, Marianne A Jonker
arXiv Computer Vision
Aug 28

Accurate Measurement of 3D and 2D Circular Centers With Application to LiDAR-Camera Extrinsic Calibration

The paper presents a method for accurately measuring 3D and 2D circular centers, a key geometric sensing problem for instruments like cameras and LiDARs. It integrates a conformal‑geometric‑algebra estimator with RANSAC to jointly recover the 3D center, normal, and radius from noisy LiDAR points, and uses a chord‑length‑variance criterion to estimate the 2D projected center, resolving ambiguity with homography validation or a quasi‑RANSAC fallback. Experiments on synthetic and real sensors demonstrate improved center measurement accuracy and reduced extrinsic calibration error across various target and sensor setups.

By Jiajun Jiang, Xiao Hu, Wancheng Liu, Wei Jiang
arXiv AI
Aug 28

CounterVid: Counterfactual Video Generation for Mitigating Action and Temporal Hallucinations in Video-Language Models

CounterVid introduces a scalable counterfactual video generation framework that creates videos differing only in actions or temporal structure while keeping scene context intact. The approach uses multimodal LLMs for action proposals and diffusion models for editing, producing a synthetic dataset of ~26k preference pairs for action recognition and sequence ordering. With the MixDPO optimization method, the authors demonstrate significant improvements in action recognition and temporal ordering on Qwen2.5‑VL and InternVL3 backbones, while maintaining overall video understanding.

By Tobia Poppi, Burak Uzkent, Amanmeet Garg, Lucas Porto, Garin Kessler, Yezhou Yang, Marcella Cornia, Lorenzo Baraldi, Rita Cucchiara, Florian Schiffers
arXiv Machine Learning
Aug 28

The Rashomon Effect for Visualizing High-Dimensional Data

The paper introduces the Rashomon set for dimension reduction, a collection of equally good embeddings that preserve high‑dimensional structure. It proposes PCA‑informed alignment to make axes interpretable, concept‑alignment regularization to incorporate external knowledge, and a method to extract trustworthy nearest‑neighbor relationships across the Rashomon set for refined embeddings. These techniques aim to produce interpretable, robust, and goal‑aligned visualizations by leveraging multiple valid embeddings instead of a single one.

By Yiyang Sun, Haiyang Huang, Gaurav Rajesh Parikh, Cynthia Rudin
arXiv Computation and Language
Aug 28

SkillChain: Closing the Loop on Skill Evolution for Image-Based E-Commerce AI Assistants

SkillChain is a system designed to automate the lifecycle of skills for image-based AI assistants on e-commerce platforms. It operates in three stages—Skill Creator, Route Optimizer, and Body Refiner—to bootstrap skills from task specifications, align routing, and iteratively refine skill bodies using dual-path LLM-Judge evaluation. In production, SkillChain improves response quality, especially in structural compliance and content quality, and an online A/B test shows increased user engagement, content consumption, and long‑term retention.

By Yimin Hu, Mengtao Xu, Hao Guo, Yuheng Song, Xiaoyong Zhu, Bo Zheng
arXiv AI
Aug 28

AirLLM: Diffusion Policy-based Adaptive LoRA for Remote Fine-Tuning of LLM over the Air

AirLLM introduces a hierarchical diffusion policy framework that adapts Low‑Rank Adaptation (LoRA) rank configurations for remote fine‑tuning of large language models over wireless channels. The system uses a Proximal Policy Optimization agent to make coarse decisions based on wireless and linguistic cues, then refines these decisions with Denoising Diffusion Implicit Models to produce task‑ and channel‑specific rank vectors. Experiments across different signal‑to‑noise ratios show that AirLLM improves fine‑tuning performance while substantially lowering transmission costs.

By Shiyi Yang, Xiaoxue Yu, Rongpeng Li, Jianhang Zhu, Zhifeng Zhao, Honggang Zhang
arXiv AI
Aug 28

A Safety-Gated Multimodal AI Backend for Mental-Health Support: Hierarchical State Representation, Conservative Risk Fusion, and Controlled Generation in Anian

Anian is a safety‑gated multimodal AI backend designed for perinatal mental‑health support and mindfulness‑intervention routing. It maps user input into a four‑layer hierarchical state representation—emotion, psychosocial constructs, safety risk, and intervention routes—then fuses local and external risk signals to decide whether to generate AI responses or provide fixed safety content. Prototype evaluation on large public corpora showed high classification performance and perfect high‑risk recall in a controlled stress test, though clinical validity remains unestablished.

By Lei Wang, Xiao Wang, Lei Li
arXiv Machine Learning
Aug 28

Adversarial Training Without Input Gradients via Low-Rank Householder Expansions

The paper introduces a method for adversarial training that avoids computing input gradients by using a low‑rank Householder expansion (LRHE) to directly generate small‑norm adversarial examples from a network’s parameters. This approach requires only forward passes and standard back‑propagation, eliminating the inner maximization loop and reducing computational cost to roughly 2.8 PGD steps per epoch. The resulting models achieve comparable robustness to multi‑step PGD training for small relative ε budgets, demonstrating the feasibility of gradient‑free adversarial training.

By Tiana C. Johnson, Donsub Rim
arXiv AI
Aug 28

Selection Bias Correction in Retail Intelligence

The paper examines how retail intelligence, which often focuses on high‑velocity products, can suffer from selection bias that skews inflation estimates by overlooking niche items. Using 400 Monte Carlo simulations across four data‑generating scenarios, the authors compare Inverse Probability Weighting (IPW) and stratification methods. They find that stratification generally outperforms IPW—achieving sub‑0.04 percentage‑point median error even when population breaks misalign—while IPW only excels under smooth polynomial relationships, highlighting the importance of method choice in long‑tail retail contexts.

By Spandan Ghose Chowdhury
arXiv AI
Aug 28

Why did My Robot Just Change Personality? Prompting Guidelines for a Grounded Robot Persona in LLM-Based HRI

The paper "Why did My Robot Just Change Personality? Prompting Guidelines for a Grounded Robot Persona in LLM-Based HRI" addresses the lack of clear prompt design in human‑robot interaction using large language models. It proposes a framework and a structured prompt template with eight functional components to specify, bound, and adapt robot behavior. The authors base their guidelines on a review of prior work and on survey data from 27 HRI experts, highlighting issues such as unclear robot personality, the need for user adaptation, and ethical concerns around safety, deception, and governance.

By Ashita Ashok, Franziska Babel, Patrick Holthaus, Rucha Khot, Karla Bransky, Fethiye Irmak Dogan, Karsten Berns, Silvia Rossi, Minha Lee, Guy Laban
arXiv Computer Vision
Aug 28

Cross-Architecture Knowledge Distillation from a Vision Foundation Model to a Lightweight Visual State Space Model for Tea Leaf Disease Classification

The paper presents a method for cross‑architecture knowledge distillation from a fine‑tuned DINOv2 Vision Transformer teacher to a lightweight bidirectional Visual State Space Model (LVSSM) student for tea leaf disease classification. By addressing training‑stability issues with a progressive convolutional stem and gated selective‑scan block, the 4.45 M‑parameter student achieves a mean test accuracy of 95.41%—a 3.09‑point improvement over the teacher’s 92.32%—while using only one‑fifth of the teacher’s parameters. Ablation studies show that simple logit‑level distillation outperforms intermediate feature alignment, and the gains are specific to students that start below the teacher’s performance.

By Zibo Zhou, Zongsen Qiu, Rui Chen, Yujie Yao, Yue Zhou, Jianjun Wang
arXiv AI
Aug 28

EduRiskX: A Neuro-Symbolic Framework with F-Logic Reasoning for Early Academic Risk Prediction

EduRiskX is a neuro-symbolic framework that combines a temporal Transformer-based predictor with an F-Logic symbolic reasoning module to forecast students’ academic risk early in online courses. The neural part models longitudinal activity sequences, while the F-Logic rule base, grounded in Engagement Theory and the Student Integration Model, offers interpretable, rule-based explanations. On the Open University Learning Analytics Dataset, EduRiskX achieves an accuracy of 0.900 and an F1-score of 0.894 by week 38, detecting risk on average by week 9.32 with a 94.30% detection rate, outperforming state‑of‑the‑art time‑series and deep learning baselines.

By Yu Fu, Yongqi Kang, Yong Zhao, Rongfang Bie
arXiv AI
Aug 28

Five Primitives for Governing Autonomous AI Agents at Runtime

The paper proposes five runtime primitives—discovery, identity, governance, attestation, and supply chain—to manage autonomous AI agents in enterprise settings. It argues that traditional control models fail because agents are transient, model-driven, and self‑discoverable, making runtime governance essential. The authors detail an implementation that mediates agent actions against policy, authorizes them via a per‑tenant vocabulary, and records them in a verifiable ledger, noting the associated operational costs and partial deployment status.

By Jiten Oswal, John Cadeddu