AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,808 stories · RSS feed

arXiv Computation and Language
Sep 18

Viveka-Insight: a cross-lingual concept graph and citation-grounded retrieval resource over the complete works of Swami Vivekananda in English and Bengali

Viveka-Insight is a bilingual resource and open‑source pipeline for Swami Vivekananda’s complete works, providing a structure‑preserving parse of 32,694 paragraphs and 168,842 sentences, a cross‑lingual concept graph with 8,362 language‑agnostic concepts, a bilingual alias inventory of 60,850 surface forms, and a human‑annotated set of 200 paragraph‑concept edges. The resource enables citation‑grounded retrieval across the English and Bengali corpora, achieving Recall@10 of 0.86 for known‑item cross‑lingual queries and demonstrating concept‑extraction precision of 0.60 (up to 0.71 with confidence filtering). The design is intended to be transferable to other multilingual classical corpora.

By Tamal Maharaj
arXiv Computation and Language
Sep 18

The Neutral Mask: How Alignment Training Provides Shallow Alignment while Leaving Partisan Structure Intact in a Large Language Model

The paper investigates how alignment training, specifically reinforcement learning from human feedback (RLHF), affects the internal partisan structure of a large language model. Using a mechanistic case study on Llama 3.1 8B, the authors find that alignment training does not erase the model’s partisan geometry but compresses its variance, producing consistently balanced, non‑partisan outputs. Sparse autoencoder analysis and feature‑level steering experiments reveal that policy‑encoding features become inactive in the aligned model, indicating a causal disconnect rather than structural removal of partisan knowledge.

By Wendy K. Tam
arXiv Computer Vision
Sep 18

SnapPhysics: A Physics-Aware Scene Graph from a Single View for Interactive Mixed Reality Scenes

SnapPhysics is a training‑free framework that reconstructs 3D objects and estimates their physical properties—mass, friction, and center of gravity—from a single image. It combines instance‑level 3D reconstruction with a physics‑aware scene graph to provide geometric grounding and inter‑object relationships for vision‑language model reasoning. Experiments on 3D‑FRONT and real captured scenes show significant improvements over existing methods, enabling physically interactive mixed reality experiences without manual tuning.

By Suji Kang, Seok-Young Kim, Young Bin Kim, Taewook Ha, Dieter Schmalstieg, Shohei Mori, Woontack Woo
arXiv Computer Vision
Sep 18

GRF-Recon: Global Ray-Field Optimization for Long-Sequence Feed-forward Reconstruction

The paper introduces GRF-Recon, a framework for stable and scalable feed-forward 3D reconstruction from long monocular image sequences. It combines coarse-to-fine trajectory alignment, lightweight geometric prior injection via LoRA adaptation, and a hybrid-weight sparse ray-field optimization to refine local point clouds while enforcing cross-frame consistency. An efficient trajectory stitching strategy with joint ray-error optimization further reduces accumulated drift, achieving competitive trajectory accuracy compared to SLAM systems while maintaining globally consistent reconstructions in large-scale scenarios.

By Enpeng Li, Yunzhou Zhang, Zhiyao Zhang, Dexuan Lyu, Chenyu Wang, Chiyuan Cui, Cheng Cheng
arXiv Computer Vision
Sep 18

ArtNVG: Content-Style Separated Artistic Neighboring-View Gaussian Stylization

ArtNVG is a 3D stylization framework that uses reference style images to produce stylized 3D scenes with high reconstruction quality. Built on 3D Gaussian Splatting, it introduces Content‑Style Separated Control and Attention‑based Neighboring‑View Alignment to maintain local color and texture consistency across views. Experiments show that ArtNVG outperforms existing methods in content preservation, style alignment, and local consistency.

By Zixiao Gu, Mengtian Li, Ruhua Chen, Zhongxia Ji, Sichen Guo, Zhenye Zhang, Guangnan Ye, Zuo Hu
arXiv Computer Vision
Sep 18

State-Change Learning for Prediction of Future Events in Endoscopic Videos

The paper introduces SurgFUTR, a state‑change learning framework for predicting future events in endoscopic videos. Instead of forecasting raw observations, it classifies transitions between current and future states using a teacher‑student architecture and an Action Dynamics module. The authors also present SFPBench, a benchmark with five short‑ and long‑term prediction tasks, and demonstrate consistent improvements across multiple datasets and procedures, including cross‑procedure transfer.

By Saurav Sharma, Chinedu Innocent Nwoye, Didier Mutter, Nicolas Padoy
arXiv Computer Vision
Sep 18

AGORA: Adversarial Generation Of Real-time Animatable 3D Gaussian Head Avatars

AGORA is a new framework that extends 3D Gaussian Splatting with a generative adversarial network to produce high‑fidelity, animatable 3D head avatars. It introduces a lightweight FLAME‑conditioned deformation branch that predicts per‑Gaussian residuals for identity‑preserving, fine‑grained expression control, and a dual‑discriminator training scheme that enforces expression fidelity. The system achieves real‑time inference at 250 FPS on a single GPU and, for the first time, CPU‑only animatable 3DGS avatar synthesis at ~9 FPS.

By Ramazan Fazylov, Sergey Zagoruyko, Aleksandr Parkin, Stamatis Lefkimmiatis, Ivan Laptev
arXiv Computer Vision
Sep 18

Fast Preemptive Robustification: High-Frequency Response Anti-Aligns Shared Vulnerability

The paper introduces Fast Preemptive Robustification (FPR), a lightweight defense that enhances the robustness of deep neural networks against transferable adversarial attacks. By sharpening Laplacian responses through a single 3×3 channel‑wise convolution, FPR eliminates the need for surrogate models, iterative optimization, or specialized training. Experiments show that FPR lowers untargeted attack success rates by 12.7% and reduces targeted attack success from 10.7% to 4.1%.

By Jiaming Liang, Chi-Man Pun
arXiv Computer Vision
Sep 18

HunyuanOCR-1.5: Making Lightweight OCR VLMs Faster and Better

HunyuanOCR-1.5 is a lightweight, end‑to‑end OCR‑specialized vision‑language model that unifies document parsing, text spotting, information extraction, text‑image translation, and multi‑image document understanding. It builds on the HunyuanOCR‑1.0 architecture, improving efficiency with DFlash‑based OCR decoding for faster inference (6.37× Transformer speedup, 2.14× under vLLM) and enhancing capability through an Agentic Data Flow system that autonomously constructs high‑quality training data for long‑tail OCR tasks. The model achieves top‑tier performance on OmniDocBench v1.6 and sets new milestones in ancient‑script OCR, chart/table parsing, multilingual parsing, and hallucination evaluation, while remaining lightweight for deployment.

By Gengluo Li, Xingyu Wan, Shangpin Peng, Weinong Wang, Hao Feng, Yongkun Du, Binghong Wu, Zheng Ruan, Zhiqiong Lu, Liang Wu, Pengyuan Lyu, Huawen Shen, Zibin Lin, Shijing Hu, Jieneng Yang, Hongbing Wen, Guanghua Yu, Hong Liu, Bochao Wang, Can Ma, Han Hu, Chengquan Zhang, Yu Zhou
arXiv AI
Sep 18

AgentPProf: Semantic Profiler for Long Horizon AI Agents

AgentPProf is a new semantic profiler designed for long‑horizon AI agents that aggregates agent trajectories into pprof‑compatible profiles, enabling flame‑graph visualization and hierarchical attribution of tasks and subtasks. It introduces a semantic operation stack model and recursive operation segmentation to replace traditional call‑stack profiling, addressing the challenge of profiling agent intent rather than code paths. In evaluations, AgentPProf achieves high F1 scores against human annotations and significantly improves problem‑localization metrics, demonstrating its effectiveness in attributing resources, locating issues, and optimizing token cost.

By Yusheng Zheng, Chaokun Chang, Yu Mao, Tianyuan Wu, Yuxi Huang, Tao Ma, Wenan Mao, Shuyi Cheng, Andi Quinn, Wei Wang
arXiv AI
Sep 18

PAPC: Platform Mediation for Privacy-Propagation Externalities in AI-Mediated Workflows

PAPC is a platform-mediated mechanism designed to mitigate privacy-propagation externalities in AI‑mediated workflows. It intercepts information‑moving events—such as memory writes, shared‑workspace updates, inter‑agent messages, or tool events—before they update shared state or external channels, using policy, provenance, topology/fanout, privilege, and content signals to decide whether to release a policy‑safe abstraction, quarantine raw content, block a transition, or narrow onward rights. Experiments on retrieval‑memory and multi‑agent workflow benchmarks show that PAPC preserves deterministic task completion while eliminating measured exact raw‑value and external raw‑value exposure, demonstrating that event‑level mediation can serve as a platform‑governance primitive for agent‑mediated online work.

By Tao Huang, Guosen Wu, Chen Hou, Guolong Zheng
arXiv AI
Sep 18

Robust Workflow Generation via Adversarial Learning for Audio Deepfake Detection

The paper introduces ROGUE, a framework that builds robust audio deepfake detection workflows by combining multiple detection tools. ROGUE treats workflow creation as a sequential decision problem and uses a dual-agent system: a perturbation agent generates audio distortions while a policy agent selects and executes detection tools that are resilient to those perturbations. Experiments on several datasets and real-world corruptions show that ROGUE consistently outperforms strong baselines in robustness and generalization, demonstrating the value of adversarially optimized workflow generation for reliable deployment.

By Xiang Li, Pin-Yu Chen, Wenqi Wei
arXiv AI
Sep 18

Harm Laundering in GPT Models: Evidence That Gender Discrimination Is Transformed Rather Than Reduced Across Safety-Trained Generations

The paper investigates how safety evaluations for large language models may mask ongoing gender discrimination by transforming harmful content rather than eliminating it, a phenomenon termed "harm laundering." Analyzing 450,000 gender‑directed completions across GPT‑2 to GPT‑5, the authors find that sexual violence content directed at women disappears while men receive more positive representations, with GPT‑5 showing stark disparities such as framing breast cancer as a men’s rights debate. The study introduces a formal test and detection protocol for harm laundering, demonstrating that reduced toxicity scores do not necessarily reflect reduced representational harm.

By Sarah Wyer, Sue Black, Noura Al Moubayed
arXiv AI
Sep 18

Behavioral Coherence: A Method for Sensitive-Domain LLM Evaluation

The paper introduces behavioral coherence evaluation, a design‑time method that uses validation evidence from an established instrument to test relationships among outputs of large language models (LLMs). Using the Individual Level Abortion Stigma Scale, the authors prompted five LLMs to complete questionnaires for 627 personas and found that the models scored personas lower on self‑judgment but higher on worries about judgment, often reversing the reference direction for Black personas. Expert review highlighted that disclosure guidance from the models requires context about relationship safety, legal risk, and trusted support.

By Anika Sharma, Malavika Mampally, Chidaksh Ravuru, Kandyce Brennan, Neil Gaikwad
arXiv AI
Sep 18

FORGE: Forensic Reasoning with Grounded Evidence

FORGE is a forensic deepfake analysis system that provides region‑grounded natural language explanations for image manipulations. It addresses the inductive bias mismatch of multimodal large language models by adding a Vision‑Only Model trained on dense patch prediction, allowing the language model to interleave tokens with preserved spatial correspondence. Across face‑manipulated and fully synthetic content, FORGE delivers fine‑grained attribute queries and outperforms in‑domain baselines, with region‑specific evaluation and human studies confirming explanation faithfulness.

By Rohit Kundu, Shan Jia, Vishal Mohanty, Athula Balachandran, Amit K. Roy-Chowdhury
arXiv AI
Sep 18

SGM: A Statistical Godel Machine for Risk-Controlled Recursive Self-Modification

The paper introduces the Statistical Godel Machine (SGM), a framework that replaces formal proof requirements with statistical confidence tests to safely allow recursive self-modification in machine learning systems. SGM uses e-values and Hoeffding bounds to certify improvements at a chosen confidence level while managing a global error budget, and introduces Confirm-Triggered Harmonic Spending (CTHS) to allocate risk more efficiently. Experiments on supervised learning, reinforcement learning, and black-box optimization demonstrate that SGM can certify genuine gains on CIFAR-100, reject false improvements on ImageNet-100, and maintain robustness across benchmarks.

By Xuening Wu, Shenqin Yin, Yanlan Kang, Xinhang Zhang, Qianya Xu, Zeping Chen, Wenqiang Zhang
arXiv AI
Sep 18

FedVideoMAE: Efficient Federated Video Moderation with Differential Privacy and Secure Aggregation

FedVideoMAE is a federated learning framework for short‑form video moderation that keeps raw videos on the device and uses a frozen VideoMAE backbone with lightweight LoRA and prompt parameters. Each training round performs self‑supervised masked video reconstruction while applying client‑side differential privacy and pairwise masked aggregation of adapter updates, keeping violence labels out of the federation. On the RWF‑2000 dataset, the method reduces the model‑state payload by 28.3× and achieves 77.25% accuracy without privacy mechanisms, dropping to 65.25–66.00% under differential privacy and secure aggregation.

By Ziyuan Tao, Chuanzhi Xu, Sandaru Jayawardana, Adnan Mahmood, Wei Bao, Kanchana Thilakarathna, Teng Joon Lim
arXiv AI
Sep 18

How Humans and LLMs Read Gender into "Gender-Neutral" Physical Descriptions

The study introduces GAPA, a dataset of 316 physical attributes with 14,706 gender-association ratings from 304 US annotators, showing that such descriptions carry structured gender associations. It evaluates 16 LLMs, finding they partially mirror human ratings but exhibit biases such as compressed distributions, weaker alignment for men, and asymmetric abstention toward non‑binary identities. A proxy model trained on these data is released and applied to analyze character descriptions in LitBank, illustrating the persistence of gendered interpretations in ostensibly neutral language.

By Yingjia Wan, Lin Lin, Elisa Kreiss