AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,896 stories · RSS feed

arXiv AI
Aug 19

The Acknowledgment Point Is the System: Durable Policy-Decision Receipts for AI Audit Evidence

The paper presents RuntimeGuard‑AI, a prototype that links each deterministic AI policy decision to its source code, writes a privacy‑minimizing record at a chosen synchronization point, and returns an Ed25519‑signed receipt indicating whether the write succeeded. After a crash, the system validates the integrity of records, manifests, shard placement, sequence continuity, and replay identity, while an independent attestation path chains committed records into signed Merkle epochs for auditor verification. Performance results on an Apple M4 Pro show high throughput (up to 27,193 requests/s) with low latency when buffering, but throughput drops and latency rises when per‑record data and full synchronization are used, illustrating a clear durability‑latency trade‑off.

By Neeraj Kumar Singh Beshane
arXiv AI
Aug 19

MobileWorldSafety: Benchmarking GUI Agent Safety Against Environmental Injection Attacks in Android Apps

MobileWorldSafety is a benchmark that evaluates the safety of large language model–powered GUI agents on Android by exposing them to 142 real-world risk tasks involving environmental injection attacks. The benchmark uses a two‑stage verification pipeline—rule‑based checks for clear cases and an LLM judge for ambiguous ones—to distinguish safety failures from capability failures. Experiments on six agents show high vulnerability, with attack success rates between 40.4% and 66.9%, highlighting that current agents often fail to remain safe when faced with adversarial content presented as normal mobile context.

By Sujin Chen, Lijun Li, Tianyi Du, Jing Shao
arXiv AI
Aug 19

Q-Learning With World Models

The paper introduces QWM, a framework that integrates world models with standard Q‑learning to perform test‑time search over imagined trajectories. By training the policy and value function solely on real transitions, QWM avoids compounding model bias while still benefiting from predictive search. Experiments on the Robomimic and LIBERO manipulation benchmarks show that QWM outperforms strong prior state‑of‑the‑art methods in both sample efficiency and performance.

By Perry Dong, Yueru Jia, Chelsea Finn, Dorsa Sadigh
arXiv AI
Aug 19

CoAL-RAG: A Complexity-Aware Legal Retrieval-Augmented Generation Method

CoAL‑RAG is a complexity‑aware legal retrieval‑augmented generation method that adapts its retrieval strategy based on a multi‑dimensional evaluation of question essence and retrieval consistency. It quantifies reasoning demand from the logical structure of a question and uses the discrepancy between semantic and keyword retrieval to gauge problem complexity, thereby selecting the most suitable retrieval approach and filtering context dynamically. Experiments show that CoAL‑RAG outperforms baseline models on Chinese legal benchmarks (SocialLawQA, LawBench) with a 42.5% BLEU improvement and 3.6× ROUGE‑L, while also achieving strong cross‑jurisdictional performance on English datasets (LexGLUE, CaseHold).

By Jin Su, Zhuofeng Zhao, Huanhuan Wang, Hao Chen
arXiv AI
Aug 19

COMIC: Reference-Aware Safety Gating for Multimodal Large Language Models

The paper introduces COMIC, a reference‑aware safety gate designed for multimodal large language models (MLLMs). COMIC detects the operation requested by a user, identifies visual targets through OCR and open‑vocabulary proposals, and evaluates safety on explicit operation‑target pairs, using max‑risk aggregation and quality‑aware routing to decide whether to allow or block a request. Experiments on several open‑source MLLMs and jailbreak benchmarks show that COMIC improves robustness while maintaining benign utility and efficiency.

By Md Abdullahil Oaphy, Anhao Xiang, Zongxing Xie, Huayue Gu, Chenyu Wang, Honghui Xu
arXiv AI
Aug 19

HMS-SCP: Task-Oriented Multi-Scale Semantic Communication for V2X Cooperative Perception

The paper introduces HMS‑SCP, a hierarchical multi‑scale semantic‑aware cooperative perception framework for V2X communication. It uses a spatial importance predictor to select task‑relevant grid elements at multiple scales and maps them directly into complex‑valued symbols for joint source‑channel coding, achieving ultra‑low symbol rates and noise resilience. Experiments on OPV2V and DAIR‑V2X show that HMS‑SCP maintains high‑confidence far‑field detection with sub‑16 ms latency even under severe Rayleigh fading and extreme compression.

By Chun-Yeow Yeoh, Chee Keong Tan, Joanne Mun-Yee Lim, Heng-Siong Lim
arXiv Machine Learning
Aug 19

MemCatalyst: Amplifying Data Auditing on Vision-Language Models via Data Poisoning

MemCatalyst is a set of data poisoning tools designed to improve data auditing for Vision‑Language Models (VLMs). It introduces two poisoning strategies—Poisoning Text and Poisoning Image—to force VLMs to over‑learn inconsistencies between image features and textual semantics, thereby increasing their vulnerability to membership inference attacks. Experiments on two prominent VLMs show that MemCatalyst significantly boosts MI AUC scores with a small number of poisoned samples while barely affecting overall model performance.

By Xukun Luan, Jinyan Liu, Yuhui Gong, Yuanguo Bi, Bing Hu, Xuesong Li, Di Wang
arXiv AI
Aug 19

LLM Enhancement with Domain Expert Mental Model to Reduce LLM Hallucination with Causal Prompt Engineering

The paper proposes a causal prompt engineering framework that encodes a domain expert’s decision logic—termed an Expert Mental Model (EMM)—into large language models (LLMs) to mitigate hallucinations arising from missing knowledge. By formalizing factor formulation, factor monotonization, and monotonicity-preserving hierarchical structuring, the framework reduces the elicitation burden from millions of combinations to a manageable set, enabling a four-step EMM construction algorithm. The approach allows formal measurement of hallucination rates in tacit expert knowledge and is validated in grant proposal evaluation, cybersecurity design, and clinical diagnosis.

By Boris Kovalerchuk, Brent D. Fegley
arXiv AI
Aug 19

TabularQGAN: A quantum generative model for tabular data synthesis

The paper introduces TabularQGAN, a quantum generative adversarial network designed to synthesize tabular data with both categorical and numerical features. It proposes flexible data encoding and a novel quantum circuit ansatz, and evaluates the model on MIMIC‑III and Adult Census datasets, benchmarking against classical methods such as CTGAN, CopulaGAN, VAE‑GMM, and an LLM‑based approach. Results from noiseless statevector simulations show competitive or leading performance in overall similarity scores and demonstrate strong generalization through custom metrics.

By Pallavi Bhardwaj, Caitlin Jones, Lasse Dierich, Aleksandar Vu\v{c}kovi\'c
arXiv AI
Aug 19

Every Picture Tells a Dangerous Story: Memory-Augmented Multi-Agent Jailbreak Attacks on VLMs

The paper introduces MemJack, a memory‑augmented multi‑agent framework that automatically generates jailbreak attacks on Vision‑Language Models (VLMs) using benign natural images as visual anchors. MemJack discovers visual anchors, camouflages them semantically, evaluates responses, repairs via reflection, and replans dynamically, forming a closed‑loop attack pipeline. The authors also create MemJack‑Bench, a dataset of over 113,000 interactive multimodal jailbreak trajectories, and show that MemJack achieves a 71.48% attack success rate against Qwen3‑VL‑Plus, reaching 90% under extended budgets, outperforming other baselines on natural‑image evaluation.

By Jianhao Chen, Haoyang Chen, Hanjie Zhao, Haozhe Liang, Zheng Wang, Tieyun Qian
arXiv AI
Aug 19

TRUSS: Towards Task-Reliable and User-Safe Automated Agent Skill Generation

TRUSS is a framework that generates and verifies automated agent skills, ensuring they are both functionally effective and safe. It evaluates candidate skills against source evidence and nine safety properties, then tests them in a controlled environment to capture execution traces and identify failures. The system iteratively refines skills based on these results, achieving high precision in vulnerability detection and significantly improving task performance and security rates.

By Zhibo Zhang, Zhen Ouyang, Ling Shi, Kailong Wang
arXiv AI
Aug 19

Leveraging generative hallucination and biophysics-informed modeling for unified biomolecular sequence-structure co-design

The paper introduces MCTH (Monte Carlo Tree Hallucination), an inference-only framework that performs all‑atom biomolecular sequence‑structure co‑design by treating pretrained folding and inverse‑folding models as black‑box operators. MCTH uses Monte Carlo Tree Search to allocate a fixed inference budget across competing design trajectories, incorporating model confidence, uncertainty, and cross‑expert consensus. Experiments across protein‑RNA, protein‑DNA, protein‑protein, and protein‑ligand design show that adaptive search outperforms simpler sampling strategies, and evaluations with AlphaFold3 and Chai‑1 demonstrate transferability beyond the search‑time oracle.

By Xuefeng Liu, Mingxuan Cao, Xiao Luo, Songhao Jiang, Tobin Sosnick, Jinbo Xu, Louis Maher, Rick Stevens
arXiv AI
Aug 19

Cross-Model Memory Transfer via Target-Side Reader Adaptation

The paper investigates how Engram-style hashed memory can be transferred between different language model backbones. By freezing a memory table trained on a source model and attaching it to a target model with only a lightweight reader, the authors find that both the memory content and correct addressing are important, but the reader must be aligned to the target to make the memory useful. In question‑answering experiments, a dual‑layer, four‑branch reader nearly matches same‑model performance, and when the reader interface is directly compatible, the frozen memory alone provides substantial benefit, with optional reader adaptation offering further gains.

By Mingyuan Li, Guangsheng Yu, Xu Wang, Shaoxiong Ji
arXiv Machine Learning
Aug 19

Pathology Transport: Optimal-Transport Explanations for Clinical Data, and When Their Heatmaps (Fail to) Localize Disease

The paper presents an optimal‑transport based generative model that learns the distributional differences between healthy and diseased patients, producing per‑patient counterfactuals and label‑free attribution heatmaps. On tabular breast cancer data the model achieves high malignancy scoring (AUROC ≈ 0.91) and its attributions correlate moderately with a supervised classifier, yet it does not surpass logistic regression. In chest X‑ray experiments the transport heatmaps capture population‑level signals but fail to localize real lesions, revealing a synthetic‑to‑real gap that challenges the reliability of label‑free explanations.

By Lalit Kumar
arXiv AI
Aug 19

LoopVLA: Learning Sufficiency in Recurrent Refinement for Vision-Language-Action Models

LoopVLA introduces a recurrent Vision‑Language‑Action architecture that learns to refine multimodal representations, predict actions, and estimate when further refinement is unnecessary. By iteratively applying a shared Transformer block and producing a sufficiency score at each step, it decouples refinement from fixed layer indices and aligns confidence scores with action quality through a self‑supervised objective. Experiments on LIBERO, LIBERO‑Plus, and VLA‑Arena demonstrate that LoopVLA reduces model parameters by 45% and boosts inference throughput up to 1.7× while matching or surpassing strong baselines in task success.

By Boyang Shen, Kaixiang Yang, Hao Wang, Qiuyu Yu, Qiang Xie, Qiang Li, Zhiwei Wang
arXiv AI
Aug 19

LEGO-RL: Harness-Native Reinforcement Learning for Coding Agents

LEGO-RL is a framework that connects native coding-agent harnesses with scalable policy‑gradient training without altering the harnesses’ internal flow. It achieves faithful optimization through in‑process LLM proxying, reliable execution via sandbox orchestration, and observable training with automated validation and a Live UI. Experiments show LEGO‑RL improves the Qwen3.5‑35B‑A3B model’s performance on three native harnesses while preserving high rollout‑training probability correlation.

By Yiming Du, Yuxin Jiang, Tao Yuan, Jianbo Dai, Shaowei Wang, Jierun Chen, Chaofan Tao, Xianzhi Yu, Lifeng Shang, Kam-Fai Wong, Xiaohui Li, Haoli Bai
arXiv AI
Aug 19

Wuying-Browser-Agent: Real-World Centric Fundamental Long-Horizon Browser Agents

Wuying-Browser-Agent is a unified framework designed to improve long-horizon browser agents by aligning execution, supervision, optimization, and evaluation. It introduces a structured browser harness, reflection and UI-specialized Curriculum SFT (RUIC‑SFT) for recovery and complex UI interactions, and Divergence‑Aware Online GRPO (DAO‑GRPO) for better credit assignment. The framework is evaluated on BrowserBench—a bilingual real‑web benchmark of 350 tasks—and achieves state‑of‑the‑art results on multiple browser‑use benchmarks, while also transferring well to other agentic tasks.

By AIMAE Team, Tianxiang Chen, Yan Cheng, Zhangye Han, Xiaowei Li, Chang Liu, Cheng Liu, Zhongqiang Ma, Long Peng, Xiaobing Tu, Yinggui Wang, Hongliang Wei, Chen Wu, Daiping Xin, Kunyu Zhou, Pengyang Zhou, Peiyuan Chen, Ziyuan Chen, Yutao Deng, Chunyu Dong, Xiangyu Fu, Yicheng Feng, Ruian He, Haochen Li, Miancan Liu, Zhengqin Liu, Wei Peng, Jinkui Ren, Haoyu Tan, Dong Xiao, Rongkun Xue, Shujian Yang, Xianhang Ye, Ziqi Yuan, Ziyang Yu, Linghan Zhang, Xiantao Zhang, Xuanpu Zhao, Yinan Zhao, Zhenghui Zhao, Bin Zhu, Likai Zou
arXiv AI
Aug 19

SkillEffect: Checked Lowering for Memory-Bounded Agent Tools

SkillEffect is a checked‑lowering runtime that ensures agent tool calls stay within memory limits by verifying each proposed program against an immutable input before execution. It uses audited relation plugins to provide source recognition, bounded intermediate representation construction, and postconditions, while a shared runtime handles selection, bounded VM execution, and atomic capacity leasing. Experiments across six operator families show that bounded access significantly reduces peak memory usage and improves completion rates under fixed memory caps.

By Yinuo Wang, Yiyu Shi
arXiv AI
Aug 19

DMT-Dens: Density-preserving manifold visualization for biological data

DMT‑Dens is a parametric manifold‑visualization technique that uses a latent‑token Transformer encoder to produce two‑dimensional embeddings of high‑dimensional biological data. It preserves sampling density by aligning rank‑based manifold structures and optimizing a Pearson‑correlation loss on k‑nearest‑neighbor log‑radius estimates. Benchmark tests show that DMT‑Dens maintains density fidelity while achieving competitive label separability on biological datasets.

By Ruizhe Wang, Yixuan Dong, Bolin Yang, Bingo Wing-Kuen Ling, Fuji Yang, Zelin Zang