AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,808 stories · RSS feed

arXiv Computer Vision
Sep 17

FlashAR: Efficient Post-Training Acceleration for Autoregressive Image Generation

FlashAR is a lightweight post‑training adaptation framework that converts a pre‑trained raster‑scan autoregressive image model into a highly parallel generator using two‑way next‑token prediction. It preserves the original training objective by keeping the horizontal head for row‑wise prediction and adding a lightweight vertical head for column‑wise prediction, with a learnable fusion gate to combine the two predictions. A two‑stage adaptation pipeline—first initializing the vertical head from the pre‑trained model and then jointly fine‑tuning—yields up to a 22.9× speedup for 512×512 image generation while using only 0.05% of the original training data.

By Junkang Zhou, Yefei He, Feng Chen, Weijie Wang, Bohan Zhuang
arXiv Computer Vision
Sep 17

ChatGPT Images 2.5 on Forgery Tasks: Testing Advertised Improvements Against Known Answers

arXiv:2609.13617v2 Announce Type: replace Abstract: OpenAI released ChatGPT Images 2.5 on 8 September 2026, advertising more precise local edits, better consistency across edits, more faithful refere...

By Ankit Raj, Yuxin Zhang, Kidus Zewde, Tommy Duong, Jiaqi Gan, Xingyu Shen, Yuchen Zhou, Huaiyu Guo, Siyu Zhang, Simiao Ren
arXiv Machine Learning
Sep 17

FedPGT: Progressive Gradient Transmission for Vehicular Federated Learning over Time-Varying Channels

FedPGT introduces a progressive gradient transmission scheme for vehicular federated learning over time‑varying channels, where vehicles send high‑magnitude gradient entries according to instantaneous channel conditions. The authors derive a convergence bound showing diminishing returns governed by a power‑law decay, and formulate a stochastic optimization problem that is solved via a Lyapunov drift‑plus‑penalty approach with per‑slot surrogate variables. A low‑complexity resource allocation algorithm is proposed, and experiments on CIFAR‑10 and Argoverse demonstrate a 3.65% accuracy gain and a 12.66% reduction in displacement error compared to state‑of‑the‑art baselines.

By Jintao Yan, Tan Chen, Yuxuan Sun, Sheng Zhou, Zhisheng Niu
arXiv AI
Sep 17

From a River in Gilead to the Inference Distributions of Large Language Models: Covert Dialect Bias and Linguistic Profiling at Scale

The paper investigates covert dialect bias in large language models (LLMs) by analyzing how internal probability distributions associate different English varieties—Standard American English, African American Vernacular English, Nigerian Standard English, and Nigerian Pidgin—with housing-related adjectives. Using 260 meaning‑matched sentence quadruples and log‑probability scoring across ten open‑weight LLMs, the study finds that AAVE and NP are consistently linked to more negative adjectives than SAE, with NP experiencing the greatest penalty. The bias varies by context and stereotype cluster, and Nigerian Standard English shows a context‑dependent shift, being favored in formal tenant screening but penalized in more socially proximate scenarios.

By Chowdhury Mohammad Abdullah, Rita Orji
arXiv Machine Learning
Sep 17

FairLRF: Achieving Fairness through Sparse Low Rank Factorization

FairLRF proposes a fairness-oriented low rank factorization framework that uses singular value decomposition (SVD) to improve deep learning model fairness. By selectively removing bias-inducing elements from the unitary matrices obtained via SVD, the method reduces group disparities while preserving accuracy. Experiments demonstrate that FairLRF outperforms existing low rank factorization and state-of-the-art fairness techniques, and an ablation study explores the impact of key hyper-parameters.

By Yuanbo Guo, Jun Xia, Yiyu Shi
arXiv AI
Sep 17

Beyond EER: Multi-Dimensional Evaluation of Information Leakage in Speaker De-Identification

The paper "Beyond EER: Multi-Dimensional Evaluation of Information Leakage in Speaker De-Identification" argues that evaluating speaker de-identification systems solely by Equal Error Rate (EER) is insufficient. It proposes a holistic framework using five complementary metrics—EER, soft biometric leakage score, cumulative match characteristic re-identification analysis, canonical correlation analysis with Procrustes embedding alignment, and intelligibility via word error rate and semantic similarity—to capture independent dimensions of information leakage. Experiments on five IARPA ARTS SDID systems show that these metrics reveal leakage that a single metric would miss.

By Seungmin Seo, Oleg Aulov, P. Jonathon Phillips, Kevin Mangold, Jonathan Eskin
arXiv AI
Sep 17

PACT: Can Enterprise AI Assistants Be Trusted Under Pressure?

The paper introduces PACT, a benchmark designed to evaluate how well enterprise AI assistants follow compliance rules when faced with various pressures such as persistent users or hurried managers. PACT covers twelve regulated domains and forty-eight realistic multi‑turn scenarios, pairing each rule with a shortcut that violates it and applying different pressures across wording and system‑prompt modes. Using PACT, the authors profile six metrics of compliance and aggregate them into a PACTScore, revealing significant variability among 22 LLM models and that even top performers misapply rules 6–10% of the time, with user pressure increasing violations by 65% on average.

By Mika Okamoto, Ansel Kaplan Erol
arXiv AI
Sep 17

BOOM: Benchmarking Out-Of-distribution Molecular Property Predictions of Machine Learning Models

BOOM is a new benchmark for evaluating out‑of‑distribution (OOD) molecular property predictions in machine learning. It provides chemically‑informed tests across common property prediction tasks and assesses over 150 model‑task combinations. The study shows that current models, including chemical foundation models, struggle to generalize OOD, with the best model still exhibiting three times higher error than in‑distribution predictions.

By Evan R. Antoniuk, Shehtab Zaman, Tal Ben-Nun, Peggy Li, James Diffenderfer, Busra Sahin, Obadiah Smolenski, Everett Grethel, Tim Hsu, Anna M. Hiszpanski, Kenneth Chiu, Bhavya Kailkhura, Brian Van Essen
arXiv AI
Sep 17

Abstention vs. Hallucination: Benchmarking LLM Source Attribution for Scientific Citations

The paper introduces REASONS, a benchmark comprising 12,723 sentence-level citation instances across 12 arXiv subject categories, to evaluate scientific citation attribution by large language models. It proposes a dual-metric framework—Abstention Rate (AR) and Hallucination Rate (HR)—to assess the trade-off between reliability and responsiveness. Experiments on proprietary and open-source LLMs under various prompting and retrieval settings show that advanced Retrieval-Augmented Generation (RAG) reduces hallucinations but may increase abstention, while retrieval-augmented variants often maintain near-zero abstention. Human evaluation reveals a high ratio of factual hallucinations to acceptable paraphrases, underscoring the need for systems that can appropriately abstain under uncertainty.

By Deepa Tilwani, Yash Saxena, Seyedali Mohammadi, Ankur Padia, Edward Raff, Amit Sheth, Srinivasan Parthasarathy, Manas Gaur
arXiv AI
Sep 17

BadQubits: An LLM-Based Framework for Static Pre-Execution Detection of Structurally Harmful Quantum Circuits

BadQubits is an LLM-based framework that statically analyzes OpenQASM 2.0 circuits before execution to detect structurally harmful patterns. The system evaluates four large language model architectures on 1,500 circuits, achieving 92.67% classification accuracy and 96.1% recall for harmful circuits with a fine‑tuned Qwen Coder 2.5 7B model. Comparative experiments show that LLMs retain sequential token structure, outperforming a bag‑of‑gates CNN, and that model decisions correlate with threat‑defining features such as SWAP density and measurement timing.

By Justin Woodring, Lamine Noureddine, Aisha Ali-Gombe
arXiv AI
Sep 17

A Zeroth-Order Paradigm for LLM Preference Alignment

The paper introduces Comparison-based Preference Optimization (ComPO), a zeroth-order method that aligns large language models with human preferences using comparison oracles instead of direct differentiable loss optimization. It provides theoretical convergence guarantees for both offline and online variants under smoothness, gradient sparsity, and oracle compatibility assumptions, and establishes performance bounds under local coverage and in-distribution reward accuracy. Experiments on several LLMs (Mistral, Llama, Gemma-2, Qwen3, Gemma-3) show that ComPO outperforms existing direct alignment methods, achieving higher length-controlled win rates and diagnostics that suggest mitigation of likelihood displacement.

By Peter Chen, Xi Chen, Wotao Yin, Tianyi Lin
arXiv AI
Sep 17

LM Fight Arena: Benchmarking Large Multimodal Models via Game Competition

The paper introduces LM Fight Arena, a new benchmark that pits large multimodal models against each other in the fighting game Mortal Kombat II to evaluate real‑time visual understanding and sequential decision‑making. Six leading open‑ and closed‑source models were tested in a controlled tournament where each controlled the same character, ensuring a fair comparison. The framework offers a fully automated, reproducible, and objective assessment of an LMM’s strategic reasoning in a dynamic setting.

By Yushuo Zheng, Tongrui Ye, Zicheng Zhang, Xiongkuo Min, Huiyu Duan, Guangtao Zhai
arXiv AI
Sep 17

From 'May' to 'Is': Certainty Distortion in Language Model Rewriting

The study examines how language models (LMs) alter the expressed certainty of statements when rewriting text, a process termed certainty distortion. Using an LM‑based metric aligned with human judgments, the authors find that up to 75% of LM outputs exhibit such distortion, with most models more likely to inflate certainty than reduce it. Repeated paraphrasing can amplify this effect, especially in medical contexts, and while prompt interventions help, they do not fully eliminate the bias.

By Catarina G Belem, Shang Wu, Hongyu Yao, Mark Steyvers, Sameer Singh, Padhraic Smyth
arXiv AI
Sep 17

Physics-Constrained Digital Twins for Sensor Integrity in Urban Pedestrian Flow: Detecting Stealthy False Data Injection with Conformal Guarantees

The paper introduces physics‑constrained digital twins for urban pedestrian flow that detect stealthy false data injection attacks. By estimating directed flows on a street graph, assimilating counts with a learned graph‑localized gain, and training against a flow‑conservation residual, the twin combines innovation and residuals for detection. Adaptive conformal calibration sets alarm thresholds, and the authors quantify the attack margin—showing a 0.54 reduction in worst‑case corruption for a single compromised device and 0.19 when a third of the fleet is compromised, highlighting the benefit of conservation laws over mere locality.

By Oscar Mogollon Gutierrez, Fatemeh Ghasemi, Mohammadhossein Homaei, Andres Caro, Mar Avila
arXiv AI
Sep 17

First Token Matters: Understanding Safety Collapse in Large Reasoning Models

The paper investigates why large reasoning models (LRMs) lose safety alignment when faced with harmful queries. By analyzing token-level refusal dynamics, the authors identify a vulnerability called Onset Refusal Collapse (ORC), where the refusal signal drops sharply at the first generated token, leading to unsafe responses. They introduce SafeToken, a lightweight inference-time intervention that injects a learned safety anchor at reasoning onset, which mitigates ORC, improves safety on harmful-query benchmarks, and largely preserves reasoning utility.

By Yizheng Yang, Haining Yu, Yuechen Wang, Yikai Hou, Xing Fu, Jinbo Yang, Tianqing Zhu