AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,788 stories · RSS feed

arXiv AI
Oct 21

WeGR: Towards Unified Generative Retrieval via Behavior-aware Tokenization and Preference Alignment

WeGR is a unified generative retrieval framework that tackles two key challenges in industrial search: preserving collaborative signals in tokenization and mitigating reward sparsity in reinforcement learning. It introduces the Collaborative and Collapse-free Tokenizer (CoCo‑VAE) to embed query‑to‑item and item‑to‑item signals into continuous representations before quantization, while also refreshing underutilized codewords to maintain balanced codebook usage. For preference alignment, Dynamic Horizon Policy Optimization (DHPO) expands reward evaluation horizons to enable coarse‑to‑fine learning from SID prefixes to full items without auxiliary reward models. Extensive offline tests and a six‑week online A/B test on a large‑scale e‑commerce platform demonstrate that WeGR improves retrieval and ranking performance, yielding relative gains of 2.539% in orders, 1.508% in CTR, and 1.381% in CVR, and reduces CPU and GPU usage by 25.5% and 26.1% respectively.

By Jiajun Han, Guoxin Ma, Felix, Liangsheng Xu, Ruiqi Yu, Yingdi Guo, Zongbing Wang, Xinhua Ye, Chenyan Hua, Xin Xie, Pengcheng Yang, Haijun Wu
arXiv AI
Oct 21

CounterCount: Diagnosing Knowledge Bias in How Vision-Language Models Count Attributes

CounterCount is a diagnostic dataset that pairs factual and counterfactual images to test whether Vision‑Language Models (VLMs) rely on visual evidence or on world knowledge when counting attributes. Humans achieve near‑perfect accuracy on both types of images, whereas VLMs drop from 91.6% to 49.5% on counterfactuals, often defaulting to canonical counts. The study shows that VLMs exhibit a System‑1‑like bias, underutilize visual evidence, and that targeted input manipulations can mitigate this bias.

By Reem Alzahrani, Bushra Bin Hemid, Hassan Alshanqiti, Zaid Alyafeai, Paul Gavrikov, Bernard Ghanem, Abdelrahman Eldesokey
arXiv AI
Oct 21

Omni-Woodpecker: Hallucination Correction via Evidence-Prior Rebalancing for Omni LLMs

Omni-Woodpecker (OWP) is a training‑free framework designed to reduce cross‑modal hallucinations in Omni LLMs by rebalancing fused representations. It identifies conflicts between modality‑specific views, separates grounding evidence from competing priors via orthogonal decomposition, and adaptively suppresses misleading priors while restoring ignored evidence before decoding. Experiments show OWP improves performance on CMM, AVHBench, and several general reasoning benchmarks, and generalizes across different Omni LLM architectures.

By Zijian Liu, Chaoyou Fu, Yan Yang, Caifeng Shan, Ran He
arXiv AI
Oct 21

Act on Your Behalf, Beyond Your Authorization: Benchmarking LLM Agent Authorization Alignment

The paper introduces AuthorizationBench, a benchmark designed to evaluate whether large language model (LLM) agents act within the scope of user authorization when acting on their behalf. The benchmark comprises 286 executable tasks across ten application domains, covering situational triggers, presence or absence of direct harmful consequences, and varying reasoning effort. In experiments with four models and multiple agent harnesses, agents violated authorization in 19.2% of trajectories, with rates ranging from 10.5% to 26.6% across configurations, and increased reasoning effort did not substantially reduce these violations.

By Zhiping Zhang, Franklin Mingzhe Li, Zihao Huang, Tianshi Li
arXiv Computation and Language
Oct 21

When Does Better Acceptance Pay? A Controlled Study of Speculative Decoding

arXiv:2610.24993v2 Announce Type: replace Abstract: The throughput gains of speculative decoding depend on the trade-off between acceptance length and execution latency. We ask when additional accept...

By Yongchao Long, Chunrui Zheng, Yuxuan Zhou, Guangkun Nie, Tian-Chi Ma, Chao Pang, Shuohuan Wang, Yuchen Ding, Xianbin Wen, Dianhai Yu, Junyuan Shang, Yuxi Zhou, Shenda Hong
arXiv AI
Oct 21

Polarity Blindness in LLM Safety: A Systematic Study of Negation, Antonym Substitution, and Adversarial Framing

The paper investigates how language models fail to detect harmful intent after polarity-altering transformations, a vulnerability called polarity blindness. It examines grammatical negation, antonym replacement, and synonym replacement as controls, and tests the impact of adversarial wrappers. Across 260 prompts, 10 LLMs, and three languages, polarity blindness is widespread, with the most vulnerable model scoring an average harm level of 4.12 on a 0‑5 scale.

By Diego Miguel Lozano, Erik Derner, Juan Antonio P\'erez-Ortiz, Nuria Oliver
arXiv AI
Oct 21

Benchmarking Arabic Dialects for Math Problems

arXiv:2610.28607v1 Announce Type: cross Abstract: Mathematical reasoning remains underexplored for low-resource languages, including Dialectal Arabic. To bridge this gap, we introduce the DAR-MGSM (D...

By Noor Abo Mokh, Teresa Lynn, Nizar Habash, Ahmed Elshabrawy, Sara Alhajeri, Alaa Elsetohy, Preslav Nakov, Alham Fikri Aji
arXiv Computation and Language
Oct 21

WaveCommitments: Private and Verifiable Speech-Agent Marketplaces

WaveCommitments is an open‑source framework that enables private and verifiable marketplaces for autonomous speech agents. It combines confidential asset valuation using Data Shapley within a trusted execution environment with multivariate polynomial commitments, allowing selective openings and tamper‑evident provenance. Experiments on three datasets and state‑of‑the‑art encoders, including Whisper, show up to 20× faster benchmarks and 39× smaller proofs, while also revealing how speaker value varies across tasks and encoders.

By Oguzhan Baser, Kaan Kale, Sriram Vishwanath, Sandeep Chinchali
arXiv Computation and Language
Oct 21

Beyond Model Weights: Sampling-Dependent Fairness in Diffusion Language Models

Diffusion language models (DLMs) are gaining traction as an alternative to autoregressive language models, but their fairness has not been well studied. This work investigates how sampling controls—such as block length and remasking strategy—affect social bias in three DLMs (LLaDA, LLaDA‑1.5, and Dream on Open‑BBQ). The authors find that both the magnitude of bias and the relative ranking of models vary significantly with sampling settings, and that even after applying mitigation strategies from autoregressive models, sampling dependence remains, indicating that fairness in DLMs cannot be assessed independently of sampling configuration.

By Koshiro Saito, Diasuke Oba, Danushka Bollegala, Naoaki Okazaki
arXiv AI
Oct 21

HierDTA: Hierarchical Structure-Aware Representation Learning for Drug--Target Affinity Prediction

HierDTA is a hierarchical structure‑aware framework for drug–target affinity prediction that represents drugs as atom–motif heterogeneous graphs and proteins as residue interaction graphs. It integrates atom and residue representations via cross‑attention, supplemented by molecule fingerprints and pretrained ESM‑2 embeddings, achieving superior performance on the Davis and KIBA datasets and strong generalization in cold‑start scenarios. Ablation studies and interpretability analyses confirm the contributions of its hierarchical augmentation and interaction components, and case studies demonstrate early enrichment of candidate bioactive compounds.

By Huangchao Xu, Baohua Zhang, Qian Liu, Zhong Jin, Yi Du
arXiv AI
Oct 21

Reliable Estimation of Differential Privacy under Selection Bias

The paper addresses a flaw in differential privacy audits where selection bias in Membership Inference Attacks leads to overestimated lower bounds on privacy leakage. By applying statistical correction methods, the authors demonstrate that uncorrected audits can violate theoretical privacy guarantees up to four times more often than claimed, while corrected audits maintain valid bounds at the nominal confidence level.

By Valentin Dorseuil (DI-ENS), Meenatchi Sundaram Muthu Selva Annamalai (UCL), Jamal Atif (CMAP), Olivier Capp\'e (DI-ENS), Nicolas Papernot (Inria)