AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,808 stories · RSS feed

arXiv Computation and Language
Sep 15

To Each Language Its Tokenizer: Modular Tokenizers for Efficient Multilingual LLMs

The paper proposes a modular tokenizer framework for multilingual large language models, allowing the creation of language‑specific subtokenizers that match monolingual compression quality. It introduces a pretraining strategy that samples these subtokenizers to limit predictions to relevant vocabularies, enabling efficient training and inference. This approach reduces memory usage and speeds up inference without compromising performance.

By Franck Signe, Hippolyte Pilchen, Fran\c{c}ois Yvon, \'Edouard Grave
arXiv Machine Learning
Sep 15

From Network Inequality to Network Fairness: A Perspective on Responsible Decision-Making

The article discusses how social networks influence decision-making and opportunity distribution, noting that network-generating mechanisms often mirror existing inequalities and can amplify disparities when used in technology. It identifies ten network effects that bias the link between intended measurements and observed data, using academic hiring as a case study to show that network biases can be neither inherently harmful nor beneficial. The authors argue for a comprehensive, networked fairness framework that incorporates both distributive and procedural justice and involves all stakeholders.

By Lisette Esp\'in-Noboa, Tina Eliassi-Rad, Pak-Hang Wong, Erich Prem, Meike Zehlike, Ricardo Baeza-Yates, Suresh Venkatasubramanian, Fariba Karimi
arXiv Computer Vision
Sep 15

Hyperspectral Image Restoration and Super-resolution with Physics-Aware Deep Learning for Biomedical Applications

The paper introduces a self‑supervised, physics‑aware deep learning method for hyperspectral image restoration and super‑resolution in biomedical imaging. It achieves 16× pixel super‑resolution and 12× faster imaging without external training data, preserving biological integrity across synthetic and experimental samples. The approach also reveals disease‑associated metabolic changes and offers physical insights into the model’s workings, with all code released as open source.

By Yuchen Xiang, Zhaolu Liu, Monica Emili Garcia-Segura, Daniel Simon, Boxuan Cao, Vincen Wu, Kenneth Robinson, Yu Wang, Ronan Battle, Najah Sobhan, Robert T. Murray, Xavier Altafaj, John Marshall, Luca Peruzzotti-Jametti, Zoltan Takats
arXiv AI
Sep 15

GRIN+: Towards Fast Yet Effective Machine Unlearning for Imbalanced Medical Data

GRIN+ is a new machine unlearning framework that targets fast and precise data erasure in imbalanced medical datasets. It separates unlearning‑specific knowledge from general representations by analyzing gradient contributions of forget and retain sets, introduces a class‑adaptive influence scoring to counter gradient dominance, and uses a direction‑constrained update to protect essential clinical knowledge. Benchmarks on skin cancer, brain tumor, and breast ultrasound data show that GRIN+ balances privacy, efficiency, and utility, achieving high diagnostic accuracy and faster runtime than existing methods.

By Minghui Huang, Junxiao Wang
arXiv Computer Vision
Sep 15

SLAM Adversarial Lab: An Extensible Framework for Visual SLAM Robustness Evaluation under Adverse Conditions

SLAM Adversarial Lab (SAL) is a modular framework designed to evaluate visual SLAM systems under adverse conditions such as fog, rain, camera, and video transport perturbations. It transforms existing datasets into adversarial versions by applying perturbations with severity levels expressed in real‑world units (e.g., meters for fog visibility). SAL’s extensible architecture separates datasets, perturbations, and SLAM algorithms via common interfaces, allowing users to add new components without rewriting integration code, and includes a search procedure to identify the severity at which a SLAM system fails.

By Mohamed Hefny, Karthik Dantu, Steven Y. Ko
arXiv AI
Sep 15

EEG-Xplain: Decoding Neural Black-Boxes of EEG Foundation Models

EEG-Xplain introduces a unified attribution framework to interpret EEG foundation models such as BIOT, LaBraM, and EEGMamba. The framework combines gradient, perturbation, and activation-based methods to analyze model behavior across spatial, temporal, and frequency dimensions, identifying critical channels, decision-relevant signal segments, and contributions of canonical EEG rhythms. It evaluates explanation reliability with population-level metrics and uses large language models to convert structured attributions into natural-language reports, demonstrating consistency with known neurophysiological markers on benchmark datasets.

By Hansong Ma, Junxiao Wang
arXiv AI
Sep 15

One Example Is Enough to Pass Fairness Benchmarks: Rethinking Fairness Evaluation for Aligned LLMs

The paper demonstrates that a single example from the BBQ fairness benchmark can dramatically improve a model’s performance, raising accuracy from 79.9% to 92.9% with Group Relative Policy Optimization and to 99.0% with one-shot in-context learning. This effect is consistent across different model families and is driven by the model’s reasoning traces, which adopt a category‑agnostic "missing evidence" pattern. The authors argue that BBQ-style multiple‑choice abstention tests capture only a single structural cue and therefore do not guarantee true fairness, calling for broader evaluation suites.

By Naihao Deng, Samee Arif, Shuaichen Chang, Yulong Chen, Rada Mihalcea
arXiv AI
Sep 15

Personalizing Personal Health Interfaces: Co-Design with Generative AI

The paper explores how generative AI can lower the barrier to personalizing health dashboards by enabling users to co-design interfaces in Figma Make. In a study with 14 participants, redesigns of Google and Apple Health focused on personal context, future planning, and interactive experiences, though conversational AI designs tended toward chat-window conventions. AI facilitated the materialization of loosely articulated ideas, yet model defaults and generation latency influenced iteration, and the process highlighted interpretability and accountability over privacy, trust, and emotional safety.

By Karthik S. Bhat, Vidhi Shah, Vedika Agnihotri, Dong Whi Yoo, Koustuv Saha
arXiv AI
Sep 15

An Evolutionary Computation Framework for Multi-Agent Q-Learning with Mean-Field Environmental Feedback

The paper presents a mean‑field framework for studying multi‑agent Q‑learning in networked populations, where agents update stateless Q‑values on a fixed graph while the average behavior of the population feeds back to modify the payoff matrix. A deterministic transport equation for the distribution of Q‑values is derived and coupled with a discrete update for the environmental state, and the model is validated against Monte Carlo simulations on several random graph topologies. Results show that the mean‑field system captures macroscopic cooperation dynamics, that environmental feedback reshapes action‑value ordering, and that the timescale of environmental response critically influences learning outcomes.

By Lichen Wang, Shijia Hua, Linjie Liu
arXiv AI
Sep 15

Domain-Specific Jargon in Large Language Models: A Comparative Analysis between General-Purpose and Specialist Models

The paper investigates how large language models handle domain-specific jargon, comparing a general-purpose Llama‑3.1 with a version fine‑tuned on medical data. Two new medical jargon benchmarks reveal that the general model actually outperforms the fine‑tuned variant, and interpretability tools show the fine‑tuned model over‑emphasizes a few components linked to jargon predictions. Reweighting these components narrows the performance gap, and some jargon‑sensitive components also aid materials‑science tasks, indicating a partially domain‑agnostic representation of specialized terminology.

By Darin Keng, Zhewei Sun
arXiv AI
Sep 15

SGWIB:Sliced Gromov-Wasserstein Information Bottleneck for Video Highlight Detection

The paper introduces SGWIB, a single‑modal video highlight detection framework that applies an information‑bottleneck approach while preserving inter‑segment temporal structure through a new Sliced Gromov‑Monge Gap regularizer. It also proposes Home‑Away‑Related Contextual Pseudo‑Labels and a contextual disentanglement module to mitigate sports‑specific bias. Experiments on MrHiSum and MoSu datasets show SGWIB outperforms existing methods on multiple ranking and accuracy metrics.

By Hanjuan Huang, Yung-Chieh Yeh, Hsing-Kuo Pao
arXiv AI
Sep 15

Useless but Safe? Benchmarking Utility Recovery with User Intent Clarification in Multi-Turn Conversations

The paper introduces CarryOnBench, an interactive benchmark that tests whether large language models can revise their interpretation of user intent and recover utility while staying safe in multi‑turn conversations. Using 398 harmful‑looking queries with benign intents, the benchmark simulates 5,970 conversations across 14 models, evaluating both intent‑aligned utility and safety with a new metric called Ben‑Util. Results show that models often withhold information due to misinterpretation, but most can recover with clarifications, revealing failure modes such as unsafe and redundant recovery that single‑turn tests miss.

By Mingqian Zheng, Malia Morgan, Liwei Jiang, Carolyn Rose, Maarten Sap
arXiv Computer Vision
Sep 15

Anchor3R: Streaming 3D Reconstruction with Transient Anchors for Long-Horizon Visual Mapping

Anchor3R is a streaming 3D reconstruction framework that predicts window-relative poses and local geometry in the current‑frame coordinate system, forming a dense relative‑pose graph for online pose updates and loop‑aware motion averaging. It improves long‑horizon pose accuracy and dense reconstruction quality on indoor, outdoor, driving, and RGB‑D benchmarks, and generalizes from 48‑frame training sequences to streams exceeding 10,000 frames while keeping GPU memory bounded. The method addresses issues of train‑test mismatch, early‑anchor bias, and accumulated drift found in previous streaming models.

By Peilin Tao, Chong Cheng, Yuansen Du, Caiwei Song, Zhengqing Chen, Xiaoyang Guo, Wei Yin, Weiqiang Ren, Qian Zhang, Hainan Cui, Shuhan Shen
arXiv AI
Sep 15

Delegating Authorization to Misaligned Agents: Coalitional Alignment and Safe Control

The paper studies how to safely delegate action approval to multiple AI reviewers when the reviewers themselves may be misaligned. It introduces a weaker condition—k‑robust coalitional alignment—under which a threshold rule that tolerates up to k disapprovals guarantees that the principal’s expected utility is at least as good as a baseline policy. The authors extend this characterization to sequential decision‑making in discounted MDPs and show that full‑panel coverage of reward functions ensures safety in Nash equilibria, while more permissive thresholds can lead to unsafe outcomes. Experiments demonstrate that collective review can remain sound even when individual reviewers are not fully aligned, provided some disapprovals are allowed.

By Natalie Collina, Surbhi Goel, Aaron Roth, Sikata Bela Sengupta
arXiv Machine Learning
Sep 15

Data Attribution at Scale via Influence Matrix Estimation

Data Attribution at Scale via Influence Matrix Estimation proposes a scalable approach to quantify how individual training examples influence a model’s predictions. The authors introduce two algorithms, MAGE and SPELL, that reconstruct an influence matrix from a limited number of measurements without extra computational cost, improving over existing baselines across various training scales and budgets.

By Yuxi Chen, Hamza Golubovic, Han Tong, Arian Maleki, Andrew Ilyas
arXiv AI
Sep 15

Hallucination in Multimodal Foundation Models: A Survey on Causes, Corrections, and Evaluations

The article surveys hallucination issues in Large Vision‑Language Models (LVLMs), a type of multimodal foundation model that blends visual data with large language models. It categorizes hallucination causes into model architecture and data quality, presents a taxonomy of mitigation strategies, and critically evaluates existing evaluation benchmarks from both discriminative and generative viewpoints. The survey also outlines open challenges and future research directions to improve LVLM reliability and trustworthiness.

By Yinghao Guo, Wei Lan, Wenyi Chen, Qingfeng Chen, Shichao Zhang, Shirui Pan, Huiyu Zhou, Yi Pan
arXiv AI
Sep 15

Horizon-specific Expert Fusion for Photovoltaic Power Forecasting

The paper presents a hierarchical ensemble for short‑term photovoltaic power forecasting that fuses temporal neural models, historical analogs, state climatology, and gradient‑boosted trees with horizon‑specific convex weights. Solar geometry and numerical weather forecasts provide expected generation conditions, while a calibration step corrects recent bias using historical forecast errors. Evaluated on PVDAQ and GEFCom2014 data, the ensemble reduces mean absolute error by up to 6% compared to strong individual models, though its advantage varies with dataset and horizon.

By Xu Yuqing, Zhou Liguo, Sun Ze, Yu Lei, Jiang Mingming
arXiv AI
Sep 15

Recurrent GraphNeural NetworkswithSet-BasedAggregation

The paper introduces recurrent Graph Neural Networks (GNNs) that use set-based aggregation and establishes conditions that can be verified directly from the network weights. It proves a two‑directional equivalence between these networks and the Boolean closure of reachability and safety properties, corresponding to the fragment BΣ◦₁ of the modal μ‑calculus. This equivalence allows for verifiable symbolic explanations of networks that satisfy the identified conditions, without relying on counting logic or external halting signals.

By Blai Bonet