AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,808 stories · RSS feed

arXiv Machine Learning
Sep 10

Trust-But-Verify: Poisoning-Resilient Locally Private Graph Learning Protocols

The paper introduces VERITAS, a poisoning‑resilient protocol for locally private graph learning that combines local differential privacy with a trust‑but‑verify approach. VERITAS employs a verification list to encode peer trust, performs local data perturbation, server‑side malicious node pruning, dual denoising, and robust private graph learning. Experiments on four real‑world datasets show that VERITAS defends against data poisoning attacks while improving downstream graph learning utility under strict privacy guarantees.

By Longzhu He, Li Sun, Hao Peng, Ruijie Wang, Raymond Chi-Wing Wong, Sen Su
arXiv Machine Learning
Sep 10

On the Recall Scaling Laws in Mamba: A Theoretical and Mechanistic Study via Hashing

The paper investigates Associative Recall (AR) in the Mamba architecture, showing that Mamba implicitly learns linear hash functions to perform recall. It identifies the low‑level circuit responsible for this behavior and develops a theoretical framework—Recall Scaling Laws—based on similarity‑preserving hashing principles. The framework predicts embedding and state dimensions for perfect recall, recall success probability, and analyzes multi‑layer and multi‑head SSM patterns, with empirical results confirming its accuracy.

By Yuval Koren, Assaf Ben-Kish, Raja Giryes, Lior Wolf, Itamar Zimerman
arXiv Machine Learning
Sep 10

BAFF: Bid-Aware Filter Family for Mitigating Training Data Interference in RTB A/B Tests

The paper introduces BAFF, a Bid‑Aware Filter Family that mitigates training data interference in real‑time bidding (RTB) A/B tests by applying (k,l)-parameterized hard filters to control bias from ad‑ranking and bid‑pricing disagreements. It proposes a three‑stage online measurement protocol to evaluate data‑sharing strategies against an interference‑free reference model. Experiments show that BAFF variants outperform both log‑sharing and log‑splitting in offline simulations and live DSP deployments, preserving key business metrics more closely.

By Jeonglyul Oh, Ikkyu Choi, Inseop Youn, Youngjae Kim
arXiv Machine Learning
Sep 10

Do Reasoning Representations Help Humans Evaluate LLM Outputs?

The paper investigates whether reasoning representations—explanations for large language model outputs—aid humans in evaluating those outputs. A controlled human study tested six reasoning formats across tasks of varying complexity, measuring structural understanding, error detection, and trust calibration. Results revealed a mismatch: participants favored planning- and decomposition-based representations, yet simpler chain-of-thought traces better supported verification, trust, and interpretability, while preferred formats increased calibration risks.

By Jaewoo Lim, Sungbok Shin, Sanghyun Hong
arXiv Machine Learning
Sep 10

ZetaDial: dialing net charge of protein binders at inference time for therapeutic developability

ZetaDial introduces a post‑sampling, per‑protein secant controller that adjusts the net charge of protein binders generated by fixed‑backbone ProteinMPNN, enabling inference‑time tuning of developability‑relevant charge. On benchmark datasets, the secant loop reduced mean absolute charge error compared to fixed‑slope and global‑bias approaches, achieving significant improvements on RCSB complexes and comparable performance on Cas13 monomers. The method also highlights trade‑offs between charge calibration and foldability, with exploratory tests on PD‑L1, IL‑7R‑alpha, and RBD designs illustrating its impact on design success rates and interface properties.

By Mohammed Sameer Syed, Tamara Dinneen
arXiv Machine Learning
Sep 10

A Specialized Large Multimodal Model for Interpreting PET/CT in Head and Neck Cancer

A specialized large multimodal model, LLaVA‑NeXT, was fine‑tuned on a curated two‑level curriculum of PET/CT image‑conversation pairs to interpret head and neck cancer scans. In external validation across four institutions, the model achieved high ROUGE and similarity scores and outperformed generalist models such as ChatGPT, with primary tumor classification accuracy of 83.14% internally and 69.03% externally. The study demonstrates that domain‑specific LMMs can provide fast, accurate diagnostic support for PET/CT imaging.

By Haengbok Chung, SunGyu Kim, Joo hyun Lee, Sangjin Bae, Min Jeong Cho, Minseok Suh, Jae Sung Lee
arXiv Machine Learning
Sep 10

Cross-modal learning for SAR target recognition using optical vision foundation models

The paper proposes a cross‑modal framework that uses a frozen DINOv3 optical vision foundation model to create class‑level prototypes for Synthetic Aperture Radar (SAR) target recognition. By aligning SAR embeddings to these optical prototypes, the SAR model learns to classify SAR images without needing paired optical data. Experiments on the UNICORNv2 dataset show that this prototype alignment improves SAR classification accuracy compared to baseline methods and yields clearer class separation in the embedding space.

By Lucas Hirsch, James R. Hopgood, Javid Khan, Yoann Altmann, Mike E. Davies
arXiv Machine Learning
Sep 10

MamMA: A Mamba-Based Pedestrian Trajectory Prediction Algorithm Considering Occupancy Map and Pedestrian Awareness States

MamMA is a pedestrian trajectory prediction algorithm that leverages LiDAR-generated occupancy maps and egocentric vision sensor data. It partitions the occupancy map into patches to extract obstacle features and incorporates pedestrian awareness states, which influence perception and speed. Using a Mamba-based model, MamMA predicts future trajectories and outperforms state‑of‑the‑art methods on multiple benchmark datasets.

By Juncen Long, Xiaofeng Jin, Gianluca Bardaro, Simone Mentasti, Matteo Matteucci
arXiv Machine Learning
Sep 10

When Topology Betrays Privacy: Lattice-Based Reconstruction Attacks on Secure Aggregation in Decentralized Federated Learning

The paper demonstrates that in decentralized federated learning, secure aggregation implemented through local neighborhood aggregation can leak private model updates. By exploiting the asymmetric aggregate views available to colluding semi‑honest nodes, the authors formulate the problem as a Hidden Subset Sum Problem and develop a lattice‑based reconstruction attack. Experiments on image, tabular, and text datasets show that attackers can recover honest participants’ updates and reconstruct private training data.

By Wenrui Yu, Changlong Ji, Johannes Bjerva, Qiongxiu Li
arXiv Machine Learning
Sep 10

A Closed-Form Estimator and Diagnostic Battery for Anchor-Judge Error Correlation, Under a Single-Common-Factor Model

The paper presents a closed‑form estimator for the contamination correlation between anchors and judges under a single‑common‑factor model, requiring at least two judges and two anchors. It introduces a diagnostic battery—including judge‑covariance dispersion, over‑identification tests, a family‑block test, bootstrap confidence intervals, and a weak‑identification screen—to validate the estimator and detect violations. The authors also discuss identification limits for ordinal data and report that real panels have not yet passed the model‑adequacy pre‑test, while simulation studies confirm the estimator’s performance.

By Veerendra Kumar Sunkavalli
arXiv Machine Learning
Sep 10

Mixing Makes Markovian Contexts Cheap for Linear Bandits

The paper extends the idea that contexts are cheap for linear bandits from i.i.d. settings to Markovian context processes. By assuming uniform geometric ergodicity, the authors construct a stationary surrogate action set and use a delayed‑update scheme to mitigate bias from nonstationary conditional context distributions. They provide a phased algorithm for unknown stationary distributions and achieve high‑probability regret bounds comparable to standard linear bandit oracles in fast‑mixing regimes, with empirical validation showing gains over LinUCB.

By Kaan Buyukkalayci, Osama Hanna, Christina Fragouli
arXiv Machine Learning
Sep 10

Branch Geometry and Finite-Radius Sensitivity of Hard-ReLU Training

The paper investigates how hard‑ReLU training behaves when perturbations have a finite radius. It shows that the usual infinitesimal sensitivities are insufficient to predict the response at a chosen radius, and it characterizes the intermediate regime where the perturbation radius scales with the gradient‑descent step. The authors derive crossing indices, a uniform endpoint expansion for separated transverse events, and provide explicit remainder terms in contractive affine regions to certify finite candidate comparisons, supported by experiments on nonlinear networks.

By Xiaoyang Li, Runni Zhou, Xinghao Yan
arXiv Machine Learning
Sep 10

Interpretable Network-assisted Random Forest+

The paper introduces Interpretable Network-assisted Random Forest+ (RF+), a family of flexible models that combine the predictive power of random forests with network information. It offers intrinsic interpretability by providing global and local feature importance measures, as well as sample influence metrics, allowing researchers to assess both feature effects and the contribution of network neighbors. The authors claim that RF+ achieves competitive prediction accuracy while remaining transparent, making it suitable for high-impact problems where understanding model decisions is crucial.

By Tiffany M. Tang, Elizaveta Levina, Ji Zhu
arXiv Machine Learning
Sep 10

Atomistic Modeling of Chemical Disorder in Materials: Bridging Conventional Methods and AI-Assisted Approaches

The article reviews strategies for modeling chemical disorder in materials, addressing the gap between experimental descriptions of disorder and the detailed configurations required for atomistic simulations and AI workflows. It evaluates traditional approaches such as mean-field theories, cluster expansion, and Monte Carlo, alongside emerging AI-powered methods like universal interatomic potentials and generative models. The review also discusses how AI can accelerate computational schemes and enable disorder-native capabilities, providing a roadmap for integrating disorder into realistic AI-accelerated materials discovery.

By Jiayu Peng, Peichen Zhong
arXiv Computation and Language
Sep 10

X2-NativeCursor: Native-Token Text Progress Tracking for Incremental-Text Streaming Codec TTS

X2-NativeCursor is a lightweight observer that tracks text progress in incremental‑text streaming TTS by aligning native speech tokens to the original text before waveform decoding. It uses a normalization plan and a local matcher to estimate the current label position, converting these estimates into a non‑backtracking cursor. The method achieves a mean absolute error of 0.151 Chinese characters with 80‑ms lookahead, outperforming a waveform‑based baseline and reducing alignment real‑time factor from 0.3598 to 0.0180.

By Zehan Liu, Carl Chen, Rime Wen, Kaiqi Fu, Altman Lin, Shawn Qin, Lights Shi, Roy Gan, Hao Wang, Qian Wang
arXiv Computation and Language
Sep 10

From Retrieval to Weights: Parametric Individualization of Small Language Models with Individual Text Corpora

The paper explores how individual text corpora (ITCs) can be integrated into small language models (SLMs) using DoRA fine‑tuning. By training a DoRA adapter for each of 150 participants, the authors show that the adapter can encode a participant’s own ITC into the model’s weights, improving fit to that participant’s held‑out text. However, while the adapter improves log‑loss on a generalized knowledge test, it does not enhance accuracy under a bias‑corrected PMI readout, and adding retrieval does not provide further benefit.

By Christoph Wigbels, Ali Abusaleh, Markus T. Jansen, Alexander Mehler, Markus J. Hofmann
arXiv AI
Sep 10

Safety boundary maintenance in consumer AI systems responding to pediatric health queries: a cross-platform benchmark evaluation under naturalistic and adversarially pressured conditions

arXiv:2601.09721v2 Announce Type: replace-cross Abstract: Consumer artificial intelligence chatbots are now accessed by hundreds of millions of users seeking health information, yet systematic evalua...

By Vahideh Zolfaghari, Leila Mashhadi, Mitra Ahadi, Farzaneh Sedaghatkar, MohammadReza Kargozari
arXiv Machine Learning
Sep 10

The BatchNorm Illusion: Diagnosing Normalization Artifacts in Machine Unlearning Evaluation

The paper uncovers a hidden flaw in evaluating machine unlearning on BatchNorm-based models: a single forward pass over retained data can alter the model’s normalization state without changing weights, misleadingly restoring performance. This effect, formalized as a weight‑preserving fixed‑point operator, shows that apparent forgetting can be entirely due to BatchNorm running statistics, not to the unlearning method itself. Experiments demonstrate that the artifact can inflate forget accuracy by up to 78 percentage points across nine methods, and that switching to GroupNorm eliminates the problem.

By Aaryaman Kalani, Murari Mandal, Dhruv Kumar, Mohan Kankanhalli, Yash Sinha