AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,798 stories · RSS feed

arXiv Machine Learning
Sep 21

LLMs as Feature Engineers for Text-and-Tabular Prediction

The paper presents an iterative framework that uses large language models (LLMs) to automatically extract interpretable, schema‑bound categorical features from unstructured text for use in tabular prediction models. A generator LLM proposes semantic definitions, an extractor LLM materializes the features, and a downstream tabular model evaluates their predictive performance, with error‑driven natural‑language feedback guiding the search. Across three public datasets, the error‑driven loop speeds up feature discovery up to three times and the resulting features outperform any subset when combined with TF‑IDF and dense embeddings, while also providing instance‑level interpretability through SHAP importance rankings and a semantic audit trail.

By Merwan Barlier, Blaz Skrlj
arXiv Machine Learning
Sep 21

BrainWideBench: Benchmarking large-scale pretraining and across-animal transfer in multi-region neural recordings

BrainWideBench is a benchmark that evaluates across‑animal transfer on multi‑region neural recordings from 139 mice, covering 276 brain regions. It comprises three task suites—behavior decoding, neural activity prediction, and anatomical organization recovery—to test whether learned representations support diverse downstream objectives. The benchmark shows that while pretraining improves performance over single‑session baselines, current methods vary in transfer ability and none perform uniformly well across all suites, highlighting the challenge of developing general‑purpose neural representations.

By Alexandre Andre, Shivashriganesh P. Mahato, Vinam Arora, Keshav Balaji, Divyansha Lachi, Nanda H. Krishna, Jingyun Xiao, Yizi Zhang, Ximeng Mao, Wenrui Ma, Han Yu, International Brain Laboratory, Daniel Birman, Niccol\`o Bonacchi, Gaelle A. Chapuis, Joana A. Catarino, Felicia Davatolhagh, Mayo Faulkner, Laura Freitas-Silva, Fei Hu, Julia M. Huntenburg, Anup Khanal, In\^es Laranjeira, Petrina Lau, Guido T. Meijer, Nathaniel J. Miska, Jean-Paul Noel, Alejandro Pan-Vazquez, Georg Raiser, Cyrille Rossant, Karolina Z. Socha, Anne E. Urai, Miles J. Wells, Steven J. West, Olivier Winter, Blake Richards, Guillaume Lajoie, Cole Hurwitz, Mehdi Azabou, Matthew R. Whiteway, Liam Paninski, Eva L. Dyer
arXiv Machine Learning
Sep 21

PRIVET: PRoximIty leakage detection Via Extreme value Theory

arXiv:2510.24233v2 Announce Type: replace Abstract: Deep generative models are often trained on sensitive data, such as genetic sequences, health data, or more broadly, any copyrighted, licensed or p...

By Antoine Szatkownik (TAU, BioInfo), Aur\'elien Decelle (TAU), Beatriz Seoane (TAU), Nicolas B\'ereux (TAU), L\'eo Planche (BioInfo), Guillaume Charpiat (TAU), Burak Yelmen (BioInfo, TAU), Flora Jay (BioInfo, TAU), Cyril Furtlehner (TAU)
arXiv AI
Sep 21

Detecting Hallucination in LLMs: Tracing the Topological Signatures of Impaired Context Sharing

The paper investigates how the topology of attention graphs can differentiate hallucinated from non-hallucinated responses in large language models. By analyzing Forman-Ricci curvature, the authors identify structural bottlenecks and develop a method that captures both semi-local and global information-flow characteristics of attention heads associated with hallucinations. Extensive evaluation across multiple LLMs and benchmarks shows that this single-pass approach consistently outperforms existing attention-based and multi-response baselines, while also revealing that impaired context sharing—such as over-reliance on self-attention and information over-squashing—correlates strongly with hallucination occurrences.

By Amir Jalilifard, Anderson Rocha, Eric Wong, Marcos Medeiros Raimundo
arXiv AI
Sep 21

Ability-Residual Decoupled Modeling for Affective Cognitive Diagnosis

The paper introduces an ability‑residual decoupled framework for affective cognitive diagnosis, which first isolates unmodeled cognitive residuals—such as item calibration bias, concept bias, and student‑concept deviations—using student, item, concept, student‑concept, and low‑rank student‑item components. It then applies an affective module that modulates guess/slip effects, with a Q‑matrix‑constrained concept residual attention mechanism to aggregate only item‑relevant concept residuals. Experiments on multiple datasets and backbones demonstrate improved response prediction and better affect alignment, while ablation and analysis studies show that the residual modeling reduces cognitive contamination in the affective branch and enhances robustness and accuracy.

By Boyuan Zhao, Meng Ye
arXiv AI
Sep 21

Dual-Interest Sequential Product Recommendation With Multi-Granular SSM

The paper introduces DSRec, a dual‑interest sequential recommendation model that separates item representations into long‑term and short‑term semantic contexts. Long‑term embeddings capture stable preferences through historical aggregation, while short‑term embeddings focus on local session intent modulated by inter‑click time intervals. Each branch is processed by a distinct State Space Model— a full‑sequence Mamba for long‑term modeling and a time‑modulated SSM for short‑term dynamics— and a residual cross‑fusion mechanism aligns the two granularities while preserving their independence. Experiments on public benchmarks show that DSRec outperforms state‑of‑the‑art methods.

By Shuiying Liao, P. Y. Mok
arXiv AI
Sep 21

MIST: Multimodal Survival Prediction with Genomic-Guided Histology Attention

MIST is a multimodal survival prediction framework that fuses whole-slide images and genomic profiles by representing genomic features as tokens that query histology context tokens derived from a foundation model. The architecture enriches molecular information with histology context before survival prediction, avoiding late-stage merging of separately encoded modalities. Training incorporates discrete-time survival prediction, genomic feature masking, WSI dropout, and contrastive alignment, and demonstrates improved external C-index across colon, renal, lung, and glioblastoma cohorts compared to standard fusion baselines.

By Muhammet Sami Yavuz, Sabri Mustafa Kahya, Richard R. Chen, Jana Lipkova, Benedikt Wiestler
arXiv AI
Sep 21

Aligning with Lived Experience: Heterogeneous Benefits of Fine Tuning in Mental Health Support Generation

The paper introduces the COmmunity-centered Peer Engaged Support (COPES) dataset and a three‑axis evaluation framework to gauge how well Large Language Models (LLMs) align with community perspectives on mental‑health support queries. Experiments show that fine‑tuning LLMs on COPES improves strategy alignment and emotion‑tone alignment by over 50% for general‑purpose models, yet these gains are uneven across subreddits and coping strategies. The study also finds that post‑training shifts the model’s recommendations toward problem‑focused advice while reducing emotion‑focused responses, indicating persistent disparities in performance across different communities and needs.

By Mohit Chandra, Nabin Kim, Eli Min, Aamogh Sawant, Tanmay Sutar, Munmun De Choudhury
arXiv AI
Sep 21

AtomEgo: Exploring Ego-Robot Integration for Embodied Foundation Model Pretraining

AtomEgo investigates how to integrate large-scale egocentric human interaction data into embodied foundation model pre‑training. The study uses a curated 2,659‑hour corpus and a scalable data pipeline to evaluate three co‑training paradigms across vision‑language‑action and world‑action architectures. Results show that the benefit of egocentric data depends on both its scale and the quality of alignment with robotic embodiment, offering practical guidance for scalable ego‑robot pre‑training.

By Di Wu, Dongchen Zheng, Junhe Sheng, Zhongxing Wei, Songxin Zhang, Zejian Xie, Xiaoquan Sun, Junyang Zheng, Zhuoyang Song, Jiaxing Zhang, Jiayu Chen
arXiv AI
Sep 21

HE-Guardrail: A Homomorphic Guardrail Against Jailbreak Attacks for Encrypted Large Language Model Inference

HE-Guardrail is a framework that applies homomorphic encryption to enforce guardrails against jailbreak attacks during encrypted large language model inference. It evaluates guardrail mechanisms—Llama Guard, JBShield, and GradSafe—directly on encrypted data, deciding whether to return the model’s response to the client. The approach preserves confidentiality while closely matching the decisions of plaintext guardrails, offering varied security-efficiency-utility trade‑offs.

By Byeongseo Min, Yongwoo Lee, Young-Sik Kim, Yongjune Kim
arXiv AI
Sep 21

TERMon: Detecting Persistent Behavioral Threats in Edge AI via Hardware-Native Ternary Runtime Monitor

TERMon is a lightweight hardware runtime monitor designed for edge AI accelerators in safety‑critical environments. It detects persistent behavioral threats—such as model corruption, distribution shift, and adversarial inputs—by observing inference behavior through hardware‑efficient ternary patterns matched against a thermometer‑encoded fingerprint. Implemented on a PYNQ‑Z2 FPGA, TERMon operates with a two‑cycle decision latency and requires no on‑chip block RAM or DSPs.

By Arish Sateesan, Edlira Dushku