AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,798 stories · RSS feed

arXiv Computation and Language
Sep 24

"AI Is Turning Too Human": How Teenagers Experience and Negotiate AI in Everyday Life

The study examines how teenagers discuss and navigate generative AI in their daily lives, using data from r/teenagers between January 2023 and July 2026. Analysis of 11,083 posts identified eight interconnected domains of experience, with everyday and social use being most common. Over time, discussions shifted toward concerns about authenticity, personal control, safety, and the future role of humans versus AI, highlighting adolescents’ active negotiation of AI’s place in their lives.

By Jianfeng Zhu
arXiv Computation and Language
Sep 24

Reference-Based Analysis of Coherence and Diversity in Open-Ended Text Generation

The paper introduces a reference-based framework to analyze coherence and diversity in open-ended text generation. It evaluates these properties by aligning them with human trajectories, comparing them to human continuations, and estimating their likelihood under a human reference distribution. Experiments show that diversity alignment and mean-based comparisons correlate with human quality ratings, while reference likelihood also associates positively, though results vary by configuration.

By Esteban Garc\'es Arias
arXiv Computation and Language
Sep 24

SafeTutors: Benchmarking Pedagogical Safety in AI Tutoring Systems

SafeTutors is a benchmark designed to evaluate both safety and pedagogical effectiveness of AI tutoring systems across mathematics, physics, and chemistry. It introduces a risk taxonomy of 11 harm dimensions and 48 sub‑risks based on learning‑science literature, focusing on issues such as answer over‑disclosure, misconception reinforcement, and loss of scaffolding. The study finds that all tested models exhibit broad harms, that larger scale does not mitigate these issues, and that multi‑turn interactions significantly increase pedagogical failures from 17.7% to 77.8%.

By Rima Hazra, Bikram Ghuku, Ilona Marchenko, Yaroslava Tokarieva, Sayan Layek, Somnath Banerjee, Julia Stoyanovich, Mykola Pechenizkiy
arXiv Computer Vision
Sep 24

Damnatio Memoriae: Adversarially and Selectively Forgetting Identities in the Embedding Space of Face Recognition Models

The paper introduces "open‑set adversarial forgetting" to make selected identities unlinkable in face recognition models while keeping the system functional for other users. Three loss functions are proposed: one disperses embeddings from their centroid, and two map each image to a near‑orthogonal target—either learned via the classifier head or fixed as an almost‑orthonormal frame. Experiments on multiple backbones and forget scales show that these losses, especially the orthonormal‑frame approach, effectively render the targeted identities unidentifiable and outperform prior methods while preserving higher retain rates.

By \"Unsal \"Ozt\"urk, Vedrana Krivoku\'ca Hahn, Sushil Bhattacharjee, S\'ebastien Marcel
arXiv Computer Vision
Sep 24

S2A:Semantic-to-Spatial Alignment for Alignment-Free RGB-T Salient Object Detection

The paper introduces S2A, a semantic-to-spatial alignment framework designed for alignment‑free RGB‑T salient object detection. It employs a global‑guided hierarchical fusion module to refine intra‑modal features, an alignment‑free cross‑modal channel attention module to exchange semantic information, and a spatial deformable cross‑attention module to recover local spatial correspondence. These components collectively reduce misalignment‑induced feature contamination and achieve competitive performance on public benchmarks without additional bells and whistles.

By Qiangqiang Zhou, Yang Luo, Yong Chen, Jiawei Xu
arXiv Computer Vision
Sep 24

SynSeq: End-to-End SYNTAX Score Prediction from Coronary Angiography Videos

SynSeq is a video‑based method that directly predicts the SYNTAX score from coronary angiography videos, using targeted preprocessing and a zero‑inflation‑aware loss with linear target scaling. On the CardioSyntax dataset it outperforms prior state‑of‑the‑art approaches, improving $R^2$ by 0.55, reducing prediction bias by 93.1%, and delivering consistent performance across three expert graders. The model also achieves a weighted $F_1$‑score of 0.80 for revascularization treatment recommendations, approaching inter‑expert agreement.

By Christoph Baumann, Ronny Schweitzer, Noemi Pavo, Ulrike Attenberger, Christian Loewe, Philipp Seeb\"ock
arXiv Computation and Language
Sep 24

Consequential Behaviour and Representational Fairness in the Validation of Synthetic Research

Researchers use synthetic survey respondents generated by large language models as substitutes for human samples, but current validation methods often compare them to human surveys in ways that may not reflect real-world consequential behaviour. The authors propose a new validation framework that requires explicit statements of how well synthetic data correspond to human behaviour, specifies which diagnostics are addressed, and demands subgroup-level validity claims to avoid misrepresentation. The framework operationalises distributional, procedural, and recognition justice dimensions and introduces within-persona counterfactual experiments, illustrated with a case study on electric vehicle charging tariffs and concluded with a reporting checklist for researchers.

By Florian Kutzner, Celina Kacperski, Laura de Moli\`ere, Edoardo Chidichimo, Min Jun Jung, Felix Patrick Sedgwick Wallis, James Kunling He
arXiv Computation and Language
Sep 24

Realize What Matters: Principled Context Representation for Large-Scale Reasoning

The paper introduces R3Con, a framework that applies principled design rules—derived from cognitive relevance theory—to construct effective context representations for large-scale reasoning tasks. By systematically organizing vast, heterogeneous information into manageable structures, R3Con outperforms nine state‑of‑the‑art baselines on two large‑document reasoning benchmarks, achieving up to 20 percentage points higher accuracy. Notably, R3Con enables smaller models (4B–9B) to surpass larger 35B models and even Claude Code, demonstrating that principled context representation can reduce dependence on model scale.

By Michael Theologitis, Dean Light, Shuyue Stella Li, Benjamin Newman, Yulia Tsvetkov, Dan Suciu
arXiv AI
Sep 24

PASTABench: Proactive Assessment of Sequential Trajectories for Agent Safety

PASTABench introduces a benchmark of 1,139 multi-turn trajectories to evaluate proactive safety monitoring in large language models. It formalizes three dimensions of intervention—whether, when, and what risk—to address gaps in step-level isolation and post-hoc trajectory assessment. The study finds that proactive intervention is largely unsolved, with the best model achieving only 40.74% optimal-timing interventions, and reveals that smaller models’ safety scores are often driven by lexical overfitting rather than true risk comprehension.

By Jiapeng Sun, Yujin Zhou, Han Zhu, Pengcheng Wen, Jiayi Zhou, Sirui Han, Yike Guo
arXiv AI
Sep 24

ANO: Robust Policy Optimization via Bounded, Redescending Gain Fields

The paper introduces Anchored Neighborhood Optimization (ANO), a new policy‑optimization method that directly designs a smooth, bounded gain field for the probability‑ratio surrogate objective. ANO anchors the identity map at a ratio of one, peaks at a specified trust‑region boundary, and limits the influence of extreme off‑policy samples while providing a bounded, redescending pull on outliers. Empirical results show ANO consistently outperforms existing methods on Atari and MuJoCo benchmarks, and it remains robust under aggressive learning‑rate settings.

By Yiheng Zhang, Yiming Wang, Kaiyan Zhao, Zhenglin Wan, Jiayu Chen, Leong Hou U
arXiv AI
Sep 24

Radiomics and artificial Intelligence for thyroid cancer diagnosis: Concepts, challenges, and solutions

This review examines the use of artificial intelligence and radiomics for diagnosing thyroid cancer, summarizing 42 studies that demonstrate the effectiveness of ultrasound-based radiomics in identifying malignancies. It highlights challenges such as interpretability, limited datasets, and operator dependence, and calls for standardization, prospective multicenter trials, and advances in explainable AI and personalized medicine. The authors emphasize that multidisciplinary collaboration and further algorithm refinement could enhance diagnostic accuracy and patient outcomes.

By Milad Yousefi, Hadi Vahedi, Shadi Farabi Maleki, Mahya Ahmadpour Youshanlui, Aida Jafari, Parisa Rostami, Kais I. Abdul-Lateef Al-Abdullah, Ryszard Tadeusiewicz, Pawel Plawiak, Roohallah Alizadehsani, Siamak Pedrammehr
arXiv AI
Sep 24

Offline A/B Testing of Slate Recommendation Systems with LLMs: Reducing the Dependency on Pre-Collected User Interaction Data

The paper explores using large language models (LLMs) to generate pairwise preferences between slates for synthetic A/B testing of slate recommendation systems. It introduces a validation protocol that checks how well these synthetic preferences align with traditional RecSys metrics and satisfy preference axioms, and examines how LLM pre‑training and configuration influence preference articulation. By combining the synthetic preferences with a generalized Rao‑Kupper model, the authors show that LLM‑based A/B testing can recover stable ranking orderings across different utility weightings, offering a cost‑effective screening step before conducting live experiments.

By Baptiste Bonin, Maxime Heuillet, Audrey Durand
arXiv AI
Sep 24

Joint Interference Detection and Identification via Adversarial Multi-task Learning

The paper introduces a theoretically grounded multi‑task learning framework, AMTIDIN, for joint interference detection, modulation identification, and interference identification. It derives an upper bound linking MTL performance to task similarity measured by Wasserstein distance and adaptive coefficients, and employs adversarial training to reduce distributional gaps across tasks. Experiments show AMTIDIN outperforms single‑task models and other MTL baselines, especially when training data is limited, signals are short, and SNRs are low.

By H. Xu, L. Hu, B. He, S. Wang
arXiv AI
Sep 24

Safeguarding LLM Agents against Long-Horizon Threats via Shadow Memory

The paper introduces ShadowMem, a defensive framework that protects large language model agents from long-horizon threats by maintaining a dedicated safety-focused memory. Inspired by the shadow stack concept, ShadowMem stores safety-critical context throughout an agent’s execution and uses this shadow memory to evaluate the risk of upcoming actions before they are carried out. Experiments show that ShadowMem outperforms existing defenses in detection accuracy, detects most attacks early, and adds minimal overhead to agent performance.

By Yuhui Wang, Tanqiu Jiang, Jiacheng Liang, Charles Fleming, Ting Wang