AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,808 stories · RSS feed

arXiv Computer Vision
Sep 18

Seeing Abnormal from Normal: Glomerular Abnormality in Representations of Normal Renal Morphology

The paper introduces NoRDeC, a one‑class anomaly detection framework that uses a frozen residual U‑Net backbone pretrained on normal renal morphology to identify and characterize glomerular abnormalities. By combining Mahalanobis normal‑reference scoring with layer‑wise representation analysis and centered kernel alignment, NoRDeC achieves high AUROC scores across multiple abnormality categories, outperforming existing methods such as PaDiM and PatchCore. The study demonstrates that a model trained solely on normal glomeruli can detect unseen pathological subtypes and reveal subtype‑dependent changes in inter‑layer relationships without requiring abnormal training examples.

By Greta Hasko, Rachit Saluja, Tianyu Shi, Leiyue Zhao, Yuechen Yang, Daniel Reisenbuechler, Tianyuan Yao, Zhenhao Guo, John Cannon, Haichun Yang, Yuankai Huo, Yuling Chi, Lorraine Gudas, Mert R. Sabuncu, Yihe Yang, Ruining Deng
arXiv Machine Learning
Sep 18

Embedding Models Measure in Peculiar Ways

The paper investigates whether embedding spaces capture objective physical measurements such as mass, distance, time, and volume. It finds that these embeddings only weakly model such measurements and instead exhibit peculiar patterns. Further analysis shows that superficial string similarity heavily influences the representation of physical measurements, and recalibrating similarity does not significantly improve alignment.

By Juri Opitz, Andrianos Michail
arXiv Machine Learning
Sep 18

Placement Is Free, Composition Is Not: The Latin Square as a Provably-Balanced Construction for Heterogeneous Sequence-Mixer Stacks

The paper introduces Aether‑7B‑5Attn, a 6.59B‑parameter mixture‑of‑experts Transformer that arranges seven distinct sequence‑mixing mechanisms in a 7×7 Latin square across 49 layers, ensuring each mechanism appears exactly once per row and column. Experiments with a 4×4 Latin‑square proxy and larger 1.514B‑parameter models show that balanced heterogeneous composition across depth yields near‑optimal performance, while clustering mechanisms or using a homogeneous stack incurs measurable penalties. The authors provide detailed cost profiles, multilingual evaluations, a causal‑safety audit, and release all code and weights.

By Taebong Kim, Youngsik Hong, Minsik Kim, Sunyoung Choi, Jaewon Jang, Minseo Kim
arXiv AI
Sep 18

Local Sparsity Enables Unsupervised LLM Safety Detection

The paper proposes a new unsupervised safety detection method for large language models that relies on anomaly detection rather than supervised training on unsafe data. By leveraging local sparsity in a linear representation space obtained via a sparse autoencoder, the authors develop a framework for locally masked SAE-based anomaly detection, providing theoretical support and empirical validation across multiple architectures and datasets. When calibrated with only 1% out-of-distribution data, the method achieves near‑optimal performance while using just 1–2% of SAE neurons for computation.

By Xin Chen, Gil Kur, Alexander Shevchenko, Andreas Krause
arXiv Computation and Language
Sep 18

PersonalAI 2.0: Enhancing knowledge graph traversal/retrieval with planning mechanism for Personalized LLM Agents

PersonalAI 2.0 (PAI‑2) is a new framework that improves large language model (LLM) systems by integrating external knowledge graphs through a dynamic, multistage query‑processing pipeline. It performs adaptive, iterative information search guided by extracted entities, matched graph vertices, and generated clue‑queries, and has been evaluated on five benchmarks (Natural Questions, TriviaQA, HotpotQA, 2WikiMultihopQA, and MuSiQue). The results show a 9% average gain in factual correctness over comparable methods, an 18% boost from the search‑plan enhancement mechanism, and state‑of‑the‑art performance on the MINE‑1 benchmark, indicating that PAI‑2 effectively reduces hallucinations and increases precision for personalized AI applications.

By Mikhail Menschikov, Matvey Iskornev, Alexander Kharitonov, Alina Bogdanova, Mikhail Belkin, Ekaterina Lisitsyna, Artyom Sosedka, Victoria Dochkina, Ruslan Kostoev, Ilia Perepechkin, Evgeny Burnaev
arXiv AI
Sep 18

VLN on the Fly: An Onboard Vision-Language Navigation Stack for Aerial Robots

The paper introduces VLN on the Fly, an onboard vision‑language navigation stack for aerial robots that separates grounding, planning, and control into inspectable stages. A quantized vision‑language model grounds instructions to a coarse image cell, depth estimation lifts this to a 3D goal, a fast B‑spline planner generates a feasible trajectory, and a pretrained reinforcement learning policy translates the trajectory into motor commands. In controlled indoor flights, the stack achieved the target in 13 of 15 trials with a mean goal error of 5.72 cm and 39.3% GPU utilization, and successfully tracked collision‑free trajectories in cluttered environments.

By Marco S. Tayar, Felipe Tommaselli, Gianluca Capezutto, Pedro Antonio Rabelo Saraiva, Pedro H. V. de Freitas, Lucas Kido, Guilherme Sonego, Ricardo V. Godoy, Marcelo Becker
arXiv Machine Learning
Sep 18

Multi-center Medical Data Mining with FL-Net - A One-stop Shop for Federated Learning

FL-Net is a federated learning framework designed for medical research that addresses five key requirements identified in the literature. It offers modular data harmonization, discovery, disclosure control, versioned tools, and containerized workflow execution, enabling reuse of harmonized data and workflows across studies. The framework was evaluated on MIMIC and US-130 datasets, supporting up to 50 concurrent clients and demonstrating reproducible, audited federated workflows.

By Simon S\"uwer, Julian Klemm, Elisa Acitelli, Mathieu Almeida, Lucia Altucci, Zsolt Bagyura, Michelangela Barbieri, Zsolt-Zolt\'an Bed\H{o}, Rosaria Benedetti, B\'ela Bihari, Csongor Csal\'oka, Lucia Dicunta, Stanislav Ehrlich, Bjoern M. Eskofier, S\'andor-J\'ozsef Fej\'er, Georg Fr\"owis, Walter H\"otzendorfer, Alexandra Kautzky-Willer, Jens Johann Georg Lohmann, Marianna Maranghi, Lorenzo Marconi, Rudolf Mayer, Wouter Leonard Megchelenbrink, Monika Moga, Adham Mottalib, Sanjeev Mehta, Madeleine M\"uller, Thomas Nystr\"om, Bal\'azs-Attila Orb\'an, Paul O'Toole, Giuseppe Paolisso, Paolo Parini, Matteo Pedrelli, Enrico Petrillo, Philipp Poindl, Niklas Probul, Anastasia Pustozerova, Tanja \v{S}ar\v{c}evi\'c, Lukas Weilguny, Jan Baumbach, Andreas Maier
Hugging Face Trending Papers
Sep 17

Harm Laundering in GPT Models: Evidence That Gender Discrimination Is Transformed Rather Than Reduced Across Safety-Trained Generations

The paper investigates how safety evaluations for large language models may mask ongoing gender discrimination, a phenomenon the authors term "harm laundering." By analyzing 450,000 gender‑directed completions across GPT‑2 to GPT‑5, they show that harmful content directed at women is transformed rather than removed, while men receive more positive representations. The study introduces a formal test and detection protocol for harm laundering, demonstrating that reduced toxicity scores do not necessarily indicate reduced representational harm.

Hugging Face Trending Papers
Sep 17

Video DeltaNet: A Video-Native Hybrid Attention for Livestream Video Generation

Video DeltaNet (VDN) introduces a hybrid attention mechanism for video diffusion models, combining local Softmax attention with a bidirectional linear memory branch called Video Delta Attention (VDA). The design updates memory once per frame, uses separate output projections and learnable gates to balance the two branches, and employs a staged teacher‑alignment recipe to integrate the new pathway into pretrained models. When applied to MiniMax H3, VDN achieves a 14.5× speedup, completing 14.3‑second, 768p video denoising in 6.70 seconds on eight NVIDIA B200 GPUs compared to the 50‑step dense baseline.

Hugging Face Trending Papers
Sep 17

Q&A on Any Spreadsheet Requires Interpreting Its Grid Structure

The paper proposes a new framework that uses semantic cell annotation to split spreadsheets into interpretable chunks for large language model (LLM)-driven Retrieval-Augmented Generation (RAG) systems. This approach improves answer generation by providing richer context rather than merely enhancing retrieval accuracy. However, the authors argue that the inherent two‑dimensional, unstructured nature of spreadsheets imposes a hard ceiling on classification‑based methods, suggesting that future work should focus on dimensionality‑reduction techniques to flatten spreadsheets into one‑dimensional text for easier processing by RAG.

Hugging Face Trending Papers
Sep 17

UniPolicy: Unified Objective-Specific Policies for Generative Search Advertising

UniPolicy is a multi-policy alignment framework for search advertising that jointly optimizes relevance, click propensity, and commercial value. It uses objective-specific prefix tokens, sparse MoE-LoRA routing, and residual FFNs to decouple parameters within a shared backbone, and builds pairwise preferences from multi-stage behavioral feedback to improve generation. In large-scale offline tests and a 7‑day online A/B test, UniPolicy achieved balanced gains across metrics, boosting CTR by 0.71%, RPS by 1.58%, and revenue by 1.32% while keeping latency stable.

Hugging Face Trending Papers
Sep 17

Local Sparsity Enables Unsupervised LLM Safety Detection

The paper proposes a novel unsupervised safety detection method for large language models that relies on local sparsity in a linear representation space recovered via a sparse autoencoder. By masking SAE neurons based on shared active support among nearby points, the authors develop a locally masked anomaly detection framework with theoretical backing. Experiments across multiple architectures and datasets—including capability‑testing and safety‑specific sets—show that using only 1–2% of SAE neurons and a small amount of out‑of‑distribution data yields near‑optimal safety detection performance.

Hugging Face Trending Papers
Sep 17

Dynamic Generalized Gromov-Wasserstein Optimal Transport

Dynamic Generalized Gromov-Wasserstein Optimal Transport extends classical optimal transport by incorporating structure-aware transport costs, which is especially relevant for spatial transcriptomics where preserving tissue structure is crucial. The paper introduces TP-DATE, a simulation-free framework that generalizes GW-OT dynamically, formulating static and dynamic Quadratic-form OT through path actions and proving their equivalence. TP-DATE employs travelling-pair flow matching to enable interacting conditional paths, resulting in better preservation of spatial structure and improved continuous 3D dynamics reconstruction on both synthetic and real spatial transcriptomics data.

Hugging Face Trending Papers
Sep 17

Neuro-Symbolic Agentic AI for Networked Low-Altitude UAVs

The article explores neuro-symbolic agentic AI (NSAAI) as a framework for enhancing the reliability and adaptability of networked low‑altitude UAVs. It outlines NSAAI’s strengths in data efficiency, compositional generalization, continual learning, and zero‑shot transfer, and presents a reference architecture that integrates task management, neuro‑symbolic planning, verification, metacognition, skill execution, and network interaction. A case study of an urban fire‑inspection mission demonstrates how a UAV can coordinate sensing, cloud access, and verified image‑delivery skills under intermittent connectivity, illustrating NSAAI’s potential for reusable skills, evidence‑grounded decision‑making, and adaptive mission execution.

Hugging Face Trending Papers
Sep 17

Graph-Based Stochastic Power-UCT: Monte-Carlo Graph Search with Power Mean Estimation

The paper introduces Graph-Based Stochastic-Power-UCT (GS-Power-UCT), a Monte‑Carlo graph search algorithm that shares states reached at the same planning depth while maintaining separate values for different depths. It achieves an $O(n^{-1/2})$ convergence rate for the root estimate in finite‑horizon stochastic MDPs, matching tree‑based methods but with better sample reuse. Two full‑state variants, GS-Power-UCT‑F and GS-Power-UCT‑F$^+$, further explore sample sharing and bias control, with GS-Power-UCT‑F$^+$ converging to the optimal infinite‑horizon discounted value when the cross‑depth gap vanishes. Experiments on stochastic planning benchmarks demonstrate improved sample efficiency over existing tree‑based and graph‑based baselines.

Hugging Face Trending Papers
Sep 17

Reproducing Transparent and Scrutable Recommendations: Exploring Open-Weight Models via Natural-Language User Profiles

The study reproduces a prior work on recommender systems that use generated natural‑language user profiles to enhance transparency and user control. It confirms that the User Profile Recommendation (UPR) model performs competitively and that altering these profiles uniformly shifts predicted ratings without changing ranking order. Additional experiments include context ablation, multi‑seed stability, and mechanistic interpretability analysis with the nnsight framework.

Hugging Face Trending Papers
Sep 17

SnapPhysics: A Physics-Aware Scene Graph from a Single View for Interactive Mixed Reality Scenes

SnapPhysics is a training‑free framework that reconstructs 3D objects and estimates their physical properties—such as mass, friction, and center of gravity—from a single image. It combines instance‑level 3D reconstruction with a physics‑aware scene graph that encodes inter‑object relationships, providing structured context for vision‑language model reasoning. Experiments on 3D‑FRONT and real captured scenes show significant improvements over prior methods, reducing errors in mass estimation and enhancing scene‑level F‑Score.

Hugging Face Trending Papers
Sep 17

SoK: Trading Agents or Market Crashers? Dissecting Robustness and Security Failures in Academic Financial LLM Trading Schemes

The paper "SoK: Trading Agents or Market Crashers? Dissecting Robustness and Security Failures in Academic Financial LLM Trading Schemes" introduces FARSIGHT, a framework that evaluates financial LLM agents on robustness to market turbulence and security against three attack types. Applying FARSIGHT to 15 academic schemes reveals that 80% fail at least one robustness metric and all exhibit security vulnerabilities, highlighting the risk that a single compromised agent can trigger market-wide crashes.