AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,808 stories · RSS feed

Hugging Face Trending Papers
Sep 3

Differentiable Interval Bottlenecks for Interpretable Anomaly Detection in Numerical Data

DIFFINT is a reconstruction‑based anomaly detector that replaces the opaque latent bottleneck of a standard autoencoder with a set of soft, axis‑aligned interval memberships learned directly from raw numerical data. Each latent unit represents a human‑readable hyper‑rectangle, and an instance’s anomaly score is its reconstruction error weighted by how strongly it falls inside these intervals. The method provides a certified lower bound on reconstruction error for points outside all active intervals, a graded suppression mechanism for sparse anomalies, and a closed‑form, label‑free importance ranking for each (unit, feature) pair, achieving top performance on 48 ADBench benchmarks against 22 baselines. whyItMatters":"DIFFINT offers the first interpretable anomaly detector that maintains competitive performance while revealing which feature ranges drive each anomaly score, enabling practitioners to audit and understand model decisions without requiring anomaly labels."

Hugging Face Trending Papers
Sep 3

Pushing the (Decision) Boundaries: Dynamically Calibrating Differentially Private Noise to Explainability in Federated Learning

The paper introduces XCal-FL, a federated learning algorithm that dynamically calibrates differential privacy noise using three signals—prediction logit variations, counterfactual margins, and saliency concentration—to improve both predictive accuracy and explanation fidelity. Experiments on medical imaging datasets demonstrate that XCal-FL outperforms static-noise and state‑of‑the‑art adaptive DP methods, achieving over 10% better accuracy and up to five‑fold higher explanation fidelity while using privacy budgets more efficiently. The study reveals that explanation fidelity behaves non‑linearly with privacy loss, indicating that explainability is a separate dimension of the privacy trade‑off that cannot be inferred from utility alone.

Hugging Face Trending Papers
Sep 3

SVG-Score: Human-Aligned Evaluation of Text-to-SVG Generation

SVG-Score introduces a human‑aligned evaluation framework for text‑to‑SVG generation, addressing the inadequacies of existing image‑based metrics like CLIPScore that poorly capture SVG‑specific errors such as color, count, and spatial inaccuracies. The authors first demonstrate that CLIP‑based scores are largely insensitive to these errors and that off‑the‑shelf Vision‑Language Models respond unevenly across error types and styles. They then create a human‑annotated Semantic Alignment dataset and develop two evaluators: a CLIP scorer adapted to vector graphics and a VLM judge trained with supervised fine‑tuning and reinforcement learning, enabling both fast large‑scale and expressive, interpretable assessment of SVG generators.

Hugging Face Trending Papers
Sep 3

IndicSafeEval: Safety Robustness of Large Language Models under Multilingual Persuasive Jailbreak Attacks

IndicSafeEval is a new evaluation framework that tests the safety robustness of large language models against persuasion-based jailbreak attacks in Indian languages. The benchmark covers ten safety-critical content categories, six persuasive strategies, and four languages—Hindi, Bengali, Marathi, and Punjabi—producing 7,200 adversarial prompts. Experiments show that model safety varies significantly across languages, prompt styles, and risk categories, highlighting gaps in current English-centric safety assessments.

Hugging Face Trending Papers
Sep 3

Rent-a-RAG: Embedding-Space Watermarks for Auditing Third-Party RAG

Rent‑a‑RAG introduces DirBucket, a provider‑side semantic watermarking system that embeds documents with meaning‑preserving paraphrases biased toward secret directions, allowing black‑box auditing of document reuse in multi‑provider retrieval‑augmented generation (RAG). The method consistently detects non‑compliance on a challenging benchmark, achieving strong target detection with no false positives and surviving adversarial post‑answer laundering. DirBucket’s detection transfers unchanged to real‑world corpora in clinical, cyber‑threat‑intelligence, and legal domains, demonstrating that embedding‑space watermarking can make third‑party RAG document reuse statistically auditable.

Hugging Face Trending Papers
Sep 3

Federated Causal Discovery via Regression-Directed Cumulants

The paper investigates federated learning for linear non‑Gaussian acyclic models (LiNGAM), proposing the FedRCD family of algorithms that use higher‑order cumulants to enable privacy‑preserving causal discovery across distributed clients. It addresses limitations of existing federated methods, such as FedISHC’s failure under near‑symmetric noise, and introduces variants that balance communication rounds with algebraic noise handling. Experiments reveal that cumulant‑based federated approaches rank variables by a variance ladder induced by the DAG rather than by population asymmetry, and that marginal standardisation degrades performance while scale‑invariant DirectLiNGAM remains robust.

Hugging Face Trending Papers
Sep 3

Understanding Autonomous Driving Datasets by Describing Differences between Image Subsets in Natural Language

The paper introduces set difference captioning for autonomous driving datasets, aiming to generate natural‑language descriptions of differences between two image subsets. It adapts a two‑stage approach to focus on object‑centric patches, enabling attribution of differences to specific objects or categories. A new benchmark, AD‑Diff Bench, is presented to evaluate this method, especially for sparse, real‑world differences, and the authors provide open‑weight models and code for reproducibility.

Hugging Face Trending Papers
Sep 3

Local Updates, Global Learning (LUGL): Playing Games with non-incremental Learners

The paper introduces Local Updates, Global Learning (LUGL), a framework that separates data collection from model training, allowing non‑incremental learners such as gradient‑boosted trees (LightGBM) to be used in reinforcement learning for games. LUGL alternates between a local phase—where self‑play generates tabular updates—and a global phase—where these updates train a function approximator before resetting the table. Experiments on both perfect‑information and imperfect‑information games show that LightGBM agents perform competitively or better than neural‑network baselines like DQN and DeepCFR.

Hugging Face Trending Papers
Sep 3

EraseSAE: Surgical Concept Erasure in Text-to-Video Diffusion Models via Sparse Autoencoders

EraseSAE introduces a surgical concept erasure method for text-to-video diffusion models, using sparse autoencoders to decompose activations into interpretable, monosemantic features. The framework employs a contrastive attribution mechanism to isolate concept-specific kernels and applies timestep-resolved masks during inference to remove target concepts while preserving unrelated content. Experiments show that EraseSAE achieves precise, robust concept removal with minimal quality loss, outperforming existing methods.

Hugging Face Trending Papers
Sep 3

SV-WAM: An Efficient Surround-View World-Action Model for End-to-End Autonomous Driving

SV-WAM is a surround‑view world‑action model that keeps all six camera views for autonomous driving while enabling efficient inference by discarding the video branch during deployment. It uses future‑video prediction as dense training supervision and introduces an action‑centered causal mask to prevent future‑video tokens from influencing action tokens during joint denoising. A differentiable drivable‑area compliance regularizer further improves safety by penalizing vehicle‑footprint corners that approach or cross drivable boundaries. Experiments on NAVSIMv2 and nuScenes show state‑of‑the‑art planning performance with low latency and strong zero‑shot transfer.

Hugging Face Trending Papers
Sep 3

The Attention Triangle in Audio-Video Models

The paper investigates how audio‑video diffusion models use cross‑modal attention, focusing on the "attention triangle" that connects text, audio, and video streams. It finds that the audio‑video edge is bidirectional and heavily influenced by model biases, leading to semantic leakage when prompts conflict with learned priors. By extracting attention signals, the authors develop diagnostic tools and inference‑time interventions that improve cross‑modal alignment without sacrificing generation quality.

Hugging Face Trending Papers
Sep 3

HalluPeer: A Taxonomy-driven Benchmark for Detecting Hallucinations in Scientific Peer Reviews

HalluPeer is a new benchmark designed to detect hallucinations in scientific peer reviews. It provides aligned triples of paper content, human-written reviews, and hallucination-injected reviews, annotated for detection, classification, and localization. Experiments on 12K papers and 38K reviews show that current detectors struggle to distinguish hallucinations from legitimate critique, and real peer reviews contain HalluPeer-defined hallucination patterns, underscoring the need for source-aware verification.

Hugging Face Trending Papers
Sep 3

Scalable Context Orchestration for Serving LLMs Over Voice

Scalable Context Orchestration for Serving LLMs Over Voice presents llmovoice, a middleware that explicitly models voice context—including speaking rate, background noise, and packet loss—to guide large language model responses. By constructing a bounded voice context at each turn, llmovoice improves alignment with user preferences and reduces errors, achieving a 52.4% drop in speaking‑rate alignment error and a 0.9% false‑interruption rate under packet loss. In addition, it cuts model usage costs dramatically, lowering per‑turn cost by up to 24.9× while maintaining 98.7% of baseline answer quality in long sessions.

Hugging Face Trending Papers
Sep 3

Toward Physically Grounded JEPA World Models for Goal-Conditioned Robotic Planning

The paper presents an end‑to‑end JEPA world model that enhances latent prediction with inverse dynamics and state alignment to better support goal‑conditioned robotic planning. By incorporating inverse dynamics, the model prevents latent collapse and encodes action information, while state alignment ties consecutive latent states to their physical configurations and motions. Experiments on four benchmark tasks show the model achieves top success rates on TwoRoom, PushT, and OGBench‑Cube, and its state alignment consistently improves planning performance over inverse dynamics alone.

Hugging Face Trending Papers
Sep 3

WIDE: Wildcard Inference with Dynamic Expansion for Cross-Modal Generative Retrieval

The paper introduces WIDE, a method for cross‑modal generative retrieval that tackles information asymmetry between text queries and visual candidates. WIDE uses Adaptive Entropy Thresholding to set uncertainty limits, Asymmetry‑aware Wildcard Decoding to emit wildcards where the model lacks fine‑grained detail, and Blind‑Spot Re‑ranking to score an expanded candidate set with both discrete confidence and continuous similarity. Experiments on the M‑BEIR benchmark show that WIDE reduces forced hallucination and outperforms existing generative retrieval approaches while keeping index structures compact.

Hugging Face Trending Papers
Sep 3

From Matching Models to Recruiting Agents: A Systematized Narrative Review of AI Recruitment Systems, Evaluation, and Governance

The paper reviews the evolution of AI recruitment systems from simple profile matching to complex, multi‑stage workflows that retrieve evidence, compare candidates, and execute actions. It organizes 40 representative works, highlighting transitions from similarity to reciprocal suitability, from single models to compound workflows, and from offline prediction to evidence‑aligned evaluation. The authors identify persistent gaps such as confounded behavioral labels, limited data validity, hidden pipeline failures, and a lack of privacy assessment, and propose a staged mapping for defensible evaluation and an agenda for evidence‑grounded, auditable systems.

Hugging Face Trending Papers
Sep 3

Neural Video Compression Based on Deformable Temporal Alignment and Difference-aware Fusion

The paper introduces a neural video compression technique that improves temporal context quality by combining deformable temporal alignment with difference‑aware spatial selective fusion. A Context‑aware Temporal Alignment Module generates complementary temporal context, while a Difference‑aware Spatial Selective Fusion module adaptively selects reliable temporal information and suppresses misalignment. Experiments demonstrate that this approach yields better rate‑distortion performance compared to DCVC‑DC.

Hugging Face Trending Papers
Sep 3

When Retrieval Helps: Selective Retrieval for Single-Turn Mental-Health QA

The paper investigates how retrieval-augmented generation (RAG) affects single‑turn mental‑health question answering. It finds that always using retrieval improves response specificity but can lower overall quality and increase safety‑related failures. A lightweight selective retrieval policy, guided by psychoeducational, coping, and safety needs, better balances these trade‑offs by activating retrieval only when necessary.

Hugging Face Trending Papers
Sep 3

Preprocessing Failure and Adversarial Detection in Depthwise-Separable Edge Vision Systems

The paper examines how preprocessing defenses, commonly used to protect edge vision systems, perform on depthwise‑separable CNNs versus residual architectures. Six preprocessing methods were tested against adversarial attacks, revealing that depthwise‑separable models consistently fail to recover from perturbations while residual models show partial recovery. Interestingly, the same preprocessing that hinders clean predictions leaves adversarial predictions largely intact, offering a measurable detection signal, and the study also finds that typical image‑quality metrics do not reliably indicate defense success.

Hugging Face Trending Papers
Sep 3

Inferred Generative-Process Diversity Predicts Correlated Failure Across Language Models

The paper argues that traditional semantic similarity fails to capture the true diversity of language models. It introduces a new metric—generative‑process diversity—measured via Normalised Compression Distance on raw outputs, which reveals hidden population structure among 38 models. This metric predicts lower correlated failures across ten benchmark families, independent of semantic similarity or model capability.