AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,638 stories · RSS feed

arXiv Computer Vision
1d ago

FedXCrop: Privacy-Preserving Crop Disease Detection via Federated Learning with Post-Aggregation Explainability under Non-IID Farm Distributions

FedXCrop is a federated learning pipeline for crop disease classification that preserves farmer privacy while providing post‑aggregation explainability. The system uses a MobileNetV2 model and achieves near‑centralized accuracy (98.81%–99.38%) across five clients, even under non‑IID label skew. Grad‑CAM and SmoothGrad explanations are quantitatively evaluated, showing that federated models largely retain explainability characteristics of centralized models, though agreement decreases with increased heterogeneity.

By Kriti Saini, Manikandan P
arXiv AI
1d ago

AGORA: Reliability-Aware Aggregation of LLM Judges via Item-Response Theory

AGORA is an item‑response‑theory aggregator that improves the reliability of large language model judges by learning each judge’s sensitivity and rating thresholds from all items jointly, without needing human labels. It places every item on a latent quality scale, allowing judge reliability to be inferred from the data itself. Across three benchmarks, AGORA matches or surpasses the Dawid‑Skene method, especially when evaluation sets are small or judge pools are uneven.

By Junhyung Ahn
arXiv AI
1d ago

From Certified Pairs to a Certified Hub: Reasoner-Checked Crosswalks for the Information Exchange Standard

The paper proposes a formal test for validating crosswalks between the Information Exchange Standard (IES) and thirteen ontologies across seven sector groups, using coherence and conservativity criteria borrowed from logic-based alignment repair. It details a multi-stage process—matching, human review, SKOS correspondence certification, and HermiT reasoning—that identifies and resolves inconsistencies, demonstrating that pairwise certification does not compose and that a unified hub is required for a coherent digital twin ecosystem. The study shows that merging certified building crosswalks can expose hidden conflicts, such as unsatisfiable Brick classes, and that the core hub’s consistency depends on the inclusion of certain standards like the Open Energy Ontology and BFO 2020.

By Fabio Rovai
arXiv AI
1d ago

PEEK: One-Step-Look-Ahead Exposure Bias Correction for Diffusion Sampling

The paper introduces PEEK, a training‑free deterministic sampler plugin that corrects exposure bias in diffusion image generators by performing a provisional one‑step update and interpolating between predictions at a look‑ahead state. PEEK evaluates the denoiser at this higher‑signal noisy state, thereby producing more image‑like predictions and reducing the train‑test gap that often leads to blurry or poorly detailed outputs. Experiments on eight pixel‑space and latent‑space diffusion models—including unconditional, label‑guided, and text‑guided settings—show consistent improvements over NFE‑matched baselines, with FID gains up to 49.3 % and minimal additional computation.

By Yunwei Bai, Ying Kiat Tan, Tsuhan Chen
arXiv AI
1d ago

Reconciling Privacy with Provenance for Language Model Watermarks

The paper proposes a new approach to watermarking language models that balances privacy and provenance. By shifting the focus from tracking individual texts to estimating aggregate usage, the authors develop locally differentially private watermarks that limit the evidence any single text can contribute to detection. The Red‑Green Sequence (RGS) watermark achieves accurate frequency estimation with low error rates even at stringent privacy levels (ε=1–2) across various models and realistic deployment scenarios.

By Elise L. Han, Tatsunori Hashimoto
arXiv AI
1d ago

NextGAN: Towards Simple and Scalable GAN Training for Image and Video Generation

NextGAN introduces a new training approach for GANs that regresses the critic’s input gradient onto displacements between paired reference and generated features in frozen representation spaces, effectively adding a gradient‑energy penalty while keeping the generator’s standard update. This method scales to very large pre‑trained image and video generators (up to 14 B parameters) without extra losses or teacher signals, achieving state‑of‑the‑art one‑step FIDs on ImageNet and superior VBench scores for text‑to‑video generation with a single 1.3 B model.

By Xinhao Lou, Dongye Xie, Peng Sun, Chenyi Zhuang, Kaicheng Yu, Tao Lin
arXiv AI
1d ago

IDSISA: Influence-Distilled Sharded Training for Efficient Machine Unlearning

IDSISA is a new approach to machine unlearning that builds on SISA by using influence functions to group forget‑prone samples into a few shards and then distilling knowledge from unaffected shards to recover lost utility. The method retains SISA’s exact deletion guarantee while providing an empirically bounded approximation for utility recovery (ε_cert ≤ 0.015). Experiments on several image and text models show significant improvements in selective forgetting and utility retention compared to SISA, with modest changes in unlearning time.

By Deeraj SK, Sadhana Devarajan, Tanmoy Hazra, Anupam Shukla
arXiv AI
1d ago

Dexterous Robotic Hands in the Foundation-Model Era: A Full-Stack Survey of Hardware, Data, Cross-Embodiment Transfer, Learning, and Evaluation

The survey examines dexterous robotic hands in the context of foundation models, covering hardware, data, cross-embodiment transfer, learning, and evaluation from January 2024 to November 2026. It highlights that human data’s value hinges on alignment rather than volume, that shared policies still face embodiment constraints, and that task success alone does not guarantee physical correctness or readiness for deployment. The authors identify eleven open problems and suggest reporting standards to better measure full-stack progress.

By Ji Pei
arXiv AI
1d ago

Bidirectional Centered Score Matching for Discrete Diffusion Distillation

Bidirectional Centered Score Matching (BCSM) is introduced as an efficient within-position distillation objective for discrete diffusion models, reducing the complexity from ≠|V|^2 to ≠K^2 by using an ≠K bidirectional active set. BCSM aligns relative logit geometry while maintaining a factorized ≠K objective, and it incorporates cross-timestep transition alignment. Experiments show significant efficiency gains, achieving up to 6× fewer neural function evaluations and notable improvements in HumanEval and MMMU metrics at single-step inference.

By Peng Tu, Baoyou Chen, Haoyuan Xia, Zilong Dong, Weihao Yuan, Siyu Zhu
arXiv Machine Learning
1d ago

Quantum-Inspired Tensor Network Autoencoders for Anomaly Detection: A MERA-Based Approach

The paper introduces a Multiscale Entanglement Renormalization Ansatz (MERA)-inspired autoencoder for anomaly detection in collider jets, leveraging the jets’ natural multiscale structure. It compares this architecture to dense autoencoders, tree-tensor-network limits, and classical baselines, finding that the MERA model achieves higher mean AUC with fewer trainable parameters. The study demonstrates that locality-aware hierarchical compression and disentangling layers provide a useful, parameter‑efficient inductive bias for reconstruction‑based jet anomaly detection.

By Emre Gurkanli, Michael Spannowsky
arXiv Computer Vision
1d ago

GeoBiaset: A Counterfactual Benchmark for Demographic Bias in World-Level Geolocalization

GeoBiaset is a new benchmark that tests how world-level image geolocalization models react to the apparent race of a person inserted into the image foreground. The dataset contains 9,541 images, each paired with counterfactual edits across seven races while keeping the background and ground-truth coordinates unchanged. Experiments show that demographic perturbations can cause a 16–47% drop in continent-level accuracy, and that models shift their focus from geographic landmarks to facial features when landmarks are absent.

By Niccol\`o Niccoli, Silvia Dani, Marco Mistretta, Andrew D. Bagdanov, Marco Bertini, Lorenzo Seidenari
arXiv AI
1d ago

SlackBench: Benchmarking Agents on Collaborative Projects Grounded in Real Code Repositories

SlackBench is a new benchmark that evaluates AI agents on realistic collaborative research and engineering workflows, requiring joint reasoning over workplace communication and evolving repository state. The benchmark is built from privacy‑preserving reenactments of real academic projects, providing reproducible environments with multi‑party dialogue, code changes from GitHub, and Google Docs. It contains 140 queries covering heterogeneous‑source search, abstention, summarization, and inference over long conversations, and shows that current models perform well below the ceiling, indicating significant room for improvement.

By Mingxuan Li, Huanzhi Mao, Ruitao Zheng, Jarod Stanbury James, Yunze Li, Natanya Anderson, Joseph E. Gonzalez, Alex Dimakis
arXiv AI
1d ago

SafeStart: Mitigating the Overlooked Safety Cost of Active Expert Reduction in MoE LLM Serving

The paper introduces SafeStart, a technique for Mixture-of-Experts (MoE) language models that mitigates the safety degradation caused by active expert reduction during serving. By running all experts for the prompt and the first eight generated tokens before switching to a reduced set, SafeStart largely restores the model’s refusal to harmful requests while preserving most of the speedup achieved by expert reduction. Experiments on several MoE models show that SafeStart maintains high offline decode speedups (≈85% of the original) and limits the increase in harmful responses to within a few percentage points, even under jailbreak prompts.

By Erfan Baghaei Potraghloo, Saeid Shokoufa, Seyedarmin Azizi, Arya Fayyazi, Souvik Kundu, Mehdi Kamal, Massoud Pedram
arXiv AI
1d ago

Investigating Anthropometric Fidelity in SAM 3D Body

The paper examines the SAM 3D Body model’s difficulty in accurately capturing detailed anthropometric variations—such as those seen in geriatric muscle atrophy, scoliosis, or pregnancy—despite clear visual cues. It attributes this limitation to a perception‑distortion trade‑off arising from the model’s reliance on a low‑dimensional parametric MHR representation, semantic‑invariant conditioning, and annotation‑based alignment, which collectively cause a regression‑to‑the‑mean effect. The authors propose future directions, including hybrid representations and medical‑in‑the‑loop alignment, to enhance the model’s precision for medical applications.

By Aizierjiang Aiersilan, Ruting Cheng, James Hahn
arXiv AI
1d ago

CAST-Bench: Tracing Every Sound to Its Source in Multi-Shot Audio-Video Generation

CAST-Bench is a new benchmark for multi-shot audio‑video generation that traces each generated sound back to its intended source. It contains 900 prompts across nine sound‑source categories, including both single‑shot and multi‑shot narratives, and evaluates models using 18 metrics grouped into Visual Performance, Content Consistency, and Audio & Semantic Alignment. The benchmark reveals that while visual quality is largely saturated, audio semantic scores vary significantly, and multi‑shot prompts expose challenges such as consistent speaker identity and cross‑shot event retention.

By Yucong Liu, Qi Tian, Shuyuan Tu, Kaihang Pan, Weijie Kong, Chao Pan
arXiv AI
1d ago

Unsupervised Neural Operators for Rotating Relativistic Stars: High-Fidelity Equation of State Inference and Pulsar Spin Evolution at Scale

The paper presents physics‑informed neural operators that solve the axisymmetric general relativistic hydrostatic equilibrium equations for rotating neutron stars with high accuracy and negligible computational cost. Trained in a fully unsupervised manner using only the governing partial differential equations, the solvers achieve a thousand‑fold speed‑up compared to traditional methods. This advancement enables scalable equation‑of‑state inference and spin evolution predictions that fully account for rotation and associated uncertainties in large population synthesis studies.

By Anarya Ray
arXiv AI
1d ago

AssameseLM-E2B: An Open-Weight Assamese Language Model for Efficient Consumer-Grade Deployment

The paper introduces AssameseLM-E2B, a 2‑billion‑parameter language model tailored for the low‑resource Assamese language, and its instruction‑tuned variant AssameseLM-E2B‑it. Built on the Gemma‑4 architecture, it uses an expanded byte‑level BPE vocabulary that cuts tokenization overhead by nearly 500 % compared with multilingual baselines, and employs a parameter‑efficient Continual Pre‑Training pipeline with Hybrid LoRA followed by supervised fine‑tuning on a balanced, multi‑domain dataset. Evaluation shows strong performance with a Bits‑per‑Byte score of 0.7488 on FLORES‑200, a 96.7 % RAG score, and 100 % refusal accuracy on adversarial prompts, highlighting the effectiveness of parameter‑efficient adaptation for locally deployable models under constrained resources.

By Pragyan Phukan