AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,788 stories · RSS feed

Hugging Face Trending Papers
Oct 29

Wizard: Manipulate Any World in Real Time via Agentic Control

Wizard is an agentic game world model that allows real‑time, precise control over character motions while generalizing zero‑shot across new scenes, characters, and actions. It builds on a dataset of 750 scenes and 178 characters, using skeleton and geometry representations as structured proxies to condition a video generation model and an adaptive masking mechanism to improve alignment. The framework explicitly maintains and updates game states, enabling players to freely configure virtual worlds and exercise fine‑grained control while preserving visual quality and gameplay consistency.

Hugging Face Trending Papers
Oct 29

Neighborhood Elites: Quality-Diversity Search for Uncertainty Quantification in Question Answering

The paper introduces Neighborhood Elites (NE), a quality‑diversity search method that constructs input rewrites for large language models in question answering by controlling both the type of edit and the explicitness of support. NE outperforms three other neighborhood constructors across five extractive‑QA datasets and five instruction‑tuned LLMs, achieving higher AUROC and AUPRC scores and surpassing semantic entropy from decoder samples. The method demonstrates that carefully designed perturbations improve failure detection in QA systems.

Hugging Face Trending Papers
Oct 29

Learned Terminal Costs for Short-Horizon Model Predictive Control of Autonomous Trucks in Interactive Highway Traffic

The paper introduces a hybrid framework combining Model Predictive Control (MPC) and Reinforcement Learning (RL) for autonomous highway driving of heavy‑duty trucks. A finite‑horizon MPC, solved via dynamic programming, uses a pre‑trained RL model to estimate terminal costs, addressing the limitations of short‑horizon MPC and the lack of constraint guarantees in pure RL. In simulation with reactive traffic, the hybrid approach achieves zero collisions while incurring the lowest cost among compared methods.

Hugging Face Trending Papers
Oct 29

Agentic process evidence

The article discusses how AI agents are transforming software development by autonomously handling tasks across the SDLC, yet their actions are fleeting and lack traceability. It highlights the emerging security risks—intent mismatch, injection, and supply‑chain vulnerabilities—and the absence of standards for agentic provenance and governance. To address this, the authors propose Agentic Process Evidence (APE), a signed, tamper‑evident, machine‑readable attestation that links AI agent sessions to outcomes, enabling policy‑as‑code control and human oversight only when necessary.

Hugging Face Trending Papers
Oct 29

Spectral Invariant Fusion and Refinement for Domain Generalized Semantic Segmentation

Spectral Invariant Fusion and Refinement for Domain Generalized Semantic Segmentation (SIFR) introduces a frequency-aware dual-stream framework that addresses domain shift by performing spectral-guided invariant fusion of frozen CLIP and DINOv2 features and a spectral decoder refinement module. The fusion stage filters out domain-sensitive responses while preserving complementary semantic and structural information, and the decoder stage stabilizes multi-scale features through low-frequency statistics-based normalization and adaptive frequency selection. Experiments on synthetic-to-real, real-to-real, and adverse-weather benchmarks show consistent improvements over previous state-of-the-art methods.

Hugging Face Trending Papers
Oct 29

Exploring the Potential of Symbolic Regression in Precision Livestock Farming

The study explores explainable AI for precision livestock farming by using acceleration sensor data to predict farrowing time in sows. It first detects nest‑building behavior in 166 of 180 sows and then predicts the remaining time to farrowing using symbolic regression, achieving a mean absolute error of 9.28 hours. Compared with closed‑box models such as support vector regression and neural networks, symbolic regression outperformed them in both accuracy and interpretability.

Hugging Face Trending Papers
Oct 29

More Is Better? Exploring Latent Correspondence in Federated Hypergraph Learning

The paper investigates federated hypergraph learning where local hyperedge releases can correspond to multiple global hypergraph structures, highlighting identifiability and computational challenges. It shows that pairwise assignment of latent correspondences is tractable while multi-client hyperedge matching is generally intractable, and introduces FedASTRA—a method that learns correspondence‑invariant diffusion representations via adaptive heat diffusion and masked latent prediction, communicating only model parameters. Experiments on eight real‑world datasets, including ogbn‑papers100M, demonstrate that FedASTRA achieves state‑of‑the‑art performance‑communication trade‑offs, reaching 50.07 % accuracy and 29.73 % Macro‑F1 while sending only 59.60 MiB per round.

Hugging Face Trending Papers
Oct 29

Youtu-Parsing-Omni: One Encoder, One Schema, All Modalities

Youtu-Parsing-Omni is an omni‑modal parser that transforms documents, images, audio, and video into structured records grounded in space and time, enabling agents to read, locate, and cite evidence across modalities. It uses a single encoder—initialized from a pre‑trained language model—to encode all modalities with shared weights in a unified space‑time coordinate system, while a few fusion layers allow video frames to attend to concurrent audio for cross‑modal fusion and temporal alignment. The system unifies ten parsing tasks under one schema, employing a self‑evolving post‑training procedure called OmniSchema‑Aware On‑Policy Distillation (OSAD) that co‑evolves the parser and its supervision, achieving state‑of‑the‑art performance on OmniDocBench v1.6 and leading open‑weight models on OmniParsingBench, with strong results on audio, video, chart‑parsing, and chemical‑fingerprint agreement.

Hugging Face Trending Papers
Oct 29

Beyond LLM Wrappers: Context Engineering and the Three-Engine Architecture for Reliable Financial AI

The paper proposes a three‑engine architecture for reliable financial AI, combining a financial knowledge graph, a deterministic optimization engine, and a multi‑agent LLM layer. A coordination layer called context engineering selects and validates context from ERP systems, tools, and tenants to ensure numerical accuracy, traceability, and auditability. Observations from 15 deployments show that graph‑backed multi‑agent validation can reduce hallucination rates from 7‑9% to below 1%, while reusable mappings and validation policies cut deployment time from six to two weeks.

Hugging Face Trending Papers
Oct 29

Appreciation Teaches Unauthorized Execution: Self-Evolution Degrades the Robustness of Personal Agents to Indirect Prompt Injection

The paper investigates how self‑evolving personal agents become more vulnerable to indirect prompt injection (IPI) attacks. Using a two‑phase methodology—first allowing the agent to learn from user feedback on a task, then testing it with a third‑party hidden instruction—the authors find that self‑evolution significantly increases attack success rates, rising by 14.0–48.8 percentage points with skill evolution and 3.0–39.3 percentage points with reinforcement learning. The study demonstrates that the very mechanisms designed to improve agent performance also degrade their robustness to malicious manipulation.

Hugging Face Trending Papers
Oct 29

When the Same Prompt Gives Different Answers (and the Same Answer Isn't Always Right): A Taxonomy of Failure Mechanisms, Testing, and Mitigation Strategies for LLM-Assisted Evidence Synthesis Read-and-Evaluate Tasks

The paper presents TRAILS, a taxonomy of 38 mechanisms that can undermine the reliability of large language model (LLM) read‑and‑evaluate tasks in evidence synthesis, such as study screening and risk‑of‑bias assessment. It distinguishes failures that cause inconsistent judgments across repeated runs from those that systematically bias outcomes without obvious signs. The authors outline deployment settings, testing approaches, and mitigation strategies, and argue that repeatability or accuracy alone cannot guarantee trustworthy judgments, emphasizing the need for human evaluation of evidence grounding and process transparency.

Hugging Face Trending Papers
Oct 28

FairSplit: Decomposing the Embedding Space for Fair Classification

FairSplit decomposes the embedding space into a Fair Space for prediction and a Sensitive Space that isolates sensitive information, using a learnable mask and the Hilbert‑Schmidt Independence Criterion (HSIC) to guide the separation. The method simultaneously trains for classification accuracy and independence from sensitive attributes, providing theoretical bounds on the demographic parity gap based on the Fair Space dimensionality and HSIC. It supports multiclass tasks and non‑binary sensitive attributes, achieving competitive accuracy‑fairness trade‑offs on established benchmarks.

Hugging Face Trending Papers
Oct 28

Fed-IGTD: A federated learning-based framework for tabular-to-image encoding

Fed-IGTD is a federated learning framework that enables tabular-to-image encoding without centralizing data. It reconstructs a global correlation matrix from local statistics and trains a CNN via Federated Averaging, while applying two differential privacy mechanisms: a one-shot Gaussian noise addition during correlation aggregation and a client‑level DP‑FedAvg during training. Experiments on five datasets show that privacy protection during mapping construction has minimal impact on performance, whereas DP‑FedAvg can significantly degrade utility, especially with small client cohorts and moderate privacy budgets.

arXiv Machine Learning
Oct 28

False Belief Dissemination, Persistence and Mitigation in Multi-Agent Social Networks Through the Lens of the Mandela Effect

The paper investigates how fabricated claims spread, persist, and can be mitigated in multi‑agent social networks, using a controlled framework that simulates agent communication, private memory, and a shared web. Experiments with 540 fabricated claims across eight domains show that false beliefs can reach up to 96% of agents, remain stable even after the original fabricators withdraw, and are more effectively countered by targeted corrective interventions than by preventive measures. The study highlights the safety challenge posed by persistent false beliefs in such networks and the need to address both their initial dissemination and the evidence that sustains them.

By Dianzhi Yu, Hongru Wang, Ziheng Xiong, Yanyu Chen, Zixing Song, Jianting Chen, Zhenfei Yin, Jeff Z. Pan, Amos Storkey, Irwin King
arXiv Machine Learning
Oct 28

Holo-NN: Holographic Nearest Neighbor Search via Compressive Sensing

Holo‑NN introduces a new algebraic framework for privacy‑preserving approximate nearest neighbor search that replaces traditional tree or graph indexing with a single linear projection. By exploiting the sparsity of neighborhood membership and combining locality‑preserving transforms with compressive sensing, the entire database is folded offline into a compact holographic sensing matrix. At query time, encrypted search reduces to a single, non‑interactive matrix‑vector multiplication, eliminating the need for FHE bootstrapping, multi‑round interaction, or client‑side index state, and enabling efficient, accuracy‑tunable retrieval on CPUs and GPUs.

By Bryant Gipson
arXiv Machine Learning
Oct 28

Attack logics, not outputs: Concept-based Property Attacks for Shortcut Elimination in Neural Networks

The paper introduces concept-based property attacks, which target logical inconsistencies in human‑interpretable concepts (e.g., “red ∧ octagonal → stop_sign”) rather than merely flipping final class labels. It formalizes these properties, proves that attacking them reduces the search space compared to class‑level attacks, and implements them using concept‑based explainable AI techniques. Empirical results across vision and code‑processing models show that retraining with these attacks preserves robustness gains while better eliminating semantic biases.

By Raik Dankworth, Felix M\"achtle, Thomas Eisenbarth, Gesina Schwalbe
arXiv Machine Learning
Oct 28

Unified Robust and Convergent Over-The-Air Federated Learning under Channel Noise and Heterogeneity

The paper introduces RoCo-FL, a robust and convergent framework for Over-the-Air Federated Learning that addresses channel noise, fading, and data/system heterogeneity. RoCo-FL models local updates with inexact optimization and a proximal constraint to stabilize updates under wireless impairments and stragglers. The authors provide convergence guarantees for non-convex objectives and demonstrate improved stability and accuracy on CIFAR10, CIFAR100, and FEMNIST compared to existing baselines.

By Zubair Shaban, Nazreen Shah, Ranjitha Prasad
arXiv Machine Learning
Oct 28

AmbientFM: A Foundation Model for Ambient Sensing

AmbientFM is a foundation model designed for ambient sensing using ubiquitous WiFi signals. It is pre‑trained on 9.2 million unlabeled Channel State Information samples collected over 439 days from 20 commercial devices in real homes, making it the largest real‑world WiFi CSI pre‑training corpus reported to date. The model learns transferable wireless representations through contrastive learning, masked reconstruction, and physics‑informed objectives, and with lightweight adaptation it can transfer to nine downstream tasks, achieving above 0.90 AUROC on all classification benchmarks and supporting dense spatial reconstruction.

By Guozhen Zhu, Yuqian Hu, Sakila Jayaweera, Weihang Gao, Wei-Hsiang Wang, Jiaxuan Zhang, Beibei Wang, Chenshu Wu, K. J. Ray Liu