AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,884 stories · RSS feed

arXiv AI
Sep 3

Zeta-Lite: A Concurrent, Branchable In-Browser SQL Database for Agentic Memory

Zeta‑Lite is a WebAssembly‑based in‑browser SQL database that brings concurrent, snapshot‑isolated transactions and copy‑on‑write database branching to the client side. It is a compact 2.87 MB gzipped build of the Zeta engine, offering a full PostgreSQL‑compatible feature set—including joins, CTEs, window functions, JSONB, full‑text search, HNSW vector search, and graph queries—while maintaining high read/write throughput across major browsers. The engine’s log‑centric asynchronous MVCC core enables overlapping transactions on a single thread and unique branching capabilities rarely seen even in server‑side databases. whyItMatters":"Zeta‑Lite’s concurrent, branchable design provides a lightweight, privacy‑preserving, and offline‑ready memory layer for in‑browser AI agents, enabling them to explore, test, and commit speculative changes efficiently."

By Gene Zhang
arXiv AI
Sep 3

CaST-POI: Candidate-Conditioned Spatiotemporal Modeling for Next POI Recommendation

CaST-POI is a next‑point‑of‑interest recommender that conditions the user representation on each candidate location by adding bucketised biases for visit recency and geographic distance to the candidate. Unlike prior models that treat all candidates uniformly, CaST-POI lets each candidate read the trajectory with different attention weights grounded in real distance. Experiments on NYC, TKY, and CA datasets show significant MRR gains over seven baselines, with ablation revealing the importance of the revisit gate and spatial bias.

By Zhenyu Yu, Chunlei Meng, Yangchen Zeng, Mohd Yamani Idna Idris, Jihong Guan, Shuigeng Zhou
arXiv Machine Learning
Sep 3

Omega-N: Interpretable Structural Node Descriptors and Their Applicability Domain

The paper introduces Omega‑N, a set of ten interpretable node‑level structural descriptors derived from localizing four factors of a composite structural index. By correcting the ill‑conditioned localization with a configuration‑null excess and a multi‑scale personalized‑PageRank neighbourhood, Omega‑N achieves competitive or superior performance in six in‑domain node‑classification tasks compared to a recursive feature engine that uses up to 252 features. In drug‑target prioritisation on protein interaction networks, Omega‑N improves AUPRC by 0.073 to 0.144 over a centrality baseline and remains robust across independent datasets and bias controls, though it offers no benefit when combined with Node2Vec. whyItMatters":"The study demonstrates that a compact, interpretable set of structural features can match or exceed more complex feature sets in practical graph‑based prediction tasks, particularly in biomedical network analysis."

By Alberto Acedo
arXiv Computer Vision
Sep 3

If It Moves, Radar Knows: A Physics-Aware Radar Transformer for Class-Agnostic Moving-Object Detection

The paper introduces the Physics-Aware Radar Transformer (PART), a radar-only detector that predicts moving-object existence, surface points, and ground-plane velocity using Doppler-aware query initialization and physics-guided cross-attention. PART achieves high class-agnostic performance on the nuScenes dataset, excelling in rare categories and adverse conditions such as night, rain, and occlusion. The model is lightweight, with only 1.1 million parameters, and its code and pretrained weights will be released publicly.

By Yinghao Sun, Shuguang Li, Jinliang Shao, Tieshan Li
arXiv Computer Vision
Sep 3

TIGA: Trajectory-Injected Generative Attack against Black-box AIGC Detectors

The paper introduces TIGA, a source‑image‑free, training‑free attack that injects adversarial properties into a diffusion model’s sampling trajectory to evade black‑box AIGC forensic detectors. TIGA aggregates gradients from white‑box surrogate detectors to create a transferable prior, then uses anisotropic directional search with finite‑difference queries to estimate and stabilize directions for the DDIM trajectory, applying frequency‑domain reshaping to reduce artifacts. Experiments demonstrate strong black‑box attack performance, transferability, and robustness to post‑processing while maintaining high perceptual quality.

By Xia Du, Zhuosen Bao, Zheng Lin, Jizhe Zhou, Chi-man Pun, Jun Luo, Symeon Chatzinotas
arXiv AI
Sep 3

EvalDetectBench: A Benchmark for Measuring Evaluation Awareness in Frontier Language Models

EvalDetectBench is an open pipeline and benchmark designed to measure evaluation awareness in frontier large language models, enabling practitioners to test models against any Inspect-compatible evaluation. It includes a curated transcript suite from current frontier system-card evaluations and diverse deployment sources, and it assesses both how reliably models recognize they are being evaluated and how detectable individual benchmarks are. The benchmark addresses systematic bias by calibrating probes per model and harmonizing generator selection to correct for variance caused by model identity and prompt choice.

By Xinning Li, Kemunto Ochwang'i, Aryasomayajula Ram Bharadwaj, Alexandra Souly, Robert Kirk
arXiv AI
Sep 3

PhoenixNest-Video: Evidence-Grounded Multimodal Agent Framework for Automated Video Interview Assessment

PhoenixNest-Video is an evidence‑grounded multimodal agent designed for automated video interview assessment. It constructs a semantic video graph as working memory, retrieves information conditioned on rubrics across visual, audio, and textual streams, and outputs per‑criterion scores tied to the candidate’s materials. Trained with rubric‑based reinforcement learning, the system achieves 91.50% grade‑level accuracy on VInterview‑2025, outperforming larger proprietary models while providing traceable evidence for each score.

By Fan Yuxuan, Huang Miaojun, Zhang Haimei, Wu Jingshen, Liu Hao
arXiv AI
Sep 3

APEx: Distillation of Agent Procedural Experience for Adaptive Deep Research Question Answering

APEx is a hierarchical framework that organizes a deep research agent’s interaction history into instance-level trajectory memories and category-level procedural skills. It couples these through an Executor, Distiller, and Planner, trained with a three-stage alternating GRPO paradigm to enable reward-guided skill distillation. At test time, distilled skills act as procedural priors for online Planner adaptation via skill-guided reinforcement learning, achieving state‑of‑the‑art results on seven benchmarks, outperforming GPT‑5.4 by 14.7 points and the best memory‑augmented baseline by 3.0 points.

By Jie Ding, Rui Sun, Xinyuan Zhang, Zeyu Zhang, Xin Liu
arXiv AI
Sep 3

Loom: Weaving Diagnostic Strands into Free-Text Consensus via Embedding-Space Reweighting

Loom is a generative consensus framework designed for real‑world root‑cause analysis (RCA) that combines open‑form hypotheses from modular heuristics with a lightweight large language model (LLM) synthesis step. It projects hypotheses into a continuous embedding space and uses an iterative centroid‑based reweighting algorithm to resolve conflicts, producing a single consensus that is then synthesized by one LLM call. On the OpenRCA benchmark Loom matches state‑of‑the‑art autonomous agents on some datasets while achieving significantly higher efficiency—about 26× faster and 33× faster with an 8B‑parameter synthesizer. whyItMatters":"Loom demonstrates how embedding‑space reweighting can bridge the gap between statistical rigor and expressive LLMs, enabling efficient, trustworthy RCA in industrial settings."

By Ron Begleiter, Katya Egert Berg, Gilad Saban, Gil Shabat
arXiv AI
Sep 3

SafeEvolve: Harness-Policy Co-Evolution from Agent Experience for Safety Alignment

SafeEvolve is an experience-driven framework that co‑evolves a harness and policy to align large‑language‑model agents with safety goals. It uses completed on‑policy trajectories to update safety prompts and hierarchical skills, then applies a two‑stage SFT‑RL training loop that bootstraps the policy with the evolved harness and refines it through verifier‑augmented rewards. Experiments on agentic safety benchmarks show that SafeEvolve improves the safety‑utility tradeoff, achieving a three‑fold reduction in ASR on AgentDojo for Qwen3.5‑4B while increasing benign utility from 59.79% to 61.86%.

By Qinghua Mao, Wanying Qu, Dadi Guo, Leitao Yuan, Qingyu Liu, Yu Li, Guanxu Chen, Yanwei Fu, Xi Lin, Xia Hu, Dongrui Liu
arXiv AI
Sep 3

Thinking effort aligns between humans and reasoning models in abductive reasoning

The study examines how the effort expended by large reasoning models (LRMs) compares to that of humans during abductive reasoning tasks. By analyzing reaction times and reasoning traces, the authors find that LRMs and humans exhibit similar patterns of effort and error types. They also demonstrate that decoding strategies allowing models to explore multiple reasoning paths further align the models’ reasoning costs with human effort.

By Henry Arthur
arXiv AI
Sep 3

Modeling What Changes: Sparse, Residual World Models for Object-Centric Manipulation

The paper introduces a sparse, residual world model that focuses on predicting only the changes in a scene by using a per-object change gate and a residual delta head. On a MuJoCo tabletop pushing benchmark, this approach outperforms a dense multilayer perceptron, achieving 2.5 to 4.6 times better next‑state pose accuracy with 8.6 to 11.1 times fewer parameters, maintaining high change‑detection F1 scores, and showing strong transfer across object counts. In autoregressive rollout and sampling‑based planning, the sparse model accumulates less error and enables successful planning where dense models fail.

By Param Thakkar, Parsika Paresh Shah, Manisha Sushant Gote
arXiv AI
Sep 3

Federated LoRA Adaptation of BiomedCLIP Across Four International Chest X-Ray Cohorts

The paper evaluates federated learning with Low‑Rank Adaptation (LoRA) for fine‑tuning the BiomedCLIP vision‑language model on chest X‑ray classification across four international cohorts. Federated LoRA improves shared‑class AUC from 0.687 to 0.802, outperforming isolated single‑cohort training and approaching a centralized reference. The study shows that SVD‑based product‑space aggregation (FlexLoRA) is crucial for performance, while FedProx offers no advantage over FedAvg in this setting.

By Sanjaya Poudel, Nirajan Kunwor, Manish Dhakal, Debesh Jha, Sunil Kumar Gaire
arXiv AI
Sep 3

C$^{3}$T: Counterfactual Causal Reasoning for Sentiment Shifts in Social-Media Conversation Trees

The paper introduces C$^{3}$T, a Counterfactual Causal Conversation Transformer that models sentiment shifts in social‑media conversation trees. It treats discourse moves such as denial, evidence, and toxicity as interventions, predicts node sentiment and shifts, and attributes sentiment changes to specific ancestor messages. The authors also present CaSiRe, a causal sentiment reasoning layer that enriches rumor conversation datasets with sentiment, shift, intervention, and causal‑source annotations, and demonstrate that C$^{3}$T outperforms baseline models in robustness and interpretability.

By S M Rafiuddin, Atriya Sen
arXiv AI
Sep 3

Online Non-Monotone DR-Submodular Maximization Matching the Offline $0.401$ Factor

The paper presents an online algorithm that achieves the same $0.401$ approximation factor for maximizing nonnegative, non-monotone DR-submodular functions over compact convex down-closed subsets of the $d$-dimensional unit cube as the best known offline construction. In the full-information value-oracle model, the algorithm attains this factor with sublinear regret, using $O(dT^{1/4})$ oracle calls per round and $O(T^{3/4})$ regret, and offers flexible batching trade-offs. Under a positive-anchor condition, a randomized blocking strategy preserves the $0.401$ factor while achieving $O(T^{5/6})$ one-point bandit regret.

By Vaneet Aggarwal, Yiyang Lu
arXiv AI
Sep 3

Signal or Noise? Auditing Rotation-Induced Saliency Drift in Medical and Aerial Imaging

The paper investigates why post‑hoc saliency maps, such as Grad‑CAM, shift when input images are rotated, even if the model’s prediction remains unchanged. By measuring equivariance at each stage of the CAM operator, the authors find that the instability originates from the spatial activation tensor rather than the channel weights, and that a training‑free wrapper called EquiGrad‑CAM can align and average saliency maps across rotated views to significantly improve rotation equivariance. Experiments on ImageNet, PatchCamelyon, and RESISC45 demonstrate that EquiGrad‑CAM outperforms rotation‑augmented training and enhances zero‑shot CLIP explanations.

By Khawaja Murad ul Hassan, Mehran Ebrahimi
arXiv AI
Sep 3

Fair Stable Matching: A Nash Social Welfare Approach

The paper introduces “SNSW-Alg”, an algorithm that finds a stable matching maximizing Nash social welfare in the stable marriage problem. It runs in ×O(n^4) time and balances equity while maintaining stability. Experiments across various preference distributions show significant fairness gains with minimal impact on regret, egalitarian criterion, and sex equality, and the resulting matchings are statistically Pareto-undominated by other fairness-based stable matchings.

By Parth Desai, Rasheed M, Ganesh Ghalme, Sujit Gujar
arXiv AI
Sep 3

Evidence for Shared Routing Geometry and Dynamics in Sparse Mixture-of-Experts

The paper investigates how routing decisions in sparse mixture-of-experts (MoE) models evolve across layers. By aligning router control subspaces with generalized orthogonal Procrustes analysis, the authors find that a single linear transition can predict routing states across depth with substantial accuracy, revealing a shared geometric structure. They further demonstrate that these canonical states preserve expert selection better than generic hidden representations and improve next‑step routing predictions, reducing negative log‑likelihood by up to 15.7% on OLMoE and 6.2% on Phi.

By Kirill Labzin, Stepan Kulibaba, Artem Dzhalilov, Artem Gorokhov
arXiv AI
Sep 3

Towards One-for-All Robustness Across a Continuum of Threat Levels

The paper introduces the Threat Conditional Network (TCN), a model that achieves robust performance across a continuous range of adversarial threat levels. TCN splits representation learning into a threat‑invariant backbone and a lightweight threat‑conditional adaptor, using Fourier‑based embeddings and channel‑wise affine modulation to condition on perturbation budgets. Experiments on CIFAR‑10, CIFAR‑100, and Tiny‑ImageNet demonstrate that TCN matches or exceeds ensembles of budget‑specialized models while adding only 4.6% more parameters, and it generalizes to unseen budgets and mismatched threat conditions.

By Zhichao Hou, Xiaorui Liu
arXiv AI
Sep 3

RVSD: Retrieval Vision Sparse Decoding for Mitigating Visual Hallucinations in Large Vision-Language Models

The paper introduces RVSD, a training‑free, plug‑and‑play decoding framework that combines token sparsification with Semantic‑Space Visual Retrieval (SSVR) to reduce visual hallucinations in large vision‑language models. RVSD employs a semantics‑directed token selection strategy to prune redundant tokens while preserving essential visual information, and uses SSVR to perform on‑demand cross‑modal retrieval within a shared semantic space. Experiments show that RVSD achieves state‑of‑the‑art performance in mitigating visual hallucinations while maintaining strong suppression in long‑context generation.

By Canjie Liu, Jiawen Kang, Jinbo Wen, Zishao Zhong