AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,788 stories · RSS feed

arXiv AI
Oct 28

Difficulty-Estimated Policy Optimization

Difficulty-Estimated Policy Optimization (DEPO) is a new framework that improves the efficiency and robustness of reasoning alignment in Large Reasoning Models. It introduces an online Difficulty Estimator that filters training data before rollouts, ensuring computational resources focus on high‑learning‑potential samples. Empirical results show DEPO achieves about 1.86× faster rollouts than DAPO and outperforms GRPO on benchmarks at similar training costs.

By Yu Zhao, Fan Jiang, Tianle Liu, Bo Zeng, Xintong Wang, Jiaxuan Teng, Yu Liu, Yichao Du
arXiv Computer Vision
Oct 28

PIS3R: Very Large Parallax Image Stitching via Deep 3D Reconstruction

PIS3R is an image stitching method designed to handle very large parallax by leveraging deep 3D reconstruction. It uses a visual geometry grounded transformer to estimate camera parameters and reconstruct a dense 3D point cloud from two input images, then reprojects this cloud onto a reference view to create an initial stitched image. A point-conditioned image diffusion module refines the result, reducing artifacts such as holes or noise.

By Muhua Zhu, Xinhao Jin, Chengbo Wang, Yongcong Zhang, Yifei Xue, Tie Ji, Yizhen Lao
arXiv Computer Vision
Oct 28

TALON: Token-Aligned Lightweight Adapters for 6-DoF Spacecraft Pose Estimation

TALON introduces Token‑Aligned Lightweight Adapters for 6‑DoF spacecraft pose estimation, inserting spatiotemporal 3D adapters before self‑attention layers of a frozen Vision Transformer. A patch‑token alignment loss grounds adapted features to keypoint structure, enabling the frozen backbone to reason over temporally enriched tokens with minimal added parameters. On the SPADES and SwissCube datasets, TALON cuts pose error by 50% and improves ADD‑0.1d accuracy by 21.8%, respectively, while also achieving significant gains in zero‑shot sim‑to‑real cross‑domain evaluation.

By Abid Ali, Nidhal Eddine Chenni, Arunkumar Rathinam, Djamila Aouada
arXiv AI
Oct 28

ADTGuard: Adaptive Depth-Time Safeguard for Streaming Inference in Large Language Models

ADTGuard is a new safeguard for large language models that monitors each token before it is streamed to users. It uses Depth-Time Safety Dynamics to combine safety signals from multiple layers and a Causal Depth Router to select the most relevant layers for each prefix. With only response‑level labels, it refines token‑level training targets and achieves strong streaming detection performance with minimal parameters and latency.

By Chuming Wei, Churui Yang, Zihan Zhang, Ruien Li, Wusheng Zhang, Yan Zhang, Guangwen Yang
arXiv AI
Oct 28

Correctness Conditioning as Selection in Chain-of-Thought Faithfulness Evaluation

The paper investigates how restricting chain‑of‑thought faithfulness evaluation to only correctly answered instances alters the measured population. Using a seven‑variable structural causal model, the authors derive an exact expression for the observed gap and distinguish between difficulty confounding and shortcut‑induced collider bias. A review of 18 papers and empirical evaluation on arithmetic tasks reveal positive selection gaps, with the GLM‑5.3 run showing a +7.4 percentage‑point increase and a pooled analysis yielding an odds ratio of 1.70.

By Yueer Pan, Yuexi Pan
arXiv AI
Oct 28

ReForest: Interpretability Meets Reasoning in Isolation Forest

ReForest is a new framework that combines the statistical efficiency of Isolation Forest with the semantic reasoning of Large Language Models. It converts Isolation Forest’s decision paths into natural‑language explanations, allowing an LLM to refine anomaly detection through an uncertainty‑aware dynamic ensemble. Experiments on healthcare and finance datasets show ReForest improves recall by 21% and F1‑score by 10%, while providing interpretable rationales grounded in statistical evidence.

By Juyoung Seo, Geonwoo Bang, Dongho Kim, Moohong Min
arXiv AI
Oct 28

Aligning Institutional Power and Operational Ability

The paper proposes a formal framework that uses process-oriented modeling and institutional power to analyze and repair normative issues in multi-agent systems. It emphasizes aligning responsibilities with the ability to fulfill them, highlighting the importance of identifying incongruences such as unsatisfiable expectations or privilege escalation. The framework also explores how the symmetrical relationship between operational processes and institutional states influences the lifecycle of normative positions.

By Andrea Marino, Giovanni Sileno, Thomas van Binsbergen, Tom van Engers
arXiv AI
Oct 28

Constraining the Commit Boundary: A DIO/ODI-Informed Authorization Guardrail for Agentic Entity Resolution

The paper introduces a Digital Identity Optimization (DIO) and Ontology of Digital Identity (ODI)-informed authorization guardrail for Agentic Entity Resolution (ER) systems. It defines a commit boundary that restricts evidence used for persistent identity changes, employing a two‑phase supervisor to ensure claim‑specific authorization, conflict resolution, scope adherence, and version validity. The work outlines a conceptual architecture and an evaluation protocol but does not provide implementation or empirical validation.

By Daniel Ber\'anek
arXiv AI
Oct 28

Uncovering Why LLMs Fail: From Internal Mechanisms to Adversarial Reasoning Trajectories

The paper introduces CauTrace, a white‑box diagnostic framework that analyzes why large language models (LLMs) fail by examining internal states, representation propagation, and temporal coordination during reasoning. It defines seven functional states to capture support and constraint processing, tracks how these states evolve across Transformer blocks, and studies the timing of commitment and verification steps. Experiments on Qwen and LLaMA-family models show distinct failure signatures, and repair/steering tests confirm that the framework’s signals align with expected changes in the inference process.

By Zirui Ding (Institute of Computing Technology, Chinese Academy of Sciences, University of Chinese Academy of Sciences), Xinxin Fan (University of Chinese Academy of Sciences, State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences), Hao Yuan (Institute of Computing Technology, Chinese Academy of Sciences, ShanghaiTech University), Shaoye Luo (University of Chinese Academy of Sciences, State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences), Men Niu (Institute of Computing Technology, Chinese Academy of Sciences, University of Chinese Academy of Sciences), Yunfeng Lu (Beihang University)
arXiv AI
Oct 28

ReCoPII: A Relation-Grounded Framework for Building Contextual PII Masking Datasets

ReCoPII is a relation‑grounded framework that generates contextual PII masking datasets without assuming a single data subject. It constructs ontology‑constrained entity relations, writes insertion sentences with placeholders, and derives MASK/KEEP labels through automated annotation and LLM verification. Applied to Korean documents across six domains, ReCoPII‑Ko achieves high agreement with human annotators (88.42% on KEEP labels) and improves masking performance compared to SPY‑style augmentation.

By Taehwan Lee
arXiv AI
Oct 28

Who Convinced Whom? Tracing How LLMs Change Their Minds in Multi-Agent Systems

The paper introduces MAD Bench, a 2,400-question benchmark for multi‑agent debate (MAD) that tests knowledge, reasoning, values, moral judgment, safety, and refusal. It evaluates three open‑weight models and finds that peer communication can alter individual decisions, with forced‑choice readouts before rationale generation more often matching the final answer in debate than in reflection. The study shows that stronger consensus can coexist with increased shared errors, suggesting the need for evaluation metrics that consider dependence and unanimous errors alongside accuracy.

By Arnesh Batra, Aniket Khandelwal, Jashn Khemani, Krish Thukral, Arush Gumber, Denis Federico Lim, Ujwal Kumar, Phan Xuan Tan
arXiv AI
Oct 28

Generation and Verification: Human-Agent Collaboration for Safety-Critical Automotive Software Engineering

The paper introduces a coding assistant tailored for safety‑critical automotive software, offering generate, edit, and verify tools via the Model Context Protocol. It customizes a large language model through prompt engineering and fine‑tuning on a dataset of automotive production code linked to requirements and UML behavior. In a two‑week pilot, developers favored the verify tool and the prompt‑engineered model, noting fewer MISRA violations in C and better architectural conformance in C++.

By Elizaveta Zinovyeva, Aiham Taleb, Nikita Kozodoi, Daniel Schleicher, Amir Mahdi Namazi, Martin Kraus, Levent Kent, Brian Jensen
arXiv AI
Oct 28

When Connectivity Misleads: Guiding Agentic Search with Contrastive Evidence Paths

The paper introduces Contrastive Evidence Paths (CEP), a method that scores search candidates by their strongest individual paths on a query‑conditioned document graph and calibrates these scores against a background visitation distribution to reduce structural bias. Building on CEP, Structured Corpus Interaction (SCI) organizes candidates and paths into a structured map, allowing agents to verify sources and expand from anchor documents. Experiments on BrowseComp‑Plus and TREC‑COVID demonstrate that SCI achieves state‑of‑the‑art answer accuracy and nDCG@10 while cutting tool calls and generation costs by over 50% and 40%, respectively.

By Junchen Li, Liang Xu, Rongzheng Wang, Qizhi Chen, Junhao Wen, Shuang Liang
arXiv AI
Oct 28

Concorde: Geometry-Aware Link Prediction via Decoupled Energy Minimization

Concorde is an energy‑based framework for link prediction that tackles sparse graphs by decomposing link likelihood into three decoupled branches: a parameter‑free spectral anchor for topological stability, a mixed‑curvature residual on a learnable hyperbolic‑spherical product manifold regularized by a curvature proxy, and a continuous vector field on the hyperbolic tangent bundle for non‑local alignment. These branches are fused via learned gating and trained with an N‑pair ranking loss. On five diverse benchmarks, Concorde achieves state‑of‑the‑art early retrieval precision, nearly doubling P@10% over leading baselines in sparse settings.

By Ege Demirci, Ambuj Singh
arXiv AI
Oct 28

MM-Net: A Multimodal Multi-Task Network for Sleep Staging and Sleep Apnea Assessment After Ischemic Stroke

MM‑Net is a multimodal, multi‑task neural network designed to simultaneously stage sleep and detect respiratory events in patients after ischemic stroke. It fuses sleep‑adapted EEG‑EOG embeddings with engineered neural descriptors and cardiorespiratory features using cross‑attention and shared temporal modeling, and employs cross‑fitting and ridge‑based embedding alignment to maintain representation consistency across patients. In a patient‑independent evaluation on the iSLEEPS dataset, MM‑Net achieved 76.3 % sleep‑stage accuracy (κ = 0.656) and a 0.825 AUC for respiratory‑event detection, outperforming baseline models; it also performed strongly on Sleep‑EDF and ISRUC‑I datasets and retained high respiratory AUCs when applied to unseen cohorts without retraining.

By Md Wahiduzzaman Suva, Esm E Moula Chowdhury Abha, Md Imtiaj Alam Sajin, Md Iftekharul Mobin, Jia Uddin
arXiv AI
Oct 28

Benchmarking LLM Agents for Behaviorally Adequate Test Generation from Software Requirements

The paper introduces Req2Test-Bench, a benchmark of 5,086 instances linking natural‑language Gherkin requirements to developer‑written test implementations across five languages and four BDD frameworks. It evaluates eight agent–model–reasoning setups, finding that while the best configuration (Codex with DeepSeek‑V4.1‑Flash) produces passing tests for 82.8% of cases, only 33.8% are behaviorally adequate, as measured by kill‑completeness. The study highlights diminishing returns from scaling agents, the impact of under‑specified requirements, and the potential of requirement‑grounded harnesses to improve test generation.

By Binhang Qi, Yiming Liu, Yun Lin, Yangjie Zhou, Hailong Sun, Jin Song Dong
arXiv AI
Oct 28

MRI-to-CT Translation of Spinal Images via Anatomy-Guided Pseudo-Pairing and Adversarial Diffusion

The study presents a three‑stage framework for generating CT images from spinal MRI scans. First, an anatomy‑guided vertebral matching system creates pseudo‑paired data by aligning similar sagittal MRI and CT slices. Second, an adversarial diffusion model is trained on these pairs to translate MRI to CT, and third, a 3D refinement network stitches the synthesized slices into coherent volumes. The method, trained on over 10,000 unpaired slices and evaluated on 1,911 slices from 83 patients, improves vertebral Dice scores from 0.666 to 0.739 and reduces discontinuities between stacked slices.

By Eduardo Nieto, Ibai Genua, Davide Scorza, Karen L\'opez-Linares
arXiv AI
Oct 28

GCSR-OCC: Geometry Calibration and Semantic Routing for Open-Vocabulary Occupancy Prediction

GCSR‑OCC introduces geometry calibration and semantic routing to improve open‑vocabulary 3D occupancy prediction from a single RGB image. It uses an Evidence‑Conserving Gaussian‑to‑Occupancy operator to mitigate scale inflation and a Multi‑Mode Query‑First Gaussian Semantic Routing module to align multiple semantic modes with text queries, refining voxel‑level predictions. The method achieves state‑of‑the‑art results on Occ‑ScanNet, improving geometric IoU to 63.31% and semantic mIoU to 27.44%, while reducing aggregation latency from 19.76 ms to 12.88 ms.

By Bohua Wang, Congyang Zhao, Tianxiong Lv, Chenxu Zhao, He Zheng, Hangqi Fan, Leigang Luo, Zhihao Liao, Yihong Guo, Yang Cai