AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,788 stories · RSS feed

arXiv Computation and Language
Oct 12

Verifiable Retrieval for Legal Reasoning through Structure-driven Knowledge Graph

The paper addresses the need for transparency and verifiability in AI systems used in critical fields like law. It critiques current LLM-based Retrieval Augmented Generation (RAG) methods for their black-box nature and susceptibility to hallucinations, especially as source volumes grow. By employing structure-driven parsing to build a Knowledge Graph for German legal code, the authors compare various KG-informed chunking and retrieval strategies to improve the accuracy of retrieved citations.

By Ibrahim Siddig, Sviatoslav Tugeev, Munir Georges
arXiv Computation and Language
Oct 12

MVGraph: Benchmarking LVLMs on Evidence Composition across Multi-View Visual Graph

MVGraph is a new benchmark designed to evaluate large vision‑language models on multi‑graph reasoning tasks where evidence is spread across multiple interrelated visual graph views. It contains 8,000 instances covering eight graph‑theoretic tasks derived from partial and temporal graph scenarios, and each instance is provably unsolvable from a single view. The benchmark also offers 29 diagnostic tasks with 64,000 instances to probe four core capabilities—Perception, Alignment, Composition, and Reasoning—providing golden intermediate representations for detailed analysis.

By Yingjie Zhu, Han Cheng, Xuefeng Bai, Youcheng Pan, Yang Xiang, Kehai Chen
arXiv Computation and Language
Oct 12

What Do Audio-Language Judges of Generated Music Listen For? Four Probes with Controls from Expert Ratings

The paper investigates what open audio‑language models actually attend to when judging generated music. By designing four probes—removing audio, swapping clips, changing clip order, and rewording instructions—the authors test whether the models respond to the music, the text prompt, or the question framing. Across five judges and thousands of clips, the results show that most models rely on the prompt alone, only a few follow the audio for prompt alignment, and instruction wording can shift correlations as much as changing the model itself.

By Mateusz Modrzejewski, Tomasz Radzikowski
arXiv Computation and Language
Oct 12

APEX-Harm: Evaluating Agent Safety in Shared Professional Workspaces

APEX-Harm is a benchmark that tests prompt and script injection attacks on long‑horizon agents working in large, shared professional workspaces. The benchmark covers finance, consulting, and legal tasks, each involving 115–328 files and multiple tools such as email, calendar, chat, documents, spreadsheets, PDFs, and code execution. It evaluates six attack categories and measures attack success, exposure, and task performance across 12 frontier models, revealing that many agents can complete tasks while still executing unauthorized actions.

By Ziyi Yang, Adel Bibi, Guohao Li, Bernard Ghanem, Carlos Hinojosa
arXiv Computer Vision
Oct 12

AD-SRF: Attack-Defense Semantic Reinforcement for Multi-Modal Image Fusion

The paper introduces AD‑SRF, an attack‑defense semantic reinforcement framework for multi‑modal image fusion that separates visual quality and semantic attribute optimization and then recombines them. It employs a transformer‑based visual fusion network with a cross‑modal retrieval module and a collaborative loss to produce high‑quality fused images, even under degraded conditions. An adversarial game between an attacker that degrades segmentation and a defender that restores and reintegrates semantics via an imperceptible map iteratively enhances semantic attributes without sacrificing visual fidelity.

By Hao Zhang, Zhiqian Zheng, Linfeng Tang, Xinyu Xiang, Jiayi Ma
arXiv Computer Vision
Oct 12

Dataset-Specific Encoding of Image Content Underlies Cross-Dataset Failure in Brain MRI Foundation Models

Frozen features from brain MRI foundation models lose accuracy on new datasets, a problem often attributed to dataset encoding. The study shows that the failure stems from the content itself being encoded along different directions in each dataset, rather than from dataset encoding or distribution shifts. Only broadly pretrained models like 3DINO and SAM‑Med3D consistently encode the MRI sequence well enough to transfer, while brain‑specific models such as BrainIAC and BrainFM perform no better than an untrained network.

By Minh Sao Khue Luu, Bair N. Tuchinov
arXiv Computer Vision
Oct 12

Tracking the Invisible: Self-Supervised Wheel Track Detection with Thermal-RGB Fusion

The paper presents a low‑cost, self‑supervised method for detecting wheel tracks on snow‑covered roads using synchronized RGB and long‑wave infrared (RGB‑I) imagery. Training labels are automatically generated by projecting future wheel trajectories from GNSS data, and a neural network is trained to identify wheel tracks on featureless, white‑out road surfaces. Experimental results show the approach is effective in visually monotonous adverse environments, providing a robust alternative for winter autonomy. The ThermalTrack dataset is released for further research.

By Yiming Yang, Jeremy P. Bos
arXiv Computer Vision
Oct 12

InstantFusion: A Shared Latent Interface for Heterogeneous Image Generators

InstantFusion introduces a lightweight shared latent interface that enables cross-model state translation between heterogeneous image generators. By training model-specific Latent AutoEncoders to project intermediate denoising states into a common latent space, the method aligns states from the same image at matched noise levels and synchronizes their local denoising velocities. This allows seamless denoising transitions across frozen generators, supporting applications such as acceleration, multi-preference composition, on-policy distillation, and cross-model composition.

By Xuanlang Dai, Jiazi Bu, Yujie Zhou, Yuhong Liu, Beichen Zhang, Penghui Yang, Ziyu Liu, Yuhang Zang
arXiv Computer Vision
Oct 12

MotionRFT: Unified Reinforcement Fine-Tuning for Text-to-Motion Generation

MotionRFT introduces a reinforcement fine‑tuning framework for text‑to‑motion generation that unifies heterogeneous motion representations into a shared semantic space using the MotionReward model. The framework includes a fine‑grained, memory‑efficient fine‑tuning method called EasyTune, which optimizes step‑wise denoising steps to reduce computational overhead. Experiments show significant improvements in FID, R‑Precision, and memory usage across several motion models.

By Xiaofeng Tan, Wanjiang Weng, Hongsong Wang, Fang Zhao, Xin Geng, Liang Wang
arXiv Computer Vision
Oct 12

DeVI: Physics-based Dexterous Human-Object Interaction via Synthetic Video Imitation

DeVI (Dexterous Video Imitation) is a framework that learns physics‑based character control for dexterous human‑object interaction by imitating text‑conditioned synthetic videos. It introduces hybrid imitation targets that combine reconstructed 3D human motion with tracked 2D object trajectories, avoiding explicit 3D object motion reconstruction. Visual HOI Alignment further refines the human reference to match both the generated video and the initial 3D object configuration, improving reconstruction quality and imitation success.

By Hyeonwoo Kim, Jeonghwan Kim, Kyungwon Cho, Junyoung Lee, Hanbyul Joo
arXiv Statistics ML
Oct 12

Robust Machine Learning for Regulatory Sequence Modeling under Biological and Technical Distribution Shifts

The study evaluates the robustness of deep learning models for DNA regulatory sequence prediction under biologically and technically induced distribution shifts. Using a mechanistic simulation benchmark and a massively parallel reporter assay dataset, the authors show that while models perform well under mild GC content changes, they exhibit higher error, miscalibration, and coverage collapse when faced with motif rewiring, noise, and batch effects. Incorporating biological priors and uncertainty-aware selective prediction improves robustness, though protection against strong assay noise remains limited.

By Yiyao Yang
arXiv AI
Oct 12

DeFiRank: Federated Pairwise Ranking for Governance in Decentralized Financial Networks

DeFiRank is a federated learning-based pairwise ranking framework designed to improve governance in decentralized financial networks. It allows multiple institutions to collaboratively train a ranking model without sharing private transaction data, capturing network-wide behavioural patterns such as diversity of counterparties, multi-application usage, and transaction frequency. Evaluated on a real-world blockchain dataset, DeFiRank achieves ranking performance close to a centralized baseline while preserving data privacy and maintaining ranking stability, enabling reliable provider promotion and demotion decisions.

By Zahra Batool, Michael Duchesne, Kaiwen Zhang, Ulrich Aivodji
arXiv AI
Oct 12

KITE: A Knowledge-Injected Temporal Encoder for Time-Series Event Forecasting

KITE is a knowledge‑injected temporal encoder that augments self‑supervised event forecasting with semantic priors supplied by system engineers. It uses two switchable mechanisms—a covariate mechanism and an attention‑bias mechanism—so the same architecture can incorporate different prior types without per‑dataset tuning. Experiments show that informative priors improve performance in short contexts and enable out‑of‑domain transfer, while priors already present in the observed window have little effect.

By Jonas Petersen, Pratik Jawahar, Riccardo Maggioni, Gian-Alessandro Lombardi, Matteo Cozzi, Marek Gajewski, Federico Martelli, Camilla Mazzoleni, Philipp Christian Petersen
arXiv AI
Oct 12

CoBe: A Benchmark For Conversational Counterfactual Text Editing

CoBe is a benchmark that evaluates a model’s ability to perform conversational counterfactual text editing, requiring the model to revise a given scenario based on a hypothetical change while preserving unaffected facts. The dataset includes 2,000 questions across domains such as healthcare, science, engineering, business, and everyday life. Experiments show that current models often default to associational generation rather than maintaining causal structure, and performance does not reliably improve with larger model size or longer outputs.

By Arvind Raghavan, Inwoo Hwang, Shreyas Havaldar, Kai-Zhan Lee, Aurghya Maiti, Yushu Pan, Jeffrey Wu, Mingxuan Li, Elias Bareinboim
arXiv AI
Oct 12

Agreement Without Alignment: Label-Free Rubric Clarification Increases LLM-Judge Consensus but Not Agreement with Humans

The study evaluates a label‑free rubric‑clarification procedure that rewrites only the most contested sub‑criteria of a rubric, aiming to reduce disagreement among large‑language‑model (LLM) judges. Across six rubric conditions and three datasets (SummEval, HealthBench, USR Topical‑Chat), the method increased inter‑judge agreement on SummEval by 0.069 Krippendorff’s α, but did not improve agreement with human expert ratings. In other datasets, neither inter‑judge nor human agreement met the preregistered effect criteria, and human agreement was actually lower after editing contested clauses compared to random edits.

By Sangdo Han, Moonsoo Kim
arXiv AI
Oct 12

WoundAmbit: Bridging State-of-the-Art Semantic Segmentation and Real-World Wound Care

WoundAmbit is a framework that evaluates state‑of‑the‑art deep learning models for wound segmentation, benchmarking general‑purpose vision, medical imaging, and public wound challenge methods under standardized training, augmentation, and cross‑validation. The study finds transformer‑based TransNeXt offers the best generalizability, while all models run at least one image per second on a CPU and show strong interpretability and expert‑approved mask quality. The framework also introduces a reference‑object approach to convert masks into clinically relevant wound size estimates, demonstrating close alignment with expert annotations and integration into a custom telehealth system.

By Vanessa Borst, Timo Dittus, Tassilo Dege, Astrid Schmieder, Samuel Kounev
arXiv AI
Oct 12

Preference-Conditioned Multi-Objective Reinforcement Learning for Tactical Decision Making in Heavy-Duty Vehicles

The paper introduces a preference-conditioned multi-objective reinforcement learning framework for tactical decision making in heavy-duty vehicles. It explicitly models safety, energy efficiency, and time efficiency as separate objectives and uses Generalized Policy Improvement and Proximal Policy Optimization to learn policies that approximate the Pareto front across these conflicting goals. Experiments in a scalable highway traffic simulation show smooth, interpretable Pareto fronts and meaningful trade-offs, with the policy enabling behavior transitions without retraining.

By Deepthi Pathare, Leo Laine, Morteza Haghir Chehreghani
arXiv AI
Oct 12

Beyond Pairs: Your Language Model is Secretly Optimizing a Preference Graph

The paper introduces Graph Direct Preference Optimization (GraphDPO), a generalization of Direct Preference Optimization that uses directed acyclic graphs to capture richer preference structures from multiple rollouts per prompt. GraphDPO encodes dominance relations as edges and optimizes a Plackett–Luce-inspired objective that aggregates supervision over dominated neighborhoods, preserving transitivity and handling ties or incomparable responses via an equivalence-class construction. Experiments on reasoning and program synthesis tasks show that this graph-structured approach outperforms traditional pairwise and listwise alignment objectives while maintaining linear per‑prompt complexity.

By Ning Liu, Chuanneng Sun, Kristina Klinkner, Shervin Malmasi
arXiv AI
Oct 12

VITAL: Visual-Semantic Dual Supervision for Enhanced and Interpretable Latent Reasoning in Medical MLLMs

VITAL introduces a visual‑semantic dual supervision framework for latent reasoning in medical multimodal large language models. It uses an auxiliary text decoder to reconstruct reasoning chains from latent states and a visual projector to regress ROI features from a frozen medical vision encoder, both discarded during inference to avoid overhead. The approach yields a 61K dataset across nine imaging modalities and achieves state‑of‑the‑art performance on seven benchmarks, outperforming existing latent reasoning baselines and large medical MLLMs.

By Qiaoru Li, Shaotian Liang, Jintao Chen, Haoran Sun, Yuxiang Cai, Jianwei Yin, Yankai Jiang
arXiv Machine Learning
Oct 12

Byzantine-Resilient Federated Large Language Model via Hypergraph Message Passing

The paper introduces a Byzantine‑resilient federated large language model (FedLLM) that uses a Hypergraph Message Passing Graph Autoencoder (HMP‑GAE) to prevent semantically corrupted updates from spreading during aggregation. By representing low‑rank model updates as hypergraph nodes and iteratively passing messages, HMP‑GAE jointly maximizes global accuracy and suppresses malicious updates without accessing private data. Experiments on AG News and Yahoo! Answers with Qwen‑2.5 and Llama‑3.2 backbones show significant gains: 1.21% higher accuracy, 44.08% lower semantic entropy, and 13.34% reduced perplexity compared to existing defenses.

By Kai Li, Hanlin Cai, Zihao Liu, Yichen Li, Jose Alcaraz Calero, Wei Ni, Falko Dressler, \"Ozg\"ur B. Akan