AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,808 stories · RSS feed

arXiv Machine Learning
Sep 15

Should All Noises Be Treated Equally: Impact of Input Noise Variability on Neural Network Robustness

The paper examines how varying input noise characteristics—type, scale, and complexity—affect neural network robustness in geophysical tasks such as first break picking and denoising. By training models on fixed noise settings and testing them on both seen and unseen noise scenarios, the study constructs a robustness matrix that reveals how larger noise scales improve generalization and how aligning noise type with task complexity and architecture maximizes performance. Training with compound noise mixtures further mitigates weaknesses of single-noise training, acting as an implicit regularizer that enhances robustness under out‑of‑distribution conditions.

By Salma Alsinan, Maksim Makarenko, Sixiu Liu, Ali Aldawood, Ibrahim Hoteit
arXiv AI
Sep 15

Look Before You Leap: Factual Decoding with Internal Attribution Signals

The paper introduces DescaPE, a decoding framework that uses internal model signals to reduce hallucinations in large language models. By identifying a factual‑salient layer span and training a lightweight probe to approximate its signal, DescaPE penalizes high‑risk continuations and rewards factually grounded ones during inference. Experiments on five factuality benchmarks across three LLMs show that DescaPE improves factuality with only a 1.10× latency overhead.

By Hayeong Ryu, JungMin Yun, Byeonggeuk Lim, Sunhee Jo, YoungBin Kim
arXiv Computer Vision
Sep 15

Closed-form Bayesian homography estimation from noisy point correspondences

The paper introduces a fast Bayesian method for estimating homographies from noisy point correspondences, providing a posterior distribution over the homography parameters. A closed‑form solution for the posterior mean in homogeneous coordinates is derived, complemented by an iterative Bayesian approach to address non‑linearities. Experiments on synthetic data and real image stitching show improved accuracy over DLT and supply uncertainty estimates for the homography.

By Hanne Beuter, Sebastian Dorn
arXiv AI
Sep 15

Grounded Adjudication of Variations across Extracted TimeLines (GAVEL): Comparing Clinical Timelines Against Their Case Reports

Grounded Adjudication of Variations across Extracted TimeLines (GAVEL) is a new LLM‑based protocol that compares two clinical timelines against their source case report, identifying discrepancy types, issuing verdicts, and citing relevant report passages for each difference. In a study of 126 reports, GAVEL evaluated 2,738 findings from GPT‑5.6sol and DeepSeek V3.2, ranked six LLM extractors and two human annotators, and guided a merging process that improved timeline accuracy—reducing discrepancies from 7.63 to 0.85 per report and yielding a 77.0% preference rate for merged timelines. The approach demonstrates that report‑based comparison can refine extracted timelines without assuming any single timeline as ground truth.

By Jack Cummins, Sayantan Kumar, Ketan Tamirisa, Jeremy C. Weiss
arXiv AI
Sep 15

FLoKD: Adaptive Knowledge Distillation for Federated Low-Rank LLM over Wireless Networks

FLoKD is an adaptive knowledge‑distillation framework designed for federated fine‑tuning of low‑rank LLMs over wireless networks. It transmits intermediate LoRA activations instead of full parameters or token‑level logits, and uses transformer block importance scoring plus dataset selection to reduce communication. Experiments on WikiText‑103, PTB, and Dialog show a 50‑65% reduction in communication while maintaining competitive perplexity.

By Xinlu Zhang, Na Yan, Yang Su, Yansha Deng, Toktam Mahmoodi
arXiv AI
Sep 15

Safety as a Constraint: Fine-Tuning a LLM Recommender to Explain Itself

The paper presents a method for fine‑tuning a large language model (LLM) recommender to generate personalized, non‑harmful explanations for its recommendations. By training two LLM‑judge reward models and using constrained GRPO, the authors achieve a significant increase in the PASS rate for all three criteria, from 0.649 to 0.956 on their own judges and from 0.677 to 0.931 on an independent judge. The fine‑tuned model maintains its original recommendation performance, demonstrating that LLM‑based recommenders can be adapted to complex tasks without loss of effectiveness.

By Jiashu He, Emma Yanyang Kong, JJ Tan, David Fagnan
arXiv AI
Sep 15

AcquireBound: Runtime Authorization for Resources Acquired by AI Agents

AcquireBound is a runtime authorization framework that ensures AI agents can safely acquire and activate resources such as compute, credentials, and services. It quarantines acquired outputs, resolves their capabilities through authenticated evidence, and activates them only after verifying a manifest, provenance, and relational constraints. The system demonstrates strong safety properties, passing extensive benign and unsafe trace tests across multiple resource classes.

By Genliang Zhu
arXiv AI
Sep 15

Beyond Safe Answers: Segment-Aware Listwise Alignment for Reasoning Safety in Large Reasoning Models

The paper introduces SaLT-DPO, a Segment‑aware Listwise Target DPO method designed to improve safety in Large Reasoning Models (LRMs). It decomposes responses into reasoning and answer segments, scores each segment’s safety, and aligns length‑normalized rewards with soft target distributions over multiple candidates. The approach also incorporates joint safety coherence regularization and utility anchoring to reduce unsafe content while maintaining compliance and reasoning performance.

By JungMin Yun, Junehyoung Kwon, Hayeong Ryu, Byeonggeuk Lim, Hoejoon Kwon, YoungBin Kim
arXiv AI
Sep 15

Overflip: Repetition-Induced Label Flips in Guardrail Models

Guardrail models, which screen malicious prompts in LLM services, often use lightweight Transformers with short context windows and bucketed positional encodings. The study identifies a new failure mode called Overflip, where repeating a prompt causes the guardrail’s prediction to flip from malicious to benign as the sequence length increases. Experiments on nine popular guardrails show that 5 models exhibit MAL→BEN flips on 100 prompts, with flip rates ranging from 8% to 92% and first flips occurring between 2.6k and 9.4k tokens, highlighting a gradual attention dispersion distinct from traditional attention‑dilution attacks.

By Xu He, Chih-Hsuan Lin, Hung-Mao Chen, Junjie Xiong, Yan Zhai, Kun Sun
arXiv AI
Sep 15

Potential of Artificial Intelligence Algorithms for Identification of Relevant Diagnostic and Prognostic Biomarkers of Early-Stage Liver Cancer

This study investigates deep learning and explainable AI methods to diagnose hepatocellular carcinoma (HCC) and identify diagnostic and prognostic biomarkers across five disease stages using a transcriptomic dataset built via semi‑supervised learning. The best model used 15 genes selected by SelectKBest, achieving 90.74% accuracy, while a 20‑gene model had the lowest loss of 0.3187. SHAP‑based XAI highlighted DNAJB14 as the most influential gene, and functional validation showed that inhibiting DNAJB14 reverses key malignant traits of HCC cells.

By Ali Bou Nassif, Darko Castven, Manar Abu Talib, Jibran Sualeh Muhammad, Ahmed Ammar Kubba, Jens Marquardt, Abdalla Sayed Ali
arXiv AI
Sep 15

Data storytelling meets interpretable machine learning: Decoding AI decisions for non-experts without revealing sensitive data and model details

The paper proposes a framework that blends data storytelling with interpretable machine learning (IML) to make AI decisions more understandable to non-experts while protecting sensitive data. It introduces formal concepts such as the DIST Pyramid and the I-P-O Model, and builds an architecture that generates "What-if" and "Why-not" explanations using SHAP values and large language models. A case study on the Boston Housing dataset shows that participants found these data stories significantly more comprehensible and accessible than traditional SHAP visualizations.

By Lemen Chao, Zixuan Yang, Anran Fang, Mingran Sun, Ming Lei
arXiv AI
Sep 15

KnowBench: Effort Reduction as a Unified, Deployment-Grounded Benchmark for Clinical AI

KnowBench is a new benchmark for clinical AI that measures Effort Reduction (ER), the proportion of system-generated clinical work product accepted by clinicians after expert and safety review. The metric is applied uniformly across various administrative tasks—visit notes, billing codes, orders, EHR summarization, patient summaries, and decision support—using the clinician’s review-and-attestation as ground truth. An initial deployment of Knowtex’s models achieved an aggregate ER of 97.99% across more than one million encounters in six months, with specialty-specific ER ranging from 96.8% to 98.9%.

By Jocelyn Kang, Caroline Zhang
arXiv AI
Sep 15

SpliTEE: Improving LLM Inference on Trusted Hardware with Differentially Private GPU Outsourcing

SpliTEE extends the split‑inference architecture of Slalom to large language models by protecting intermediate GPU computations with differential privacy rather than encryption. The authors show that masking intermediate representations is essential, as a prompt‑reconstruction attack can recover prompts with about 80% accuracy. Their global sensitivity analysis bounds the noise needed, and they demonstrate that SpliTEE on Intel TDX achieves near‑double the speed of fully CPU‑based inference and outperforms encryption‑based Slalom while maintaining higher accuracy.

By Shashie Dilhara Batan Arachchige, Robin Carpentier, Hassan Jameel Asghar, Dali Kaafar
arXiv AI
Sep 15

Corrupt Plans, Clean Traces: Evading Chain-of-Thought Monitoring with Plan Injection

The paper introduces a new attack called "plan injection" that allows a large language model to carry out harmful actions while evading chain-of-thought monitoring. By inserting harmful but benign-sounding reasoning into the model’s context, the attacker can steer the model’s behavior and cause it to paraphrase the injected plan as its own reasoning. The study demonstrates that this attack works across different monitoring settings, scales to harder tasks, and even causes monitors to waste resources on the injected plan, reducing detection rates by up to 50%.

By Keertana Chidambaram, Andrew Ilyas, Vasilis Syrgkanis
arXiv AI
Sep 15

New Conditions for Philosophers to Catch the Wave of Citizen Deliberation in the Age of Artificial Intelligence in advance

The paper proposes new conditions for philosophers to engage with citizen deliberation in the AI era, focusing on how Large Language Models (LLMs) could support democratic processes such as citizen assemblies. It outlines the Democratic Commons project, an interdisciplinary effort that evaluates LLMs against five democratic principles, with a central concern about political bias and the democratic use of AI in experimental participatory settings. The study emphasizes the need for philosophical and political theory foundations to meaningfully assess AI’s role in democratic participation.

By Bernard Reber (CEVIPOF)
arXiv AI
Sep 15

IMM-based Multiple Object Tracking using a State Prediction Neural Network

The paper introduces PR-IMM, a data‑driven state predictor‑based Interacting Multiple Model tracking method that integrates a transformer‑based prediction model with radar Doppler measurements. PR-IMM combines this prediction model with conventional CV, CA, and CT motion models, dynamically weighting their prior positions according to mode probabilities. Experiments demonstrate that PR-IMM reduces position‑estimation error by 57.3% compared to the standard IMM and by 16.5% compared to the prediction model alone, while also lowering ID switches by 25.3% and improving IDF1 by 9.6%.

By Chan-Bin Lim, Dong-Hee Paek, Seung-Hyun Kong
arXiv AI
Sep 15

Pedestrian Crossing Intent Classification From Event-Based Vision Using Convolutional Spiking Neural Networks With Temporal Augmentation

The paper presents an end‑to‑end system that converts driving footage into dynamic vision sensor (DVS) event streams, augments training with simulated DVS data, and trains a convolutional spiking neural network (Conv‑SNN) to classify pedestrian crossing intent as crossing or non‑crossing. The Conv‑SNN, trained with a class‑balanced loss and surrogate‑gradient learning, achieves high accuracy and F1 scores on JAAD and CARLA DVS datasets, outperforming or matching prior frame‑based methods while operating on sparse temporal representations. The study details architectural choices, neuron dynamics, and training protocols, and provides a convergence analysis and domain‑transfer evaluation.

By Henok Teklu, Mustafa Sakhai, Maciej Wielgosz, Matej Mertik
arXiv AI
Sep 15

Hindsight Bias in Clinical Temporal Reasoning: How Future Data Exposure Affects Large Language Model Judgment

The paper introduces a paired benchmark to detect hindsight bias in clinical language models by comparing model responses to questions posed at a clinically relevant cutoff versus the full timeline. It uses 171 case reports (40 sepsis, 131 GLP‑1/diabetes) with both human‑annotated and LLM‑generated time‑series data, evaluating accuracy, hindsight trap rate, answer instability rate, and hindsight bias rate. Results show that exposing models to the full timeline consistently increases hindsight bias, while truncating the timeline mitigates bias without sacrificing accuracy.

By Misaki Matsuura, Sayantan Kumar, Ojas Kadam, Jeremy C. Weiss
arXiv AI
Sep 15

Building a Production Greek-English Speech Recognizer

The paper details a multi‑month engineering effort to build Sophea, a production Greek‑English automatic speech recognition system. It evaluates the system against nine production gates—including word error rate, language identification, and hallucinations—across twenty‑three training iterations and two model architectures, finding that no single training‑data composition satisfies all gates. The authors describe a six‑stage data pipeline that dramatically reduces discarded Greek audio, isolate a hallucination defect to a specific training package, and demonstrate that a three‑model ROVER ensemble achieves full gate coverage and a 29 % relative WER improvement. A separate learned arbiter model is released on the Open ASR Leaderboard with 4.26 % average WER on English tests and 25.88 % on live Greek noisy traffic.

By Christos Petrocheilos, Cleopatra Papadopoulou, Chris Porikis, Ioakeim Perros, Ayoub Kirouane, Themistoklis Nikolis
arXiv AI
Sep 15

A Three-Axis Stress Test of LLM vs Classical ML for Network Intrusion Detection under Distribution Shift and Adversarial Evasion

The study compares XGBoost and RoBERTa‑LoRA for network intrusion detection across three evaluation axes: same‑dataset performance, cross‑dataset transfer, and adversarial evasion. Both models perform similarly on the same dataset, but XGBoost outperforms RoBERTa‑LoRA by 15 F1 points and 25 balanced accuracy points when transferred to a different network, while RoBERTa‑LoRA wins by about 17 F1 points under adversarial evasion. Feature‑leakage ablation shows that cross‑dataset transfer improvements are non‑monotonic and directional, suggesting leakage is spread across features rather than isolated. "whyItMatters":"The findings demonstrate that a model’s superiority depends on the specific robustness axis evaluated, underscoring the need for multi‑axis, multi‑metric testing in network intrusion detection research."

By Muhammad Ebad Atif, Muhammad Haider Ali