AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,884 stories · RSS feed

arXiv Machine Learning
Sep 1

ARMOR: Manifold-Oriented Training for Adversarially Robust Aerial Object Detection under Data Scarcity

arXiv:2608.29510v1 Announce Type: cross Abstract: Aerial object detection is increasingly deployed in real-world applications, but models remain vulnerable to physical, universal adversarial patches...

By Haoran Wang, Matthew Lau, Alec Helbling, Matthew Hull, ShengYun Peng, Mansi Phute, Martin Andreoni, Willian T. Lunardi, Duen Horng Chau, Wenke Lee
arXiv AI
Sep 1

Responsible Integration of AI in Cancer Genomics: Barriers, Risks, and Pathways to Trustworthy Clinical Translation

arXiv:2608.30912v1 Announce Type: new Abstract: Artificial intelligence (AI) and natural language processing (NLP) are increasingly used to extract, integrate, and interpret biomedical knowledge rele...

By Bahar \.Ilgen, Yiannos Tolias, Denise K\"uhnert, Paraskevi Papadopoulou, Magnus Westerlund, Dominik Heider, Katharina Ladewig, Georges Hattab
arXiv Computation and Language
Sep 1

SpanCalib-VLM: Calibrated Hallucination Span Detection in Vision-Language Models

SpanCalib-VLM is a hybrid system for detecting hallucinated text spans in Vision‑Language Models. It combines a multimodal sequence tagger (XLM‑RoBERTa‑Large + SigLIP) with a fine‑tuned generative VLM (Qwen3.5‑4B‑SHROOM‑SFT) and uses a Union‑Calibrated Fusion strategy to re‑score candidate spans. On the SHROOM‑Visions English evaluation split, the ensemble achieves a Pearson calibration correlation of 0.41, an overall IoU of 0.39, a clean‑response IoU of 0.91, and a detection accuracy of 70.7%.

By Amanuel Gizachew Abebe, Yasmin Moslem
arXiv Machine Learning
Sep 1

Effective Graph and Rank-based Contextual Embeddings for Textual and Multimedia Data

arXiv:2608.29001v1 Announce Type: new Abstract: In a data-driven world, efficiently organizing and mapping relationships between objects is crucial. Graphs are powerful tools for modeling these conne...

By Thiago C\'esar Castilho Almeida, Gustavo Rosseto Let\'icio, Lucas Pascotti Valem, Andr\'e Freitas, Daniel Carlos Guimar\~aes Pedronette
arXiv AI
Sep 1

Stratified Consistency Distillation for Natural Language Formalization

The paper introduces Stratified Consistency Distillation, a fine‑tuning method to improve natural‑language to logical‑formula translations for neurosymbolic reasoning. It generates multiple candidate translations with a large LLM, clusters them by semantic equivalence, and selects pseudo‑labels using entropy‑based strategies—majority voting for low entropy, LLM-as‑Judge for medium entropy, and unification or abstention for high entropy. The selected pseudo‑labels are then used to fine‑tune a smaller model, yielding significant gains in Pass@K and a new Equivalent Logical Similarity metric.

By Zhichao Hou, Ferhat Erata, Joe Lilien, MohamadAli Torkamani
arXiv Machine Learning
Sep 1

TACS: Trajectory-Aware Candidate Selection for LLM Jailbreak Suffix Optimization

The paper introduces TACS, a trajectory‑aware candidate selection framework designed to improve jailbreak suffix optimization for large language models. Traditional gradient‑based methods choose candidates based solely on the lowest current loss, which the authors argue is myopic and can lead to reward hacking. TACS augments per‑step evaluation with a trajectory‑aware proxy, reference‑policy regularization, and a chi‑squared correction to encourage selections that remain effective beyond the immediate step. Experiments on HarmBench show that TACS consistently outperforms strong baselines, achieving higher attack success rates and more stable optimization behavior.

By Shiliang Xiao
arXiv AI
Sep 1

IndicDetect: Evaluating Cross-Lingual LLM-Generated Text Detection for Hindi, Telugu, and Tamil

IndicDetect is a benchmark for evaluating AI‑generated text detection in Hindi, Telugu, and Tamil. It pairs curated human‑written texts with LLM‑generated counterparts across multiple domains and generators, testing detectors under domain shift, generator shift, and adversarial perturbation. The study shows that supervised neural detectors fail to generalize to unseen generators and attacks, with Hindi experiencing the greatest degradation, indicating that robustness—not peak accuracy—is the main weakness in Indic language detectors.

By Bhaskar Ganesh Devalla, Junchao Wu, Nilesh Dokuparthi, Greeshma Yaluru, Tatiana Muniz Rodriguez, Lidia S. Chao, Derek F. Wong