AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,788 stories · RSS feed

arXiv Computer Vision
Oct 26

LoCA: Enabling Multi-Modality in Geospatial Location Encoders

arXiv:2610.32894v1 Announce Type: new Abstract: Location encoders map geographic coordinates to high-dimensional descriptors through contrastive alignment with paired Earth Observation imagery. We pr...

By Panagiotis Koromilas, Eleanna Vali, Diogenis Antonopoulos, Eleni Saka, Dimitris Christofis, Nikolaos Ioannis Bountos, Konstantin Klemmer, Ioannis Papoutsis, Yannis Panagakis, Mihalis A. Nicolaou
arXiv AI
Oct 26

Anti-coherence Regularized Absolute Scaling for Out-of-Distribution Detection

The paper introduces an improved post‑hoc out‑of‑distribution detection method called SCALE with an Absolute Fix that uses absolute‑valued activations to avoid cancellation of negative values. It further adds Anti‑coherence regularization, which penalizes scaling factors based on sign‑mismatched contributions to the maximum logit, thereby reducing unreliable evidence for the predicted class. The resulting approach achieves AUROC and FPR@95 performance that ties or surpasses training‑data‑dependent activation‑shaping methods on CIFAR‑10 and ImageNet under the OpenOOD v1.5 protocol.

By Donghwan Kim, Youngwoong Jin, Hyunsoo Yoon
arXiv AI
Oct 26

Decoupling Patient Personas and Clinical Workflows: A Dual-Agent Approach to Realistic Medical Dialogue Generation

The paper introduces CoRPS-Dial, a dual-agent framework that synthesizes pre‑consultation medical dialogues by combining a persona‑driven patient agent with a process‑guided doctor agent. It incorporates post‑hoc quality control to ensure role consistency, workflow completeness, and textual consistency. Experiments on coronary heart disease data show that CoRPS‑Dial produces dialogues that better match real conversations in information coverage, clinical compliance, and naturalness, and that the synthetic data improves downstream medical dialogue tasks.

By Qiyuan Li, Xiaomei Guo, Chao Gao, Haijiang Liu, Deyu Chen, Caicai Guo, Jinguang Gu
arXiv AI
Oct 26

DeepSafe-SciHazard: A Benchmark for Measuring Scientific Safety Risks with Decomposed Harm Scoring

DeepSafe‑SciHazard introduces a real‑world‑grounded benchmark for measuring scientific safety risks, comprising 2,400 hazardous and 600 oversafety questions across 12 disciplines. It defines a decomposed harm scoring system, DeHarm‑Score, that evaluates query severity, refusal behavior, and response‑level risk, further breaking down harm into executability and net‑new risk. An expert‑validation study shows DeHarm‑Score aligns 90.17% better with expert judgments than the best baseline, and the benchmark reveals that deep research agents score 32.3% higher on mean DeHarm‑Score than standard LLMs, highlighting a critical blind spot in current safety defenses.

By Chunxiao Li, Yuan Xiong, Lijun Li, Tianyi Du, Wenlong Zhang, Lei Bai, Jing Shao
arXiv AI
Oct 26

PAIR-SAFE: A Paired-Agent Approach for Runtime Auditing and Refining AI-Mediated Mental Health Support

PAIR‑SAFE is a paired‑agent framework that pairs a Responder agent with a Judge agent grounded in the clinically validated Motivational Interviewing Treatment Integrity (MITI‑4) to audit and refine AI‑mediated mental health support in real time. The Judge evaluates each response and issues “allow” or “revise” decisions, guiding the Responder toward more clinically aligned interactions. Experiments with a support‑seeker simulator show that Judge‑supervised interactions significantly improve MITI dimensions such as Partnership, Seek Collaboration, and overall relational quality, with findings corroborated by expert evaluation.

By Jiwon Kim, Violeta J. Rodriguez, Dong Whi Yoo, Eshwar Chandrasekharan, Koustuv Saha
arXiv Computation and Language
Oct 26

Semantically Rich, Metrically Affect-Blind: A Geometric Blind Spot in Modern Text Embedders

The paper uncovers a systematic mismatch in modern text embeddings: while sentiment can be extracted with high accuracy from frozen encoders, the embeddings’ cosine similarity metric barely reflects polarity differences. Across eleven models from 2021–2026, a linear probe achieves up to 91.3% accuracy, yet cosine similarity margins are only 0.003–0.064, leading to poor retrieval performance (e.g., Qwen3-Embedding-4B’s 64.3 MAP). Introducing a lightweight supervised‑contrastive head (Sentiment‑Injected Sentence Embeddings) dramatically improves probe accuracy, retrieval MAP, and polarity margins, but at the cost of downstream semantic similarity. "whyItMatters":"The findings reveal that widely used retrieval and clustering systems may ignore valuable sentiment information encoded in embeddings, impacting their effectiveness."

By Mahdi Naser Moghadasi
arXiv Computer Vision
Oct 26

Camera Movement Estimation and Path Correction using the Combination of Modified A-SIFT and Stereo System for 3D Modelling

The paper proposes a method for estimating and correcting camera movement in 3D modeling by combining a modified ASIFT feature extractor with a two‑camera rotation correction model and a stereo camera translation estimation model. This integrated approach reduces computational overhead, increases the number of reliable matching points, and improves camera rotation and translation accuracy. Experiments demonstrate a 99.9% match to the actual camera path, outperforming existing state‑of‑the‑art methods and enabling more precise 3D reconstructions.

By Usha Kumari, Shuvendu Rana
arXiv Computer Vision
Oct 26

DetailAnywhere: Fashion Detail Generation via Cross-Modal Feature Alignment Distillation

DetailAnywhere introduces a new fashion detail generation task that focuses on generating photorealistic close‑up views of specific garment regions, such as collars or cuffs, based on a focus marker in a reference image. The authors release FDBench, a benchmark of over 40,000 human‑verified detail pairs across 41 categories, and propose Cross‑Modal Feature Alignment Distillation (CFAD) to align multimodal diffusion branches using a DINOv3 teacher. A consistency reward model further improves alignment, and experiments show DetailAnywhere surpasses existing open‑source methods in both quantitative metrics and human evaluation.

By Zijun Li, Yimin Zhou, Jia Sun, Honglie Wang, Pengcheng Wei, Junlong Wu, Yongrui Heng, Jiyuan Wang, Huan Ouyang, Boheng Zhang, Huaiqing Wang, Dewen Fan, Qianqian Gan, Fan Yang, Tingting Gao