AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,798 stories · RSS feed

arXiv Computation and Language
Sep 23

A Semiotics-Aware Framework for Evaluating Fidelity and Coverage in Natural Language Generation

The paper introduces a semiotics-aware framework for assessing natural language generation, focusing on how well two texts align in terms of contextual meaning and discourse references. It defines two metrics—Semiotic Fidelity and Semiotic Coverage—to quantify how much of one text’s semiotic profile is supported by the other and how much of the other’s profile is recovered. Experiments reveal that coverage is usually lower than fidelity, and that language models align best with human-curated data at low sampling temperatures, with higher temperatures diminishing this alignment.

By Lorenzo Zangari, Davide Picca
arXiv Computation and Language
Sep 23

Calibration as a First-Class Criterion in LLM Evaluation

The paper argues that calibration—how well a language model’s confidence aligns with its actual correctness—should be a standard evaluation metric for large language models (LLMs). It notes that while calibration metrics exist, they are rarely applied outside specialized NLP subfields, leading to unverified confidence scores in new models, datasets, and benchmarks. The authors highlight the risks of miscalibration both at deployment (overconfident errors causing harm) and in research workflows (affecting LLM-as-a-judge, synthetic data generation, and active learning). They call for every NLP subfield to pair its primary performance metric with a calibration score, treating calibration as an essential property of every model.

By Mario Sanz-Guerrero, Katharina von der Wense
arXiv Computation and Language
Sep 23

From Pattern Recognizers to Personalized Companions: A Survey of Large Language Models in Mental Health

The article surveys how large language models (LLMs) are being applied to mental health, outlining a three‑phase evolution: Phase I uses LLMs as passive information tools and pattern recognizers for assessment; Phase II employs them as empathetic conversationalists for stateless, in‑the‑moment interactions; Phase III aims to create longitudinal, personalized companions that act as stateful cognitive agents. It systematically reviews core technologies, agent architectures (Profile, Memory, Reasoning, Planning), and the datasets and benchmarks that support this progression, offering a coherent narrative and roadmap for future research. The survey also provides a curated resource list at https://github.com/Emo-gml/Awesome-Mental-Health-LLMs.

By He Hu, Yucheng Zhou, Qianning Wang, Yingjian Zou, Chiyuan Ma, Juzheng Si, Jianzhuang Liu, Zitong Yu, Laizhong Cui, Fei Ma, Qi Tian
arXiv Computation and Language
Sep 23

Geometric Uncertainty for Detecting and Correcting Hallucinations in LLMs

The paper presents a geometric framework for quantifying uncertainty in large language models (LLMs) at both the prompt and answer levels. By modeling a prompt-conditioned semantic distribution in answer embedding space and using archetypal analysis on multiple sampled answers, the method estimates distribution entropy for prompt-level uncertainty and atypicality for individual answer reliability. Experiments demonstrate comparable or superior performance to existing techniques on short-form QA datasets and notably better results on medical datasets where hallucinations pose critical risks.

By Edward Phillips, Sean Wu, Soheila Molaei, Danielle Belgrave, Anshul Thakur, David Clifton
arXiv Computer Vision
Sep 23

MirrorDistill: Illumination-Aware Latent Distillation for Efficient Low-Light Restoration

MirrorDistill introduces an illumination‑aware latent distillation framework for low‑light image enhancement. It trains a lightweight student encoder‑decoder by aligning its intermediate features with clean‑domain targets generated by a teacher decoder, using feature mirroring and illumination‑aware weighting to emphasize underexposed regions. The method achieves state‑of‑the‑art performance on the LOL‑v2‑Real benchmark while maintaining the lowest computational complexity, and the code is released as open source.

By Farida Mohsen, Tala Zaim, Nurul Izni Rusli, Ali Al-Zawqari, Ali Safa, Samir Brahim Belhaouari
arXiv Computer Vision
Sep 23

Decoupling Disease, Covariates, and Individual Variability: A Unified Disentanglement Framework for Medical Image Classification

The paper introduces MedIDL, a Medical Imaging Disentanglement Learning framework that separates disease-related features from confounding covariates and individual variability in medical images. It achieves this by projecting image features into three orthogonal latent spaces—disease classification, covariate alignment, and a Gaussian head for individual variation—using specialized disentanglement heads. Across seven diverse imaging datasets, MedIDL surpasses state‑of‑the‑art supervised and self‑supervised methods in classification accuracy, and its latent representations and gradient‑based visualizations align with known clinical patterns.

By Shengjie Zhang, Jinglin Zhang, Zhuangzhuang Jiang, Ziqi Yu, Yipin Zhang, Qi Zhang, Xiang Chen, Haibo Yang, Fei Gao, Longbiao Cui, Yuan Zhou, Xiao-Yong Zhang, Alzheimer's Disease Neuroimaging Initiative
arXiv Computer Vision
Sep 23

Fysiverse-3D-Vision Technical Report: Generating Executable 3D Worlds from Images through Unified Spatial Reasoning

Fysiverse-3D-Vision is a unified vision‑language‑geometry framework that reconstructs executable 3D scenes from a single image. It separates spatial layout reasoning from asset synthesis, using a shared representation where spatial reasoning and geometric reconstruction reinforce each other. The model employs a Transformer that integrates textual supervision, semantic visual cues, and geometric representations, and includes an object‑conditioned layout module to predict object translation, rotation, and scale while maintaining physical consistency through collision‑aware optimization.

By Dingkang Yang, Yizhou Liu, Wendong Cheng, Zizhi Chen, Shunli Wang, Yang Liu, Hongsheng Li, Lihua Zhang
arXiv Computer Vision
Sep 23

ToW3D: Consistency-aware Interactive Point-based Mesh Editing on GANs

The paper introduces ToW3D, a method for precise and consistent control over 3D generative adversarial networks (GANs) using a Tug-of-War approach between shape deformation and appearance consistency. It addresses the challenge that 3D generators often lack generalization, leading to drastic global appearance changes when editing local mesh areas. ToW3D employs a two-step optimization—drag locally and shove globally—along with a structure adaptation module and a semantic preservation module, achieving superior appearance consistency and fidelity compared to prior methods, especially under large deformations.

By Haixu Song, Fangfu Liu, Chenyu Zhang, Yueqi Duan
arXiv Computer Vision
Sep 23

The Temporal Moderation Gap: Text-to-Video Safety Filters Are Blind to Harm in Motion

The paper identifies a flaw in current text-to-video safety filters that rely on keyword prompts and per-frame checks, showing that these systems miss harms that arise solely from the ordering of frames. Experiments reveal that a significant portion of generated videos contain harmful content that is undetected because each individual frame appears safe, and prompt engineering does not mitigate this issue. The authors propose an order-aware detector that can distinguish harmful videos from benign shuffles, achieving an AUC of 0.74 compared to chance for per-frame checking.

By Yuxin Cao, Fusen Guo, Yuezhong Wu, Huadong Mo, Wei Song
arXiv Computer Vision
Sep 23

GAD-MambaUNet: Direction-Group Mamba with Gradient-Adaptive DINOv3 Distillation for Lightweight Medical Image Segmentation

The paper introduces GAD-MambaUNet, a lightweight medical image segmentation network that integrates efficient local modeling, Direction-Group Graph Selective Scan (DG‑GSS) for structured information exchange, and training‑time supervision from a frozen DINOv3 teacher with Gradient‑Adaptive Distillation. GAD‑MambaUNet demonstrates a strong accuracy‑efficiency trade‑off compared to other lightweight and general segmentation methods, and ablation studies confirm the benefits of DG‑GSS and DINOv3‑GAD supervision. Future work aims to refine teacher‑student alignment and apply the framework to multi‑class and multi‑modal medical segmentation tasks.

By Fang Wang, Huitao Li, Wenhan Chao, Zheng Zhuo, Xinxin Yang
arXiv Computer Vision
Sep 23

HARMONY: Hierarchical Agentic Reasoning for MONocular Image-to-Scene Synthesis

HARMONY is a hierarchical chain-of-thought framework that reconstructs complete 3D indoor scenes from a single monocular image. It combines agentic reasoning with visual geometry foundation models, starting with camera calibration and semantic layout recovery, then placing objects hierarchically while refining geometry with point cloud estimations. The method achieves semantically consistent scenes that align perceptually with the input image, outperforming existing baselines on synthetic and real-world data.

By Shufan Sun, Chen Wang, Enxin Song, Jiatao Gu, Lingjie Liu
arXiv Computer Vision
Sep 23

LoRango: It Takes Two LoRAs to Unlock Hidden Behaviors in Diffusion Models

arXiv:2609.25884v1 Announce Type: cross Abstract: Users commonly combine multiple Low-Rank Adaptation (LoRA) adapters to personalize images with different subjects, styles, and visual attributes. Yet...

By Jin Wei, Rundong Li, Ruihao Yang, Yikai Wang, Xiaoyuan Duan, Jianxiong Wu, Yanbo Wang, Chang Xu, Lingyun Zhang, Zhuyang Yu, Ping Chen, Jun Dai, Xiaoyan Sun