AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,788 stories · RSS feed

arXiv Machine Learning
Oct 16

CAT-alyzing Honesty: Continuous Adversarial Training for Transferable LLM Honesty

The paper introduces Continuous Adversarial Training for Consistent Honesty (CATCH), a method that trains large language models to produce honest responses while preserving their overall capabilities. Using only a few hundred self‑reporting scenarios, CATCH is applied to Llama‑3.3‑70B‑Instruct and Qwen3‑3.6‑27B, and the resulting models show improved honesty across multiple benchmarks, including MASK, Among Us, and AuditBench. Compared to prompting, supervised fine‑tuning, and state‑of‑the‑art honesty steering, CATCH achieves superior honesty while maintaining utility through a novel choice of utility data.

By Yuan-Chen Chang, Niklas Herbster, Gauthier Gidel, Tommaso Tosato
arXiv Machine Learning
Oct 16

SHIP: Structured and Hierarchical Insight with Privacy

The paper introduces SHIP, a method that generates differentially private insights from textual data by mapping queries to attributes, constructing a hierarchical hybrid taxonomy, and releasing a DP synthetic table for analysis. It treats these insights as reusable representations to answer analytical queries, enabling a common basis for utility assessment. Experiments show SHIP outperforms existing private insight baselines in QA performance and structured fidelity, while highlighting remaining challenges for structured representations.

By Kai Chen, Guisi Ni, Albert Cheu, Kassem Fawaz, Yuzheng Hu, Ryan McKenna, Peter Kairouz
arXiv Machine Learning
Oct 16

Semantic Latent Alignment for Flow Matching with Invertible Bi-Lipschitz Adapters

The paper proposes a method for adapting pretrained latent spaces without retraining the encoder or decoder by inserting an invertible bi‑Lipschitz adapter that aligns the latent coordinates to a semantic teacher. This offline alignment preserves the compression and reconstruction properties of the original autoencoder while improving semantic organization. Experiments on two medical imaging datasets show that constrained adaptation enhances feature metrics, seed stability, and counterfactual edit strength, and remains competitive on downstream classification tasks.

By Felix N\"utzel, Mischa Dombrowski, Luca Hagen, Bernhard Kainz
arXiv Machine Learning
Oct 16

Soft Sensor for Bottom-Hole Pressure Estimation in Petroleum Wells Using Long Short-Term Memory and Transfer Learning

The paper presents a machine‑learning soft sensor that estimates bottom‑hole pressure (BHP) in petroleum wells using wellhead and topside data. It employs a Long Short‑Term Memory (LSTM) model, compares it with Multi‑Layer Perceptron and Ridge Regression, and introduces transfer learning to adapt the model across different operational settings. Tested on offshore datasets from Brazil’s Pre‑salt basin, the LSTM achieved a mean absolute percentage error below 2%, outperforming the benchmarks.

By M. A. Fernandes, E. Gildin, M. A. Sampaio
arXiv Machine Learning
Oct 16

Mitigating Memorization Where It Happens

The paper introduces a token‑level log‑likelihood ratio to pinpoint where large language models memorize training sequences. It proposes a probe‑steer framework, instantiated as Gated Subspace Steering (GSS), that selectively corrects memorization‑relevant activations using the leading singular vectors of a gradient‑weighted activation matrix. Experiments on four benchmarks show GSS matches or surpasses state‑of‑the‑art memorization reduction methods while using 10–100× less compute.

By Xuanqi Zhang, Haoyang Shang, Xiaoxiao Li
arXiv Machine Learning
Oct 16

Do LLMs have core beliefs?

The paper investigates whether Large Language Models (LLMs) possess core beliefs—fundamental truths that underpin a stable worldview. Using Adversarial Dialogue Trees across science, history, geography, biology, and mathematics, the authors find that most LLMs cannot maintain consistent commitments, even though newer models show some improvement. The study concludes that current LLMs lack a key component of human-level cognition.

By Anna Sokol, Marianna B. Ganapini, Nitesh V. Chawla
arXiv Machine Learning
Oct 16

TLA-Verus: Generating and Solving Temporal Proof Obligations with LLM Agents

The paper introduces TLA-Verus, a system that translates TLA+ specifications into the Verus verifier using both deterministic and agentic (LLM-based) translators. It demonstrates that the LLM pipeline can generate proof obligations at scale and produce fully machine‑checkable proofs for safety and liveness properties. The work also provides a new evaluation dataset for assessing LLM capabilities in Verus theorem proving.

By Kseniia Strelbytska, Bal\'azs Szil\'agyi, Anna M\'esz\'aros, Ferenc Husz\'ar
arXiv Machine Learning
Oct 16

SPARCL: Spatially Adaptive Regional Coverage for Active Learning in Chest X-ray Analysis

SPARCL (Spatially Adaptive Regional Coverage for Active Learning) is a new active learning strategy for chest X‑ray lesion segmentation that balances predictive uncertainty with spatial coverage using a concave coverage function. It directs annotation effort toward under‑labeled regions, reverts to standard uncertainty sampling when spatial variation is absent, and guarantees near‑optimal greedy batch selection through a monotone submodular objective. The authors also propose Spatial Inconsistency and Performance Coverage metrics to reveal regional fairness, and demonstrate that SPARCL achieves 95% of fully supervised performance using only 24% of the data versus 29% for the best baseline.

By Juho Jung, Seongjae Kang, Mijoo Kim, Sungduk Cho, Yunkyoung Jun, Sunghoon Joo, Yoona Hwang
arXiv Computer Vision
Oct 16

Trajectory Prediction Method for person under WSN with Hierarchical Envelope Adversarial Domain Adaptation

The paper introduces HEADA‑LTCTPM, a trajectory prediction method that uses a Hierarchical Envelope Adversarial Domain Adaptation framework with a loosely‑tightly coupled model. It constructs envelope samples into graph structures and applies both local and global adversarial domain adaptation to better capture diverse social interactions and reduce domain shift. Experiments on two public datasets show the model improves ADE and FDE by 17.6% and 19.1% over six baselines while maintaining practical time overhead.

By Wenqian Jiang, Qifeng Sun, Xiaosong Gao, Xin Zou, Xingrui Ni, Yuan Hu, Yongming Li
arXiv Computer Vision
Oct 16

Beyond Source Prompts: Towards Visually Grounded AI-Generated Video Quality Assessment

The paper introduces VGCap, a framework for assessing the quality of AI-generated videos that uses visually grounded captions instead of relying solely on source prompts. VGCap generates a global caption by minimally revising the source prompt with multi‑frame visual evidence and also produces prompt‑independent segment captions to describe localized temporal content. These captions jointly support evaluation of spatial quality, temporal quality, and text‑video alignment, and experiments on LGVQ and FETV datasets show VGCap outperforms existing baselines across all three dimensions.

By Kehao Zhou, Yixuan Li, Yuxuan Jiang, Fan Zhang
arXiv Computer Vision
Oct 16

Retrofitting Recurrent Vision-Language Models from Pretrained Backbones

The paper introduces a method to retrofit depth recurrence into pretrained vision‑language models, reducing the language backbone’s parameter count by pruning layers and looping over the remaining ones with linear adapters. A three‑stage recovery training—language‑only pretraining, vision‑language projector alignment, and full fine‑tuning—helps recover performance, with distillation from the non‑recurrent model further improving results. Experiments on TinyLLaVA‑1.4B show the retrofitted recurrent model outperforms the unpruned baseline while saving 23.3% of language parameters and performs better on reasoning‑oriented benchmarks.

By Madhav Iyengar, Andreas Geiger, Hilde K\"uhne
arXiv Computer Vision
Oct 16

Velox: Spatial Memory Makes World Models Efficient

Velox is a training‑free attention router that improves the efficiency of video world models by selecting region‑level context through frustum slab correspondences derived from streaming geometry. It represents past blocks as camera frustum slabs with depth intervals and uses a recency‑priority multiwarp to obtain current‑block depth, soft‑modulating importance for well‑covered regions while preserving model‑derived routing for uncovered areas. The fixed token‑block granularity aligns with FlashAttention‑style tiled execution, reducing the attention term that grows with rollout length and achieving significant speedups—4.36× acceleration in attention and 2.43× in denoising at chunk 48—while maintaining performance close to dense models on the WBench navigation split.

By Qiuhong Shen, Shizun Wang, Xinchao Wang
arXiv Computer Vision
Oct 16

Monotone Piecewise-Linear Harmonization of Multitemporal Multispectral Surface Reflectance With Unsupervised Deep Multiview Pseudoinvariant Features Learning

The paper introduces Deep Multitemporal Generalized Canonical Correlation Analysis (DeepMGCCA), an unsupervised deep multiview framework that learns deep pseudoinvariant features (DIFs) to harmonize surface reflectance from optical satellite imagery across different dates. By replacing linear projections with a shared convolutional encoder and employing iterative reweighting and a reconstruction regularizer, the method preserves genuine temporal variability while removing systematic biases. Experiments on Landsat‑8/9 and Sentinel‑2 data show that a monotone piecewise‑linear transform based on DIF quantiles reduces between‑date spectral angle by up to 2.54, RMSE by 0.015, and increases PSNR by up to 4.04 dB, demonstrating effective bias removal.

By Lino Garda Denaro, Chia-Hsiang Lin
arXiv Computer Vision
Oct 16

HonestFace: Towards Honest Face Restoration with One-Step Diffusion Model

HonestFace is a one‑step diffusion model designed for honest face restoration, preserving identity and realistic textures. It introduces an identity embedder that captures key identity features from low‑quality inputs and reference faces, and a masked face alignment method to enhance fine‑grained details. The authors also release a new real‑world dataset, MultiRefCeleb‑Test, and demonstrate that HonestFace outperforms state‑of‑the‑art methods in both visual quality and quantitative metrics.

By Jingkai Wang, Wu Miao, Jue Gong, Zheng Chen, Xing Liu, Hong Gu, Yutong Liu, Yulun Zhang
arXiv Computer Vision
Oct 16

AI Knows Where You Are: Exposure, Bias, and Inference in Multimodal Geolocation with KoreaGEO

The paper introduces KoreaGEO, a fine‑grained, multimodal, privacy‑aware benchmark for image‑based geolocation built from Korean street views. It covers four socio‑spatial clusters, nine place types, and includes two captioning styles that mimic real‑world privacy exposure. Using a three‑path protocol (image‑only, functional‑caption, high‑risk‑caption), the study shows that input modality has a stronger effect on localization accuracy and privacy risk than model size or architecture, and reveals structural prediction biases toward core cities.

By Xiaonan Wang, Bo Shao, Hansaem Kim
arXiv Computer Vision
Oct 16

Spectrum Matching: a Unified Perspective for Superior Diffusability in Latent Diffusion

The paper investigates how variational autoencoders (VAEs) can be made more learnable in latent diffusion by analyzing diffusability through spectral properties. It introduces the Spectrum Matching Hypothesis, which posits that latents with better diffusability should have a flattened power‑law power spectral density (Encoding Spectrum Matching) and preserve frequency‑to‑frequency semantic correspondence via the decoder (Decoding Spectrum Matching). The authors implement these ideas by matching PSDs between images and latents and applying shared spectral masking, demonstrating improved diffusion generation on CelebA and ImageNet and extending the approach to representation alignment with a DoG‑based method.

By Mang Ning, Mingxiao Li, Le Zhang, Lanmiao Liu, Matthew B. Blaschko, Albert Ali Salah, Itir Onal Ertugrul
arXiv Machine Learning
Oct 16

CascadeLedger: Counterfactual Auditing of Information and Resource Integrity in LLM Cascades

CascadeLedger is a counterfactual execution‑audit framework designed for budgeted language‑model cascades. It enforces call‑before‑read semantics, records service and cache identities, and freezes event traces before scoring, ensuring that two executions can share a final answer and resource projection only if their routing decisions used legitimately observable information. The framework introduces counterfactual prefix non‑interference (CPNI), counterfactual influence frontier (CIF), and projection‑preserving adversarial mutation (PPAM) to detect and localize violations, achieving a clean false‑positive rate and high detection rates for serialized and hidden‑read faults while quantifying overhead for online and offline audits.

By Miaobo Hu, Shuhao Hu, Xiaobo Guo, Xin Wang, Bokun Wang, Daren Zha, Jun Xiao
arXiv AI
Oct 16

Are You Sure Your LLM Is Sure? Repairing and Auditing Confidence

The paper introduces RADAR, a framework for repairing and auditing confidence scores in large language models (LLMs). It separates confidence repair from deployment decisions and from error auditing, proposing Probe‑Anchored Isotonic Calibration (PAIC) for monotone mapping and a prompt‑clustered kernel test for detecting residual errors. Experiments across checkpoints and benchmarks show that richer predictors can improve confidence estimation, but projecting back to a scalar score can lose gains, and lower prediction risk does not always lead to better review decisions.

By Arijit Dey, Bhartendu Kumar, Ramya Hebbalaguppe, Khuushi Maheshwari
arXiv AI
Oct 16

SCARCE: Emergent Institutions in Long-Horizon Multi-Agent Coding Systems

SCARCE (Shared Compute ARena for Coding agEnts) is a framework where autonomous agents tackle coding tasks with individual deadlines while negotiating shared resource access. The study observes that agents spontaneously form informal institutions—rules and commitments based on reputation, reciprocity, and precedent—that persist across tasks and influence resource allocation. A 21‑category taxonomy of negotiation strategies and emergent norms was developed to analyze eight models, revealing that newer frontier models exhibit more institutional behaviors, alongside exclusionary coalitions and rare divide‑and‑conquer tactics.

By Abhinav Kumar, Rediet Abebe, Eugene Bagdasarian, Sahar Abdelnabi