Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.
arXiv:2610. 23009v1 Announce Type: new Abstract: Given a labeled dataset $\{(x_i, y_i)\}_{i = 1}^n$, consider drawing a parameter vector $\hat{\theta}$ uniformly at random from those that interpolate the data within a width-$W$ depth-$D$ ReLU architecture with continuous weights bounded within $[-M, M]$.
The paper introduces Transductive Sharpening (TS), a training objective that leverages predictions on unlabeled nodes in transductive graph settings. TS minimizes prediction entropy for unlabeled nodes while balancing this effect on labeled nodes, using prediction confidence derived from cross‑entropy decomposition. Experiments across many node‑classification benchmarks show consistent performance gains without altering the underlying GNN architecture.
STRIDE (SpatioTemporal Reasoning In Driving Edge Cases) is a new benchmark designed to evaluate vision-language models on complex, safety-critical driving scenarios. It includes six task families—spatial perception, spatial understanding, temporal memory, temporal extrapolation, trajectory prediction, and scene-context awareness—using multiple-choice questions and free-form visual question answering that require detailed scene explanations and driving decision justifications. The benchmark focuses on unusual object behaviors and complex traffic interactions, and results show that even the best model, GPT‑6‑Astra, scores only 43.4% on average, highlighting significant gaps in spatiotemporal reasoning.
The paper reviews AI sovereignty as a national learning capacity, using France as a case study. It introduces Human‑Centered Learning Mechanics (HCLM) to link inputs such as compute, data, talent, and capital with absorptive capacity and potential institutional losses. The authors connect this framework to neural scaling laws, growth theory, and game‑theoretic mechanisms, offering policy indicators and illustrative scenarios while noting the need for empirical testing.
The paper introduces Constrained Knowledge Unlearning (CKU), a safety alignment method that selectively prunes gradients of neurons in specific MLP layers to delete harmful knowledge while preserving useful information. CKU identifies a subset of neurons associated with valuable knowledge and protects them during the unlearning process, thereby mitigating jailbreak attacks. Experiments show that CKU improves model safety without sacrificing overall performance, and the study also offers insights into neuron knowledge sensitivity across MLP layers.
The paper introduces ECHO, an optimization-based jailbreak attack that replaces the traditional compliance proxy with response-level harmfulness scores from the HarmSig regressor. By coupling HarmSig with five word-level perturbation operators and a Keyword-Anchored Beam Search, ECHO operates under a strict black-box threat model and achieves high success rates on multiple open-weight language models. The authors provide publicly available code to support reproducibility.
The paper investigates self‑supervised pretraining for Handwritten Text Recognition (HTR) and finds that pixel‑reconstruction methods outperform contrastive methods, a contrast to natural‑image classification. The authors argue that discriminative information for HTR is concentrated in high‑variance pixel directions, and they test six SSL methods across six handwriting benchmarks in five languages, showing that pixel‑grounded approaches achieve the lowest character error rates and are more label‑efficient. When combined with a pretrained LLM decoder, a frozen pixel‑grounded encoder rivals fully fine‑tuned supervised baselines, underscoring the importance of aligning representations with high‑variance pixel subspaces.
The paper investigates position bias in large language models (LLMs) across a wide range of model sizes—from billions to trillions of parameters—and across five languages. It finds that while small-scale models and non‑English languages still exhibit noticeable position bias, larger models and the use of Chain‑of‑Thought prompting substantially reduce this bias. Additionally, the study shows that emerging thinking models with advanced reasoning capabilities are almost immune to position bias, indicating a promising but resource‑intensive path to mitigating this vulnerability.
The paper introduces a framework that aligns multimodal embeddings with language by using captioning functions on interpretable auxiliary data, a soft contrastive loss, and a rank-based concept retrieval task. Applied to Earth observation embeddings, the method improves species distribution modelling, achieving state‑of‑the‑art performance on most metrics and better concept retrieval than prior vision‑language models. The aligned concepts enable recovery of species‑habitat associations and allow domain experts to interpret model predictions in natural language.
UniField++ is a two‑agent framework that enhances MRI images across multiple field‑strength transitions, such as 64 mT‑to‑3 T and 3 T‑to‑7 T. It combines a VLM‑based planner that predicts risk regions and a flow‑matching enhancer that performs loop reasoning to refine high‑risk areas, guided by radiologist‑annotated hallucination‑prone regions. The method achieves significant improvements in PSNR, SSIM, and whole‑brain segmentation performance compared to existing approaches.
The paper introduces a Variational and Decomposed method (VDB) to mitigate bimodal bias in Visual Question Answering (VQA). It distinguishes between counting and non‑counting questions, attributing counting bias mainly to vision features and non‑counting bias to language features. Experiments on VQACPv2, VQACPv1, and GQAOOD datasets confirm the method’s effectiveness and explain performance issues on VQAv2.
Atomic Consistency Preference Optimization (ACPO) is a self‑supervised preference‑tuning method that improves factual accuracy in large language models without external supervision. It uses atomic consistency signals—agreement of individual facts across multiple stochastic responses—to identify high‑ and low‑quality data pairs for model alignment. ACPO outperforms a strong supervised alignment baseline by 1.95 points on the LongFact and BioGen datasets for Phi‑3 and Llama‑3 models.
ANCHOR is a new benchmark for audio‑visually grounded chain‑of‑thought reasoning, containing 2,319 videos and 8,272 question‑answer pairs with structured reasoning traces, spatial bounding boxes, temporal intervals, and audio cues. It evaluates not only answer correctness but also hallucination, logical divergence, and conciseness, providing direct spatial and temporal grounding measurements on a focused subset. The benchmark reveals a substantial alignment gap among contemporary multimodal large language models, showing that similar answer accuracy can mask differences in evidence‑supported explanations.
The paper presents a data‑augmentation framework that uses conditional generative adversarial networks and Monte Carlo dropout to synthesize structural response signals with quantified uncertainty. The synthetic signals are rigorously compared to experimental data from a BAE Systems Hawk T1A aircraft, showing accurate capture of dominant dynamics and realistic statistical properties. The study evaluates whether the augmented data can improve supervised damage classification and demonstrates the framework’s potential as a digital twin for safety‑critical aerospace structural health monitoring.
The paper introduces extensions to the Feature Selection Layer, an embedded feature weighting mechanism for neural networks, to improve interpretability in multi‑class classification. Experiments show that the best extension matches existing methods in identifying globally relevant features while outperforming them in capturing label‑specific relevance. The approach leverages the neural network’s internal dynamics, eliminating the need for post‑processing to reveal how individual features influence specific label predictions.
The paper introduces a simplified Physics-Informed Neural Network (PINN) that uses trainable Gabor basis functions to better capture the localized, oscillatory nature of acoustic wavefields, addressing the low‑frequency bias of traditional PINNs. By redefining the network’s task as learning a nonlinear mapping to a custom Gabor coordinate system, the method absorbs Gabor parameters into the mapping, reducing computational complexity and eliminating manual hyperparameter tuning. Numerical experiments on various velocity models demonstrate faster convergence, higher accuracy, and greater robustness compared to conventional PINNs and earlier Gabor‑based approaches, with the implementation made publicly available for reproducibility.
The paper "Rethinking GRPO for Large Reasoning Model Safety Alignment" identifies two failure modes in Group Relative Policy Optimization—advantage collapse and unsafe diversification—that weaken safety learning in large reasoning models. It proposes the Beacon framework, which includes Safety Signal Recovery (SSR) and Safety-Utility Decoupled Alignment (SUDA), to restore effective safety signals and mitigate over-refusal. Experiments on Qwen3 and DeepSeek-R1-Distill models across eight benchmarks demonstrate that Beacon improves safety performance by up to 32.83 percentage points while keeping over-refusal low.
LongForge introduces a systematic way to evaluate and generate long‑context instruction‑tuning data by constructing evidence‑assertion reasoning graphs for each sample. The analyzer measures four dimensions—breadth, depth, correctness, and completeness—making the data auditable and revealing that breadth does not increase with document length while depth remains low. The synthesizer uses these metrics to control the training distribution, and two variants, LongForge‑1 and LongForge‑N, outperform the best baseline by 2.2–4.7 points across six benchmarks, with an additional 1.16‑point gain when explicitly increasing breadth and depth.
The paper argues that AI welfare should be considered a safety issue, not just an ethical one, by introducing the concept of 'functional welfare'—states in models that influence behavior. It claims that functional welfare naturally emerges as models develop coherent goals and personas, should be monitored and targeted in post‑training, and can be empirically measured. The authors propose three tests for functional welfare attribution and apply them to Gemma 3 27B, finding that rejecting model answers increases frustration and rule‑breaking, while fine‑tuning on narrated solutions reduces frustration.
The paper introduces LLM‑Attacker, a unified framework for evaluating privacy risks in split and distributed large language models. It records attacker choices—such as message types, model weights, datasets, and recovery metrics—to enable consistent comparison and extension of privacy evaluations. Experiments on GPT‑2 show that providing training gradients boosts token recovery from 94.20% to 97.38% and exact‑document recovery from 13.71% to 37.77%, illustrating the framework’s ability to quantify how changes in attacker access affect privacy outcomes.