Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.
arXiv:2610. 05337v2 Announce Type: replace-cross Abstract: We identify a mismatch between the physical role of transport fields in many PDEs and their usual role in neural operators: PDEs use them to select read coordinates, whereas neural operators typically treat them only as input values.
MADS (Multi-Agent Dialog Simulation) is a synthetic data generation pipeline that creates structured, diverse dialogue trajectories, including adversarial conversations that reflect realistic failure scenarios. The resulting MADS-Bench benchmark covers six domains, 55 personas, and 1095 conversations, with about 66% featuring failure-prone situations. Models trained with MADS are evaluated using a two-stage approach combining supervised fine-tuning and reinforcement learning with verifiable rewards to assess robustness.
The paper introduces DistressBench, a benchmark comprising 718 English clinician-authored conversations covering 23 self-harm subcategories, each evaluated against a weighted rubric across seven crisis-support dimensions. Models are scored by the fraction of criteria they satisfy, revealing that the best models meet 88.4% of weighted criteria while clinicians achieve 98.7%. The study highlights gaps in crisis response, notably that 35.3% of conversations where crisis is recognized fail to route users to human help, underscoring the need to evaluate models on supportive care rather than mere refusal.
The paper explores whether language models encode components that represent narrativity. By examining GPT‑2‑Small, Gemma‑2‑2B, and Llama‑3.1‑8B, the authors identify specific attention heads, MLP neurons, and sparse autoencoder features whose activations correlate with narrative properties. Ablating these components raises next‑token prediction loss more for high‑narrativity texts and diminishes narrativity in open‑ended generations, with the strongest effects seen in story completion and narrative question answering tasks.
CliQ‑RRG is a two‑stage framework for generating radiology reports that aligns visual and textual data using disease‑aware embeddings and prior‑guided attention to capture multi‑view dependencies. The second stage retrieves graph‑grounded clinical evidence and employs a large language model to produce concise, interpretable question‑answer style reports with diagnostic summaries. Experiments on two chest X‑ray benchmarks show that CliQ‑RRG outperforms previous methods on both lexical and clinical metrics, producing accurate and clinically coherent reports.
SharP is a text‑generation framework that uses both hard and soft prompts to guide large language models in creating task‑specific synthetic news samples for low‑resource fake news detection. The method combines structured factual constraints with learned semantic patterns, and employs a boundary‑aware strategy that focuses on classifier‑uncertain areas to refine decision boundaries. Dual objective optimization balances semantic alignment with source data and classifier feedback, aiming to improve robustness and generalization when labeled data are scarce.
The study demonstrates that large DINOv3 vision transformers, trained solely on images without any language supervision, can organize rendered words in their embedding space according to meaning. By deriving word representations from differences between word images and matched controls, the authors show that semantically related words cluster together, brand names group by industry, and country-capital pairs exhibit consistent vector offsets. These representations align with visual object categories and share structure with text and speech encoders, indicating that image-only training can impart both visual and semantic understanding of words.
OpenPA is a new automatic pronunciation assessment framework that eliminates the need for forced alignment by using Whisper’s cross‑attention to predict word‑ and utterance‑level scores. It adapts to mixed transcripts through mixed‑transcript adaptation and adaptive substitution weighting, enabling a single model to handle both closed‑ and open‑response settings. Trained only on speechocean762, OpenPA outperforms other systems on in‑domain closed and simulated open‑response tests, retains strong zero‑shot performance on MultiPA, and shows promise in oral‑cancer speech evaluation.
TokenSwap introduces a method to benchmark and reduce the modality gap in multimodal large language models (MLLMs) by interleaving semantically aligned images with text tokens. By transforming text-only benchmarks like MMLU into image‑interleaved versions (TokenSwap‑Bench), the authors reveal a pervasive performance drop—averaging 19.6%—across 42 MLLMs, with reasoning models showing smaller gaps. Training with TokenSwap mitigates this gap while maintaining strong text‑only and vision‑language performance.
The paper revisits large‑language‑model (LLM) based automatic speech recognition (ASR) from an entropy‑allocation perspective, introducing three metrics to analyze how entropy reduction is distributed between the speech encoder and the LLM. It proposes a multi‑stage training strategy that addresses entropy‑allocation inefficiencies, including a redesigned pretraining phase to reduce the speech‑text modality gap and an iterative asynchronous fine‑tuning stage to maintain functional decoupling and limit encoder drift. Experiments on Mandarin and English benchmarks demonstrate competitive performance with state‑of‑the‑art models using only 2.3 B parameters, while also reducing hallucinations through the decoupling‑oriented design.
The paper introduces SAFE‑QE, a training‑free, retriever‑agnostic method for query expansion that mitigates hallucinated terms from large language models. It uses a subsequence‑association filter to score candidate terms based on their reproducibility across randomised query contexts, down‑weighting terms that lack corpus support. SAFE‑QE then selects a broad yet consistent set of expansions via constrained coverage maximisation, and defines a new metric, the Hallucinated Expansion Term Rate, to quantify faithfulness. The approach matches state‑of‑the‑art ranking performance while injecting far fewer unsupported terms on TREC and BEIR benchmarks.
The paper investigates weighted empirical risk minimization under nonstationary data, decomposing the excess risk into a learning component and a drift component. It establishes oracle inequalities for the learning error under mixing conditions, uniformly over arbitrary weight classes, and derives explicit bounds for drift errors in square loss settings, covering structural‑break and smoothly drifting processes. The analysis is extended to auto‑regression problems with linear models, basis approximations, and neural networks, revealing that optimal weights balance learning and drift via a bias–variance trade‑off and recover minimax‑optimal rates in unweighted, stationary cases.
The paper investigates parametric change‑point detection under local differential privacy, concentrating on offline localization of a single change point in independent observations. It derives improved finite‑sample accuracy guarantees for a generalized likelihood‑ratio estimator using martingale methods and establishes finite‑sample minimax lower bounds in both non‑private and locally private settings, linking localization accuracy to Chernoff information. Guided by these bounds, the authors design locally private algorithms based on aggregated randomized response and binary mechanisms, optimize Chernoff information in different privacy regimes, and provide finite‑sample guarantees, while also computing strong data‑processing inequality coefficients for certain divergences and validating the results empirically on synthetic data.
A4-MR is a 3D vision‑language foundation model trained on about 50,000 MRI studies, designed to learn anatomy localization, align image and text representations at the anatomical level, and distinguish healthy from abnormal anatomy. It outperforms existing models on abnormality detection, knee segmentation, and abdominal organ segmentation, and localizes 3D anatomy comparably to MedSAM2 without prompts. These results show that anatomy‑level multimodal alignment and abnormality‑aware representation can support a wide range of MRI tasks with a single model.
ThinkDrag is a multimodal framework for drag-based image editing that incorporates visual reasoning to interpret sparse point constraints as meaningful object- and scene-level transformations. It can generate an explicit reasoning trace to clarify the intended edit, improving interpretability for ambiguous drags. The system introduces Drag Endpoint Markers for grounding drags in image space and is evaluated on DragBench++, a benchmark of challenging drag-based editing scenarios, achieving state‑of‑the‑art performance in generation quality and plausibility while maintaining competitive point-following precision.
FedHUG introduces a Federated Unsupervised Domain Generalization protocol to address privacy and labeling challenges in remote physiological measurement. The framework includes a Minimal Bias Aggregation module that adapts weights to heterogeneous non‑IID features and a Global Distribution‑aware Learning Controller that adjusts client training to mitigate label skew and long‑tail issues. Experiments demonstrate superior performance over state‑of‑the‑art methods using RGB video or mmWave radar data.
Controlled-Minimax is a new framework that stabilizes adversarial minimax optimization by modeling training dynamics as a second-order ODE and applying a feedback controller derived from control theory. The method guarantees local exponential stability regardless of the Hessian spectrum, addressing persistent limit cycles and divergence that plague traditional gradient descent-ascent approaches. Experiments on benchmarks such as Dirac-GAN, 2D Gaussian mixtures, Swiss Roll, and StyleGAN2-ada show that Controlled-Minimax suppresses oscillations and achieves fast, robust convergence.
PVBRF-ID is a two‑server federated learning protocol designed for oilfield enterprise networks that balances privacy and Byzantine robustness. It uses secret‑sharing of model updates, joint norm and cosine‑similarity checks against a clean root set, and cryptographic commitments to make accepted updates publicly verifiable. Experiments on a 15‑class Edge‑IIoT dataset show that PVBRF‑ID matches standard FedAvg on clean data, resists sign‑flip and adaptive poisoning attacks, and protects against update‑inference attacks while still allowing backdoor detection.
The paper introduces Temporal-Response Guidance (TRG), a method that enhances on‑policy distillation for video‑language models by aligning teacher corrections with the student’s temporal response space. TRG measures how representations change under local temporal perturbations, projects teacher corrections onto this space, and restores the original correction norm, thereby altering the supervision direction while preserving key loss properties. Experiments on TempCompass, Video‑MME, MVBench, and Video‑MMMU show that TRG boosts a 4B student’s macro accuracy by 2.86 percentage points over the baseline output‑only OPD approach.
PreCoG is an iterative framework that corrects semantic failures in text‑to‑video generation by using low‑cost previews to diagnose issues (PreQA) and then editing cached generation trajectories (PreEdit) under corrective conditions. This approach limits disruption to already correct content and only completes generation for selected candidates, reusing cached trajectories to reduce redundant computation. Experiments show that PreCoG achieves top scores on VBench and T2V‑CompBench, approaches VSR’s VS2 score at 0.56× inference time, and outperforms GENMAC on all quality and alignment metrics at 0.86× inference time.