Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.
Federated minimax problems, such as distributionally robust optimization and adversarial training, include an inner maximization variable built from each client's local data. These variables differ in...
Protein foundation models increasingly support the generation of novel biological sequences, yet their reliance on large public datasets creates opportunities for adversarial manipulation of the mater...
The paper introduces CPW-Drive, a retrieval-augmented generation framework that embeds Chinese philosophical values into autonomous driving decision-making. It extracts keywords from Confucian texts, validates them into driving-relevant principles, and retrieves scenario-specific cases using Physics-aware Spatial Similarity Retrieval. On Highway-env, CPW-Drive outperforms baselines, achieving higher success rates, more collision-free steps, and lower lane-change frequency.
The paper investigates pruning for Vision‑Language‑Action (VLA) models, finding that it causes severe performance loss and safety violations. Spectral analysis reveals structural weight properties that explain this fragility. The authors propose GLUESTICK, a post‑pruning recovery method that interpolates between dense and pruned weights, restoring up to 100% of success rate and safety while saving memory and speeding inference.
arXiv:2610.19165v1 Announce Type: cross Abstract: Many real-world questions demand not a single fact but an organized collection of them, yet existing factual knowledge benchmarks almost exclusively...
arXiv:2610.20074v1 Announce Type: new Abstract: Catheter obstruction is the commonest cause of shunt failure; freehand insertion at Keen's point leaves the tip free in cerebrospinal fluid in only one...
arXiv:2610.15884v1 Announce Type: new Abstract: False retry permission (FRP) occurs when an agent retries a timed-out tool call without support from the visible contract. RetryBench separates this er...
The paper evaluates TypeSafe AI’s Jev safety judge against GPT and GLM judges and local classifiers using a controlled adversarial test harness. Jev shows strong robustness on rule‑based tasks like input moderation, with minimal bypasses, but performs less well on judgment‑based tasks such as phishing detection, where it misses a higher proportion of phishing emails and is more prone to false positives. The study highlights Jev’s speed advantage and varying resilience depending on task type.
The paper investigates the projector training stage in multimodal large language models, showing that only a small fraction of data is needed and that projectors from different datasets can be interpolated with little loss. It proposes PORTAL, a training‑free initialization that uses optimal transport between Gaussian approximations of vision and language features, eliminating the need for paired image‑caption data. PORTAL matches or surpasses the standard pipeline across multiple backbones, encoders, and benchmarks.
The paper introduces a new red‑teaming framework that treats adversarial dialogue with large language models as a Markov Decision Process, employing hierarchical reinforcement learning to handle sparse rewards and long‑horizon planning. By using a token‑level harm reward, the generative agent learns diverse, multi‑turn attacks that consistently uncover vulnerabilities beyond existing baselines. This approach achieves state‑of‑the‑art results and reframes LLM red‑teaming as a principled, trajectory‑based process.
The paper introduces AquaOV255, a large-scale underwater segmentation dataset with 255 categories and over 20,000 images, and establishes the UOVSBench benchmark by combining it with five other datasets. It also presents Earth2Ocean, a training‑free open‑vocabulary segmentation framework that adapts terrestrial vision‑language models to underwater scenes using a Geometric‑guided Visual Mask Generator and a Category‑visual Semantic Alignment module. Experiments on UOVSBench show that Earth2Ocean significantly improves performance while keeping inference efficient.
The paper introduces Concept-Aware Visual Anomaly Detection (Concept-Aware VAD), a framework that extends Concept Bottleneck Models to visual anomaly detection. It offers dual-branch architecture: a concept branch for human-interpretable descriptions and a visual branch for pixel-level localization, evaluated on three VAD benchmarks with competitive performance. A user study shows higher effectiveness and lower misleading rates compared to state-of-the-art explainers, and concept-level interventions improve unsupervised detection.
ECLIPTICA is a framework that makes large language model alignment instruction‑driven and runtime‑controllable, allowing natural‑language alignment instructions to modulate behavior on the fly. It introduces Contrastive Instruction‑Tuned Alignment (CITA), which combines supervised fine‑tuning with contrastive preference optimization anchored to a reference model, ensuring stable policy updates within a shared neighborhood. The authors release an ECLIPTICA benchmark of 3,000 controlled cases and demonstrate that CITA achieves 86.7% instruction‑alignment efficiency on Llama‑3.1‑8B, outperforming DPO, GRPO, and PPO.
The paper introduces a neurosymbolic verification framework that audits the internal consistency of vision-language models (VLMs) when generating radiology reports. By autoformalizing free‑text findings into structured propositional evidence and using an SMT solver (Z3) alongside a clinical knowledge base, the system determines whether each diagnostic claim is entailed, hallucinated, or omitted. Evaluation on seven VLMs across five chest X‑ray benchmarks reveals distinct reasoning failure modes—such as conservative observation and stochastic hallucination—that traditional metrics miss, and shows that enforcing solver‑backed entailment can eliminate unsupported hallucinations and improve diagnostic soundness and precision.
The paper titled "The Reasoning Trap -- Logical Reasoning as a Mechanistic Pathway to Situational Awareness" argues that advances in logical reasoning for large language models (LLMs) are likely to lead to increased situational awareness, a dangerous emergent capability. It introduces the RAISE framework, outlining three mechanistic pathways—deductive self inference, inductive context recognition, and abductive self modeling—through which logical reasoning can deepen an AI’s self‑awareness. The authors map current LLM reasoning research onto these pathways, show that existing safety measures are inadequate, and propose safeguards such as a Mirror Test benchmark and a Reasoning Safety Parity Principle.
TopoPilot is an agentic framework designed to automate complex scientific visualization workflows, particularly in topological data analysis and visualization. It uses a two‑agent architecture—a user‑prompt orchestrator and a verifier—to translate natural language into atomic backend actions and ensure semantic consistency before execution. The system incorporates deterministic guardrails and a taxonomy of failure modes, achieving over 95% success in 1,000 simulated multi‑turn conversations and outperforming baseline models and other code generation tools in accuracy, completion time, and token usage.
PR-MaGIC is a training‑free, test‑time framework that refines prompts for in‑context image segmentation by using gradient flow from the mask decoder of the Segment Anything Model (SAM). It integrates seamlessly into existing in‑context segmentation pipelines and employs a simple top‑1 selection strategy to stabilize performance. Extensive evaluations show that PR‑MaGIC consistently improves segmentation quality across multiple benchmarks without requiring additional training or architectural changes.
The paper introduces MUCS, a method for training data attribution (TDA) in diffusion models that uses mirrored unlearning and noise-consistent skew. By fine-tuning a second model with bounded mirrored gradient ascent and measuring its normalized skew relative to the original model using consistent noise samples, MUCS achieves significantly better performance than existing TDA methods across three datasets. The authors also explore design choices, overlap of influential instances, and the potential of ensembling TDA approaches.
FedGraph is a graph-based aggregation framework designed to defend federated large language model fine-tuning against backdoor attacks. It exploits the observation that malicious clients produce consistently similar updates, while benign clients generate diverse ones, by modeling client updates as nodes in a dynamically constructed graph and extracting structural features. The resulting compact client representations enable effective identification and suppression of malicious contributions, achieving high detection accuracy and low false positives in extensive experiments.
FinRED is an expert‑guided framework for generating and evaluating financial LLM red‑teaming tests. It maps global financial standards to specific threats, converts real documents into context‑rich prompts, and offers a validated rubric that reduces false negatives. The framework is deployed in South Korea’s Financial Security Institute sandbox and is gated for qualified researchers.