Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.
arXiv:2608.23382v2 Announce Type: replace Abstract: Instance encoding is a popular empirical technique for privacy enhancement when sharing data to an untrusted server. It transforms sensitive data t...
arXiv:2311.16139v3 Announce Type: replace-cross Abstract: Graph Neural Networks (GNNs) have become indispensable tools for learning from graph structured data, catering to various applications such a...
FARCA is a policy‑optimization framework that addresses the problem of noisy factual credit assignment in reinforcement learning for large language models. It transforms coarse factual supervision into token‑level training signals that are both localized and weighted by a reliability estimate derived from counterfactual evidence attribution. Experiments on several factual reasoning benchmarks show that FARCA improves model factuality while maintaining general reasoning abilities.
arXiv:2608.24282v1 Announce Type: new Abstract: Adaptive LiDAR scanning concentrates a limited sensing budget on regions of interest predicted from past object tracks, lowering data volume in autonom...
arXiv:2608. 12231v2 Announce Type: replace Abstract: We study adversarial combinatorial bandits with $m$-set actions, where at each round the learner selects $m$ out of $d$ items and observes only the aggregate loss of the selected items.
AgentWorld is a simulation framework that evaluates agentic information retrieval by incorporating diverse user personalities based on the Big Five (OCEAN) traits, stateful tool-use environments, and a pass$^k$ consistency metric with structured fault classification and partial-credit scoring. It includes a risk analyzer that uses Monte‑Carlo rollouts and advanced scoring methods to quantify trajectory brittleness and attack attribution. Experiments with conversational analytics, customer‑support agents, and adversarial stress‑testing demonstrate that personality variation reveals failure modes hidden by uniform testing, such as cross‑domain leakage, contextual drift, and significant quality gaps across personas.
The paper introduces AnTrap, a benchmark that injects dynamic perturbations into Android GUI agent execution to evaluate robustness against runtime anomalies. It presents a taxonomy of anomalies across four layers—State, Thinking, Action, and Round—with ten subcategories, and a pipeline that maintains task solvability while adding realistic adversarial conditions. Experiments on 16 leading GUI models show universal vulnerability, and reinforcement learning can mitigate some traps but not deep contextual ones like state deadlock.
MetaRAG introduces a belief-action aligned policy optimization framework for agentic retrieval-augmented generation (RAG). It incorporates Verify-first Action Generation and Internal Belief Probing to assess whether the current evidence is sufficient before taking an action, and uses a consistency reward gated by answer correctness to guide training. Experiments on seven public QA benchmarks demonstrate that MetaRAG improves the accuracy-efficiency trade-off over existing RL-based agentic RAG baselines, with benefits that transfer across research settings, optimizers, and model backbones.
The paper introduces Constraint‑Guided Enterprise Data Mapping (CGM), a neuro‑symbolic approach that uses schema‑grounded admissibility constraints to steer large language models (LLMs) in aligning enterprise data. CGM operates in three stages: defining constraints with metadata, generating candidates under relaxed constraints to ensure feasibility, and ranking them with a bounded LLM. Experiments show that hard constraints dramatically reduce candidate space and improve F1 scores, enabling small models to match or surpass large LLMs at a fraction of the cost while reducing expert effort.
SA-Bench is a diagnostic benchmark that evaluates how well large language model (LLM) agents reproduce scientific papers by measuring semantic alignment between generated code and the papers’ specifications. It covers 30 recent machine‑learning papers, decomposing each into 1,491 Semantic Alignment Units (SAUs) that assess numerical, methodological, protocol, and ordering fidelity. Across 12 generator configurations, the best model (Claude+PaperCoder) scores only 0.301 on average, highlighting that current LLMs often implement requirements incorrectly or with stubs.
RAGSentinel is a training‑free, label‑free defense designed for black‑box retrieval‑augmented generation systems. It employs a surrogate encoder to detect hidden‑state shifts caused by retrieved documents, removes shared topic directions, and filters poisoned documents as geometric outliers from a robust majority consensus. The method is proven to recover a poison‑free majority context under honest‑majority and representation‑separation assumptions, and experiments show it keeps attack success rates low while maintaining accuracy across multiple datasets, LLM families, and adaptive attacks.
StepGuard introduces a step-level guard model that audits and checks tool actions before execution, addressing security risks in LLM-based agents. It is trained using StepGen, an automatic engine that generates safe and unsafe trajectories, and employs Balance-GRPO to dynamically balance learning between safe and unsafe actions. Experiments show StepGuard achieves high accuracy comparable to GPT-5.4 and significantly reduces attack success rates while minimally impacting utility.
The paper introduces a dual‑dimensional framework called Automated Item Similarity Analysis (AISA) that uses Large Language Models to assess incidental content similarity in large‑scale assessments. It combines Structured Decomposition and Semantic Relatedness to capture both structural and semantic nuances that traditional metrics miss. Psychometric validation shows that LLM‑derived metrics better align with construct‑irrelevant local dependence and produce more coherent item groupings, and simulations in Computerized Adaptive Testing demonstrate improved estimation stability and reduced bias with minimal efficiency loss.
The study audits four image aesthetic scorers—LAION-Aesthetics, PickScore, ImageReward, and HPSv2—using pixel‑level interventions on skin tone and body type in both synthetic and real images. It finds that most scorers exhibit a fidelity preference: unaltered images receive the highest scores, while perturbations in either direction are penalized in an inverted‑U pattern, and this effect is largely independent of the skin operator. Synthetic‑only audits are misleading, as the apparent preference for darker skin in synthetic faces reverses or weakens when evaluated on real faces, and cross‑scorer results vary widely, underscoring the need for real‑data, within‑image causal isolation to accurately assess demographic bias.
The paper introduces CloudEmu, an automated system that builds cloud emulators by synthesizing code from cloud documentation using a neurosymbolic approach. It combines large language models for understanding documentation with symbolic abstractions to reduce hallucinations and ensure precision, and it employs the real cloud as an oracle for testing and repair. Evaluation shows CloudEmu achieves higher coverage and accuracy than the manually developed LocalStack for AWS and GCP services.
The paper introduces Evidence-Carrying Termination (ECT), a method that allows tool‑using large language model agents to declare a task complete only when a typed certificate links every required answer claim to valid, in‑scope trace evidence and a deterministic replay confirms the claimed value. In controlled experiments across 48 synthetic tasks and 576 trajectories, ECT eliminated unsafe completions and premature unsupported terminations, outperforming existing termination critics and controllers while maintaining comparable supported completion rates.
The paper presents a systematic security analysis of Google’s Agent Payments Protocol (AP2) version 2.0, focusing on its roles, transaction lifecycle, and deployment architectures. It identifies 48 threats across five attack families, scores them with the AIVSS, and demonstrates eight high‑risk threats with proof‑of‑concept attacks and mitigations. The study also introduces a deployment‑aware scanner to map threats to various checks, showing that signed mandates alone cannot guarantee user intent when pre‑authorization context is manipulated.
NeuronGuard is a fine‑tuning defense for large language models that hardens them against both jailbreak and neuron‑level attacks. It redistributes safety signals across many neurons by identifying safety‑critical ones with per‑layer linear classifiers, enforcing refusal behavior when those neurons are ablated, and applying KL‑divergence regularization for consistency. A randomized gradient projection preserves task performance, and the authors provide a formal guarantee that NeuronGuard lowers the attack success rate upper bound, with experiments showing near‑zero success rates across multiple models and attack strategies.
arXiv:2608.24524v1 Announce Type: cross Abstract: Feature attribution is a central tool of model interpretability, yet the software through which it is applied remains fragmented: individual tools sp...
arXiv:2608.24551v1 Announce Type: cross Abstract: Machine learning models are widely used in financial fraud and credit-risk detection, yet their adversarial robustness remains difficult to evaluate...