arXiv Machine Learning

Adversary Brains: Hypothesis-Based Logical Signatures for Threat Actor Detection with LLMs

The paper introduces the concept of an "adversary brain," an LLM-based model that represents a threat actor’s behavior as an evolving logical signature. These signatures function as continuous detection mechanisms, matching activity patterns of campaigns rather than single indicators, and are built using the Analysis of Competing Hypotheses framework extended to thousands of hypotheses. Language models enable the creation, evaluation, and updating of these brains, allowing them to identify new indicators such as fresh command‑and‑control domains and transform unknown threats into known ones.

arXiv Machine Learning
5d ago

When AI Investigators Cannot Trust Their Tools: Conflict-Aware Evidence Verification for Cyber Threat Investigation

The paper investigates how large language model (LLM) agents that triage security alerts can be misled when the detection tools they rely on provide conflicting or incorrect evidence. By corrupting up to three of four heterogeneous detectors on the CIC-IDS2017 dataset, the authors show that naive LLM investigators become highly fragile, with accuracy dropping to as low as 13% and abstaining on up to 63% of flows. They propose Conflict‑Aware Evidence Verification (CAV), which combines an equal‑weight vote, an evidence‑conflict gate, and a nearest‑neighbour trust check; CAV consistently outperforms naive investigators across all corruption scenarios, improving accuracy to 84–88% when three detectors are corrupted while only querying the LLM on 0.4% of clean traffic.

By Yuchen Zhang
arXiv AI
Aug 25

Walking on the DARKSIDE

arXiv:2608.23370v1 Announce Type: new Abstract: Large Language Models (LLMs) recognise patterns but do not natively track the path of exclusions that a coherent discourse demands. When an input rests...

By Aldo Gangemi, Emanuele Bottazzi
arXiv Computation and Language
Oct 23

From Clinical Judgment to Checkable Rules: Clinician-Led Guardrails for Mental-Health Chatbots

The paper presents a clinician‑led safety guardrail for mental‑health chatbots that replaces typical prompt‑engineering or Constitutional AI methods. Two clinicians design, finalize, and adjudicate a rule base of four safety checks, which is formally verified for internal consistency and applied to LLM outputs. The verification layer’s verdicts are recomputable from a named rule and sixteen facts, and it consistently avoids false crisis‑resource demands that a matched LLM judge frequently raises.

By Chenhan Lyu, Kit Wislocki, Sarah Elizabeth Stoeckl, Maitreyi Sinha, Amir Rahmani
arXiv AI
Oct 5

Threat-Preserving Representation Sensitivity in Agent-Security Benchmarks

The paper introduces threat‑preserving representation sensitivity (TPRS) to assess how changes in agent‑visible wording affect attack success rates (ASR) while keeping the underlying task and evaluation criteria constant. Experiments on Agent Security Bench, MCPTox, and AgentDojo show that renaming threat‑related tools can significantly alter ASR—up to +13.21 percentage points on some models—yet may also degrade benign utility. The findings suggest that security scores based on a single representation may not generalize, urging robustness claims to be validated across multiple threat‑preserving representations.

By Neeraj Karamchandani, Piyush Nagasubramaniam, Xinhong Xie, Sencun Zhu, Dinghao Wu
arXiv AI
Oct 31

Hybrid AI Patterns: Thirteen Ways to Couple Language Models with Rules, and How to Choose Among Them

The article "Hybrid AI Patterns: Thirteen Ways to Couple Language Models with Rules, and How to Choose Among Them" presents a catalogue of thirteen design patterns for integrating language models with rule-based systems. It details each pattern’s intent, motivation, applicability, participants, collaborations, consequences, implementation, known uses, and related patterns, and identifies five invariants and five failure patterns that apply to all combinations. A decision guide is provided to help select an appropriate pattern based on problem context, supported by measurements of performance and design costs.

By Siegfried Handschuh, G\"otz-Henrik Wiegand, Lorena Raichle