Hugging Face Trending Papers

SecLog-Bench: A Scalable Benchmark for Cybersecurity Log Reasoning with Scaffolded Evidence

Read the original on Hugging Face Trending Papers →

SecLog-Bench is a new benchmark designed to evaluate large language models on cybersecurity log reasoning. It features an extensible simulation platform covering 29 attack scenarios, uses multi‑source logs (audit, application, network traffic), and provides dense annotations with step‑by‑step evidence chains. Evaluation of eight mainstream LLMs shows modest performance, with the best model achieving 44.44% accuracy and 45.07% Macro‑F1, and even correct predictions often accompanied by flawed reasoning.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at Hugging Face Trending Papers.

arXiv AI
Jul 2

Toward Cybersecurity-Expert Small Language Models

arXiv:2510. 14113v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are transforming everyday applications, yet deployment in cybersecurity lags due to a lack of high-quality, domain-specific models and training datasets.

By Matan Levi, Daniel Ohayon, Ariel Blobstein, Ravid Sagi, Ian Molloy, Yair Allouche
arXiv AI
Aug 24

Structured but Fragile: On the Limits of LLMs in Cybersecurity Decision-Making

The paper investigates whether large language models (LLMs) can perform structured security reasoning in cybersecurity decision-making. By testing LLMs on defense selection over attack graphs from real-world threat scenarios, the study finds that LLMs can produce coherent strategies when the attack-graph structure is explicitly provided, yet their performance is fragile, highly sensitive to prompt framing, and deteriorates with increasing graph complexity. Additionally, LLM-generated solvers recover the correct high-level formulation but scale poorly compared to specialized solvers.

By Pasquale Malacaria, Yunxiao Zhang