arXiv AI September 10, 2026 By Hanna Kim, Jian Cui, Minkyoo Song, Hwanjo Heo, Seungwon Shin, Kimin Lee, Xiaojing Liao

SCRIPTIOC-BENCH: A Benchmark for Recognizing Actionable Threat Intelligence from Script-Based Malware using LLMs

SCRIPTIOC-BENCH is a benchmark designed to evaluate how well large language models can statically extract indicators of compromise (IOCs) from script-based malware. It contains 634 manually verified JavaScript, PowerShell, and VBScript samples and covers four IOC types—URLs, domains, IP addresses, and filesystem artifacts—while distinguishing between directly exposed and encoded indicators. Experiments show that even the best models achieve only 65.4 F1, and a false‑positive taxonomy is introduced to analyze error patterns, with two mitigations (deterministic string utilities and task‑specific adaptation) improving precision and shifting errors toward sample‑grounded mismatches.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Hugging Face Trending Papers

Sep 5

SCRIPTIOC-BENCH: A Benchmark for Recognizing Actionable Threat Intelligence from Script-Based Malware using LLMs

Script-based malware remains a prevalent attack technique. These scripts often contain indicators of compromise (IOCs) that provide actionable threat intelligence. However, statically recovering such...

llms benchmarks

arXiv AI

Jul 1

AI-Generated PowerShell Malware: An Experimental Framework and Dataset

arXiv:2606. 30819v1 Announce Type: cross Abstract: Generative AI has emerged as a significant cybersecurity threat, with several recent attack campaigns leveraging LLMs to generate code for malicious purposes via scripting languages such as PowerShell.

By Luciano Pianese, Vittorio Orbinato, Pietro Liguori, Roberto Natella

llms

arXiv AI

Jun 19

Multi-View Decompilation for LLM-Based Malware Classification

arXiv:2606. 20436v1 Announce Type: cross Abstract: Malware analysts often inspect compiled binaries through decompiled pseudo-C, when source code is unavailable.

By Bercan Turkmen, Vyas Raina

llms benchmarks

arXiv AI

Jul 21

Is "Knowing It's Malicious Enough?" Evaluating LLMs for Fine-Grained Malware Behavior Auditing

arXiv:2509. 14335v2 Announce Type: replace-cross Abstract: Automated malware classifiers achieve strong detection performance, but auditing requires more than flagging a sample: analysts must explain malicious behaviors and justify them with code evidence.

By Xinran Zheng, Xingzhi Qian, Yiling He, Shuo Yang, Lorenzo Cavallaro

llms agents benchmarks

arXiv AI

Sep 15

Exploring Automated Vulnerability Identification in JavaScript Code Using Large Language Models

arXiv:2609.13816v1 Announce Type: cross Abstract: JavaScript powers approximately 98.8% of all websites, making vulnerabilities in its code a significant security risk, yet existing detection approac...

By Manit Kaushik, Ishir Bhardwaj, Pranav Gupta, Pankaj Jalote, Arun Balaji Buduru

llms fine-tuning data-engineering

arXiv AI

Aug 11

SkillsMetric: Mapping the Detection Boundary of Static Analysis for Malicious Agent Skills

arXiv:2608. 08468v1 Announce Type: cross Abstract: Agent Skills---structured packages of instructions and scripts that augment LLM-based agents---are rapidly proliferating, yet their security properties remain under-explored.

By Xinze Chen, Chi Zhang, Ping Ji, Yimin Liu

llms agents robotics safety

One email a morning, machine-written

Related stories

SCRIPTIOC-BENCH: A Benchmark for Recognizing Actionable Threat Intelligence from Script-Based Malware using LLMs

AI-Generated PowerShell Malware: An Experimental Framework and Dataset

Multi-View Decompilation for LLM-Based Malware Classification

Is "Knowing It's Malicious Enough?" Evaluating LLMs for Fine-Grained Malware Behavior Auditing

Exploring Automated Vulnerability Identification in JavaScript Code Using Large Language Models

SkillsMetric: Mapping the Detection Boundary of Static Analysis for Malicious Agent Skills