← Back to all news
arXiv AI By Oded Ovadia, Elad Ben Zaken, Elad Guttman, Orly Moreno Kadosh

MiST: Mid-Training LLMs for Cybersecurity

Read the original on arXiv AI →

MiST (Mid-trained Security Transformer) is a suite of 8B and 32B language models tailored for cybersecurity, achieving strong performance on public benchmarks. The approach uses a mid-training stage that adapts general pre-trained models to the domain by curating a compact, expert-vetted seed corpus and generating high-quality synthetic training data, rather than continual pre-training on large raw text. MiST checkpoints improve mean cybersecurity accuracy by +13.1 and +8.6 absolute percentage points over Qwen baselines for 8B and 32B models, respectively, and provide a stronger initialization for downstream task-specific fine-tuning and reinforcement learning.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv AI
Jul 3

Less Data, More Security: Advancing Cybersecurity LLMs Specialization via Resource-Efficient Domain-Adaptive Continuous Pre-training with Minimal Tokens

arXiv:2507. 02964v2 Announce Type: replace-cross Abstract: The increasing scale of AI workloads demands High-Performance Computing (HPC) infrastructure and training methodologies that are both scalable and sustainable.

By Salahuddin Salahuddin, Ahmed Hussain, Jussi L\"opp\"onen, Toni Jutila
llmsfine-tuningbenchmarks
More like this →
arXiv AI
Jul 2

Toward Cybersecurity-Expert Small Language Models

arXiv:2510. 14113v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are transforming everyday applications, yet deployment in cybersecurity lags due to a lack of high-quality, domain-specific models and training datasets.

By Matan Levi, Daniel Ohayon, Ariel Blobstein, Ravid Sagi, Ian Molloy, Yair Allouche
llmsbenchmarks
More like this →
arXiv Computation and Language
Oct 16

LLM-Based Weak Annotation for Cybersecurity Tweet Classification: A Teacher-Student Framework

The paper introduces a Teacher-Student framework that uses a large language model (LLM) to generate weak labels for unlabeled cybersecurity tweets, enabling the training of domain‑specific pre‑trained language models (PLMs) without extensive manual annotation. Experiments demonstrate that PLMs trained solely on these LLM‑generated labels achieve F1 scores above 0.92, within 3 percentage points of fully human‑supervised models across all evaluated architectures. This approach offers a practical alternative for creating cybersecurity‑specific training data when expert annotation resources are scarce.

By Ferry Afit Kurniawan, Kazushi Okamoto, Koki Karube, Kei Harada, Atsushi Shibata
llms
More like this →