arXiv Machine Learning

Multimodal Analytics of Cybersecurity Crisis Preparation Exercises: What Predicts Success?

arXiv:2603. 28553v2 Announce Type: replace-cross Abstract: Instructional alignment, the match between intended cognition and enacted activity, is central to effective instruction but hard to operationalize at scale.

arXiv AI

Jul 22

Assessment in Team Problem-Solving Exercises in Computing Education

arXiv:2607. 19209v1 Announce Type: cross Abstract: This full paper in the research-to-practice track presents methods for assessing student teams in tabletop exercises (TTXs).

By Valdemar \v{S}v\'abensk\'y, Jan Vykopal, Sukrit Leelaluk, Pavel \v{C}eleda, Fumiya Okubo, Atsushi Shimada

llms

arXiv AI

Jul 22

Find Before You Fine-Tune: A Diagnostic Study of Small LLMs for Cybersecurity QA

arXiv:2607. 18725v1 Announce Type: cross Abstract: Large Language Models (LLMs) are increasingly fine-tuned for critical-domain Question-Answering (QA), yet choosing which small model to adapt, before paying the cost of adaptation, remains difficult.

By Shaswata Mitra, Subash Neupane, Trisha Chakraborty, Himanshu Tripathi, Sudip Mittal, Aritran Piplai, Shahram Rahimi

llms fine-tuning safety

arXiv AI

Jul 2

Toward Cybersecurity-Expert Small Language Models

arXiv:2510. 14113v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are transforming everyday applications, yet deployment in cybersecurity lags due to a lack of high-quality, domain-specific models and training datasets.

By Matan Levi, Daniel Ohayon, Ariel Blobstein, Ravid Sagi, Ian Molloy, Yair Allouche

llms benchmarks

arXiv Machine Learning

22h ago

Which CS1 Students Will Fail? Identifying Digital Markers from Learning Analytics in Computer Systems and Architecture Using Weighted Academic Momentum and Interaction Logs

arXiv:2608. 16914v1 Announce Type: cross Abstract: Digital learning platforms generate rich behavioural traces (digital markers) that offer the potential to identify struggling students early.

By Lighton Phiri, Mutune Chaibela, Ivy Chisha, David Pungwa, Danny Siabbaba, Bydon Simukoko

benchmarks

arXiv AI

Jun 11

Human-Guided Agentic AI for Multimodal Clinical Prediction: Lessons from the AgentDS Healthcare Benchmark

arXiv:2602. 19502v2 Announce Type: replace Abstract: Agentic AI systems are increasingly capable of autonomous data science workflows, yet clinical prediction tasks demand domain expertise that purely automated approaches struggle to provide.

By Lalitha Pranathi Pulavarthy, Raajitha Muthyala, Aravind V Kuruvikkattil, Zhenan Yin, Rashmita Kudamala, Saptarshi Purkayastha

agents multimodal benchmarks safety

arXiv Machine Learning

Jun 9

Helpful to a Fault: Measuring Illicit Assistance in Multi-Turn, Multilingual LLM Agents

arXiv:2602. 16346v4 Announce Type: replace-cross Abstract: LLM-based agents execute real-world workflows via tools and memory.

By Nivya Talokar, Ayush K Tarun, Murari Mandal, Maksym Andriushchenko, Antoine Bosselut

llms agents benchmarks safety

arXiv AI

Jul 9

Predicting LLM Safety Before Release by Simulating Deployment

arXiv:2607. 07184v1 Announce Type: cross Abstract: Pre-deployment safety evaluations aim to inform the downstream risks of releasing a new AI model.

By Marcus Williams, Hannah Sheahan, Cameron Raymond, Tomek Korbak, Deng Pan, Peilin Yang, Leon Maksin, Ningyi Xie, Phillip Guo, Ian Kivlichan, Micah Carroll

llms safety

arXiv Machine Learning

Jun 17

Evaluating Open-Source LLMs for Multi-Label ATT&CK Technique Classification on CTI Reports

arXiv:2606. 18166v1 Announce Type: cross Abstract: Classifying Cyber Threat Intelligence (CTI) using MITRE Adversarial Tactics, Techniques, and Common Knowledge (ATT&CK) is essential for proactive defense, but historically required extensive human effort.

By Ahmed Ryan, Saad Sakib Noor, Md Erfan, Shaswata Mitra, Sudip Mittal, Md Rayhanur Rahman

llms benchmarks safety

arXiv AI

Aug 11

MCIF: Multimodal Crosslingual Instruction-Following Benchmark from Scientific Talks

arXiv:2507. 19634v4 Announce Type: replace-cross Abstract: Recent advances in large language models have laid the foundation for multimodal LLMs (MLLMs), which unify text, speech, and vision within a single framework.

By Sara Papi, Maike Z\"ufle, Marco Gaido, Beatrice Savoldi, Danni Liu, Ioannis Douros, Luisa Bentivogli, Jan Niehues

llms nlp multimodal benchmarks

arXiv AI

Jul 23

Stateful Guardrails for Multi-Turn LLM Systems: A Conversational Risk Accumulation Framework

arXiv:2607. 19361v1 Announce Type: cross Abstract: Most safety guardrails for large language models (LLMs) evaluate each prompt-response pair in isolation, which misses failures that arise only over a dialogue as benign turns compose into harm.

By Sanjay Mishra, Divya Chukkapalli, Ganesh R. Naik

llms benchmarks safety

arXiv Machine Learning

1d ago

LLMs for Zero-Shot Threat Detection via Structured Risk Indicators

arXiv:2608. 16508v1 Announce Type: cross Abstract: We propose a two-stage large language model (LLM) framework for zero-shot detection of insider threats and advanced persistent threats (APTs) from heterogeneous security logs.

By Abdullah Alghamdi, Siamak Layeghy, Marius Portmann

llms rag benchmarks

Hugging Face Trending Papers

Jul 8

Predicting LLM Safety Before Release by Simulating Deployment

Pre-deployment safety evaluations aim to inform the downstream risks of releasing a new AI model. Yet most evaluations provide limited evidence about how often undesired model behavior will occur in deployment: they generally have insufficient coverage, are unrepresentative, and are generally recognizable as tests.

llms safety

Related stories