← Back to all news
arXiv AI By Zhengyi Jin, Ru Zhang, Xiao Chen, Xinbo Liu, Jiaxuan Lin, Jia Huang, Jianyi Liu, Zhen Yang

FUSE: An Evaluating Framework for Dangerous Capabilities of LLMs

Read the original on arXiv AI →

The paper introduces FUSE, a modular framework that evaluates large language models (LLMs) for dangerous capabilities across three orthogonal pipelines: Knowledge (K), Defense (D), and Harm (H). Using a chemical‑biological module, the authors assess 12 commercial LLMs, revealing divergent profiles among models and families, and showing that newer models increase knowledge while only partially improving defense. The framework’s reliability is supported by high cross‑judge consistency and low inter‑pipeline correlations.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv Machine Learning
Oct 8

Restricting the Model, Missing the System: Measurement and Accountability in Offensive AI Governance

arXiv:2605.09504v3 Announce Type: replace-cross Abstract: We show that the instruments used to measure AI offensive capability fail in two ways: (i) they overstate harm, and (ii) they credit the mode...

By Michael Alexander Riegler, Finn Schwall, Annika Willoch Olstad, Birk Sebastian Frostelid Torpmann-Hagen, Sushant Gautam, Klas H. Pettersen, Inga Str\"umke
llmsagentssafety
More like this →
arXiv AI
Oct 9

ReSI: Recursive Safety Improvement toward Resistant and Resilient AI

ReSI is a recursive safety improvement framework that iteratively applies diverse red‑teaming methods to identify vulnerabilities in AI models, develops training recipes, and selects the update with the greatest safety gain while retaining capability. Across four dense and mixture‑of‑experts models, ReSI matches or surpasses frontier models on both in‑distribution and out‑of‑distribution safety benchmarks, and outperforms alignment baselines on nearly all safety evaluations. The framework notably reduces the mean X‑Teaming attack success rate from 86.01% to 31.45%, outperforming GPT‑5.6‑Luna’s 56.69% and demonstrating stronger resilience to unseen attacks.

By Jingnan Zheng, Dongcheng Zhang, Yi Zhang, Ming Zhang, Qiaosheng Zhang, Youbang Sun, An Zhang, Xiangnan He, Tat-Seng Chua, Xia Hu, Bowen Zhou, Chaochao Lu, Xiang Wang
llmsbenchmarkssafety
More like this →