arXiv AI By Yuzhen Wang, Chunxiao Li, Baichen Le, Lewen Yan, Jie Zhang, Yucheng T. Yang, Xing-Ming Zhao, Lijun Li, Jing Shao

DRYLAB-Bench: Benchmarking the Biosecurity Risk of Agentic LLM Mutation Design

Read the original on arXiv AI →

DRYLAB‑Bench is a new benchmark that measures the biosecurity risk of large language models (LLMs) when they design biological mutations. It presents 30 hazardous mutation‑design scenarios and evaluates LLMs in a multi‑round, tool‑augmented setting, scoring designs against 187,695 empirical wet‑lab measurements using a Biosecurity Risk Score. Initial tests on 15 frontier models show that even without tools, a majority of designs exceed average risk, and tool use further amplifies the threat.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Hugging Face Trending Papers
Jul 20

An Early Warning of Emerging Biosecurity Risks in Frontier LLMs

Frontier large language models (LLMs) are increasingly integrated into scientific workflows, yet their growing biological capabilities may outpace current safeguards. To assess the biological risks of frontier models, we develop Intern-BioBreaker, a specialized bio-red-teaming model, together with an integrated computational-to-physical framework that couples model-level stress testing with wet-lab validation.

arXiv AI
Aug 5

A Blind Spot in Alignment: Quantifying Biosecurity Risks in Large Language Models

arXiv:2608. 02684v1 Announce Type: cross Abstract: Large Language Models (LLMs) are accelerating biological research, yet this same capability poses a critical biosecurity threat: models that assist in protein engineering can equally be prompted to generate predicted toxin-like sequences, potentially lowering the barrier to biological misuse.

By Shu Quan, Tianfang Hao, Sitong Fang, He Geng, Jiayi Zhou, Boyuan Chen, Kaile Wang, Donghai Hong, Juntao Dai, Yaodong Yang, Jiaming Ji
arXiv AI
Jul 8

Evaluating calibrated refusal and safe usefulness in dual-use biology settings

arXiv:2607. 05462v1 Announce Type: cross Abstract: As AI agents are incorporated into life science workflows, the capabilities that speed discovery might also enable misuse.

By Edwin H. Wintermute, Harmon Bhasin, Christina M. Agapakis, Dianzhuo Wang, Evan Seeyave, Arjun Banerjee, Daniel Fulop, Matthew C. Watson, Adam J. Meyer, Sandrine Boissel, Jens H. Kuhn, Rishi Jain, Noah D. Taylor, Helena Shomar, Patrick M. Boyle, Kenny Workman
arXiv AI
Jul 22

BioSecBench-Refusal: A paired metric for performance and alignment in agentic biosecurity risk assessment

arXiv:2607. 05462v2 Announce Type: replace-cross Abstract: As AI agents are incorporated into life science workflows, the capabilities that speed discovery might also enable misuse.

By Edwin H. Wintermute, Harmon Bhasin, Christina M. Agapakis, Dianzhuo Wang, Evan Seeyave, Arjun Banerjee, Daniel Fulop, Matthew C. Watson, Adam J. Meyer, Sandrine Boissel, Jens H. Kuhn, Rishi Jain, Noah D. Taylor, Helena Shomar, Patrick M. Boyle, Kenny Workman
arXiv AI
3d ago

DeepSafe-SciHazard: A Benchmark for Measuring Scientific Safety Risks with Decomposed Harm Scoring

DeepSafe‑SciHazard introduces a real‑world‑grounded benchmark for measuring scientific safety risks, comprising 2,400 hazardous and 600 oversafety questions across 12 disciplines. It defines a decomposed harm scoring system, DeHarm‑Score, that evaluates query severity, refusal behavior, and response‑level risk, further breaking down harm into executability and net‑new risk. An expert‑validation study shows DeHarm‑Score aligns 90.17% better with expert judgments than the best baseline, and the benchmark reveals that deep research agents score 32.3% higher on mean DeHarm‑Score than standard LLMs, highlighting a critical blind spot in current safety defenses.

By Chunxiao Li, Yuan Xiong, Lijun Li, Tianyi Du, Wenlong Zhang, Lei Bai, Jing Shao