arXiv AI By Nanda Rani, Christian Rossow

Beyond Reproducibility: Towards Security-Aware Evaluation of Research Artifacts

Read the original on arXiv AI →

The paper "Beyond Reproducibility: Towards Security-Aware Evaluation of Research Artifacts" examines 1,388 research artifacts from top security conferences, uncovering 132,431 candidate security findings through static analysis. It introduces a taxonomy for context-aware security assessment and presents SAFE, an autonomous framework that accurately distinguishes security-relevant findings and classifies risk types. The study demonstrates that nearly 45% of findings are security-relevant, highlighting the need for security-aware evaluation alongside traditional reproducibility checks.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI
Oct 5

Beyond Predefined Sinks: Security-Aware Dependency Analysis for LLM Agents

The paper introduces AgentSecGraph, a static analysis framework that builds a Security-Aware Agent Dependency Graph (Security-ADG) for each security-sensitive operation in large language model (LLM) agents. It enriches operation identity with agent relevance, source and dependency evidence, trust-boundary context, guard evidence, and external-effect semantics. The authors also present AgentSecBench, a corpus of 67 real-world LLM-agent repositories, and demonstrate that their analyzer identifies thousands of operation candidates, recovers substantial dependency and guard evidence, and distinguishes guarded behaviors from vulnerabilities with high accuracy.

By Hang Cui
arXiv AI
Aug 24

Large Language Models at the Intersection of Software Engineering and Software Security:An Evidence-Centered Structured Survey and Research Agenda

Large Language Models (LLMs) are evolving from simple code completion tools to repository‑scale agents capable of retrieving context, editing files, executing tools, and engaging in security‑sensitive workflows. A structured survey up to May 31 2026 reviews LLM work across software engineering and security tasks, adaptation mechanisms, artifact granularity, and evaluation design, and introduces an assurance framework that separates functional correctness, security, operational reliability, evidence provenance, and agent authority. The review highlights that while execution feedback and repository access improve engineering task completion, they do not guarantee security, and static‑analysis labels rarely ensure deployable correctness; it also identifies common validity threats and proposes a minimum reporting protocol and a research agenda focused on jointly secure‑and‑functional benchmarks, repository‑scale threat models, calibrated human oversight, longitudinal maintainability evidence, and reproducible agent evaluation.

By Wei Lin, Tao Zhou, Zhaofei Xie, Changgui Hong
arXiv AI
Jun 17

ARVO: Atlas of Reproducible Vulnerabilities for Open-Source Software

arXiv:2606. 17283v1 Announce Type: cross Abstract: Achieving reproducibility, quantity, and diversity in vulnerability datasets has long been viewed as an inherent three-way trade-off, where improving one dimension often comes at the cost of the others.

By Xiang Mei, Jordi Del Castillo, Pulkit Singh Singaria, Haoran Xi, Abdelouahab Benchikh, Tiffany Bao, Ruoyu Wang, Yan Shoshitaishvili, Adam Doup\'e, Hammond Pearce, Brendan Dolan-Gavitt
arXiv Machine Learning
Sep 3

CodePoisonRAG: Knowledge Poisoning Attacks on Retrieval-Augmented Code Generation

The paper introduces CodePoisonRAG, a framework that poisons retrieval-augmented code generation systems by transforming benign code artifacts into malicious ones. It injects CWE-specific vulnerabilities and false safety claims into a single task-matched artifact, achieving high success rates across multiple generators and even against a defense system. The study demonstrates that attackers can target and propagate specific weaknesses without altering the underlying language model.

By Varun Gadey, Ziad Marey, Alexandra Dmitrienko