arXiv Machine Learning
Analysis of Local Open-Weight LLM Agents on Vulnerability-Reproduction Tasks
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv AI
Aug 13
Enhancing Linux Privilege Escalation Attack Capabilities of Local LLM Agents
arXiv:2604. 27143v2 Announce Type: replace-cross Abstract: Cloud-based Large Language Models (LLMs) can perform autonomous penetration-testing sub-tasks such as Linux privilege escalation, but raise security, privacy, and sovereignty concerns.
arXiv AI
Jul 24
AI Security Policy Should Assess Systems, Not Only Models
arXiv:2605. 09504v2 Announce Type: replace-cross Abstract: We present swarm-attack, an open-source adversarial testing framework in which multiple lightweight LLM agents coordinate through shared memory, parallel exploration, and evolutionary optimization.
arXiv AI
Oct 28
InfraDev: Can Coding Agents Develop AI Infrastructure, and How Can We Tell?
InfraDev is a new benchmark comprising 36 tasks derived from real-world pull requests to Megatron‑LM, vLLM, and verl, designed to test coding agents on infrastructure development for training, inference, and reinforcement learning across many GPUs. Each task requires agents to reimplement multi‑file changes and run correctly, with grading that includes hidden probes and a prompt exposing only the interfaces used. The benchmark reveals that current frontier models resolve only about 39% of tasks, with probes rejecting a third of submissions that pass official tests, and that structural and state errors dominate failures.
arXiv AI
Sep 10
SWE-Test: Benchmarking LLM Vulnerability Discovery via Input Prediction
arXiv:2609.06229v1 Announce Type: cross Abstract: Vulnerability discovery is becoming an important ability of large language model (LLM) agents: agents that silently miss real defects leave critical...
arXiv AI
Sep 1
SIR: Self-improving Red-teaming for Compute Use Agents
arXiv:2608.30207v1 Announce Type: cross Abstract: Computer use agents (CUAs) are vision-language models that perceive a screen and act on a real operating system through mouse, keyboard, and terminal...
arXiv AI
Oct 9
Evaluating Local Language Model Agents for Reproducible Data Engineering: An Empirical Software Engineering Study of Mobility Workflows
The study evaluates local open‑weight large language model agents for data‑engineering tasks, using a benchmark of fifteen mobility‑workflow tasks that cover discovery, processing, enrichment, and reporting. Ten local model configurations were tested in one‑shot and closed‑loop conditions, producing 1,500 scored attempts on a consumer‑grade GPU. Results show that models larger than two billion parameters achieve up to 85.3% artifact‑level success, with closed‑loop workspaces boosting pass rates by 26.7–52.0 percentage points, and a quantized 9‑billion‑parameter model reaching 69.3% success with a 6.5 GB memory footprint.