RoleFUZZER is a framework that applies coverage‑guided fuzzing to test large language models (LLMs) for adherence to role specifications such as tutor, financial advisor, AI interviewer, antagonist, and web agent. It constructs a 64‑cell user‑behaviour coverage grid based on a taxonomy of interaction types and the model’s role specification, then systematically explores plausible user behaviours to trigger violations. Across 15 simulated campaigns, RoleFUZZER achieved over 89% coverage of the grid for every tested model and role, revealing specification‑grounded failure patterns that would be missed by free‑form user prompts.
arXiv:2607. 02345v1 Announce Type: cross Abstract: Large Language Model (LLM)-based agents increasingly automate software engineering tasks through reusable skills, natural-language instruction documents that guide planning and execution.
arXiv:2605.12015v3 Announce Type: replace-cross Abstract: Reusable skills are becoming a common interface for extending large language model agents, packaging procedural guidance with access to files...
The article discusses how generative models increasingly act as builders, defenders, and breakers of software, challenging the assumption that full autonomy is the ultimate goal. It introduces a framework that defines measurable independence between lifecycle roles based on shared generative substrates, and proposes five autonomy levels, three human roles, and five decision criteria to guide oversight. The authors argue that human authority should focus on specification, accountability, and emergency intervention, and they outline testable hypotheses and protocols to evaluate independence and oversight effectiveness.
The paper introduces a six‑module scanner for detecting unintended local and composite vulnerabilities in LLM‑based agent skills. Local Analysis achieved high recall and precision on benchmark datasets, outperforming existing scanners, while Composite Analysis accurately modeled inter‑skill dependencies and identified composite flaws. The authors validated findings with proof‑of‑concept attacks, disclosed cases to maintainers, and demonstrated effective mitigations that preserved legitimate functionality.
arXiv:2607. 24343v1 Announce Type: cross Abstract: Language-model agents act through structured tool calls whose arguments carry different risks.
arXiv:2607. 16345v1 Announce Type: cross Abstract: Modern agentic systems increasingly rely on skills: installable packages of natural language and code that teach an LLM agent to perform a domain task.
Language-model agents act through structured tool calls whose arguments carry different risks. Untrusted content may safely influence an email body but should not determine a recipient, account, command, or credential.
arXiv:2608. 04719v1 Announce Type: new Abstract: Agent evaluations tell us that a model picked the wrong tool, but rarely why.
Agent evaluations tell us that a model picked the wrong tool, but rarely why. We introduce canary tools: diagnostic probe tools planted in an agent's Model Context Protocol (MCP) tool set, each engineered to probe one specific tool-selection weakness.
arXiv:2607. 22569v1 Announce Type: new Abstract: Coding agents are increasingly integrated into system operations, where their tool use can directly modify project artifacts, execution environments, and the underlying system.
arXiv:2605. 00994v2 Announce Type: replace-cross Abstract: Finetuning can significantly modify the behavior of large language models, including introducing harmful or unsafe behaviors.
arXiv:2606. 07808v1 Announce Type: new Abstract: Reasoning language models deployed in agentic workflows must follow an instruction hierarchy: when instructions from different sources conflict, the model should obey the highest-privilege applicable instruction.