arXiv AI
Jul 9
Predicting LLM Safety Before Release by Simulating Deployment
arXiv:2607. 07184v1 Announce Type: cross Abstract: Pre-deployment safety evaluations aim to inform the downstream risks of releasing a new AI model.
Predicting LLM Safety Before Release by Simulating Deployment
Pre-deployment safety evaluations aim to inform the downstream risks of releasing a new AI model. Yet most evaluations provide limited evidence about how often undesired model behavior will occur in deployment: they generally have insufficient coverage, are unrepresentative, and are generally recognizable as tests.
Related stories
arXiv AI
Aug 7
What Current AI Benchmarks Leave Unmeasured: Modality, Search, Citations, and Implications (for Safety Evaluations)
arXiv:2608. 06202v1 Announce Type: cross Abstract: Large language model (LLM) benchmark evaluations are routinely used to support claims about model safety, reliability, and deployment readiness.
arXiv AI
Jul 17
ToolAlignBench: Investigating Alignment Conflicts in Tool-Calling Enabled LLMs
arXiv:2607. 14285v1 Announce Type: cross Abstract: Safety alignment in LLMs aims to align models with human values, but which values take precedence when they conflict?
arXiv AI
Jun 11
Models That Know How Evaluations Are Designed Score Safer
arXiv:2605. 28591v2 Announce Type: replace-cross Abstract: The validity of AI safety evaluations depends on models behaving consistently across controlled and deployment settings.
arXiv AI
Jul 17
Democratizing Agent Deployment Safety: A Structural Monitoring Approach
arXiv:2607. 14570v1 Announce Type: new Abstract: AI software development agents are increasingly capable of modifying infrastructure and security critical systems, creating risks where an agent completes its assigned task while covertly weakening safeguards through actions such as broadening permissions, degrading logging, or introducing persistence mechanisms.
arXiv Machine Learning
Jun 25
ClayBuddy: A Framework, Evaluation, & Mitigation of Coding Agent Failures
arXiv:2606. 19380v3 Announce Type: replace-cross Abstract: Software engineering and deployment are increasingly delegated to AI coding agents.
arXiv AI
Aug 3
Tool Specifications Matter: Uncovering and Mitigating Safety Risks in AI Agents
arXiv:2607. 29254v1 Announce Type: new Abstract: AI agents extend large language models (LLMs) with external tools, enabling them to perform complex tasks and translate model outputs into consequential real-world actions.
arXiv AI
Jun 17
An Evaluation of Data Leakage Risks in Tool-Using LLM Agents in Realistic Scenarios
arXiv:2606. 17114v1 Announce Type: cross Abstract: AI agents are increasingly being adopted in enterprise and personal settings with access to emails, databases, documents, and other tools where they can read, update, and disseminate sensitive information.
arXiv AI
Jun 2
Benchmarking AI for low-resource contexts: Thinking beyond leaderboards
arXiv:2605. 28508v2 Announce Type: replace Abstract: Existing AI evaluation practices often fail to capture how systems actually perform in low-resource environments, where operational constraints shape usability as much as model quality.
OpenAI Blog
Jun 16
Predicting model behavior before release by simulating deployment
OpenAI introduces Deployment Simulation, a method to predict AI model behavior before deployment using real conversation data to improve safety and evaluation accuracy.
arXiv AI
1d ago
Towards Risk-free AI Agent Deployment
arXiv:2608. 16411v1 Announce Type: cross Abstract: LLM-based agents are rapidly moving from research prototypes into the core business processes of organizations, but these agents pose deployment risks to security, compliance, and functionality.
arXiv Machine Learning
Jun 19
AgentArmor: A Framework, Evaluation, \& Mitigation of Coding Agent Failures
arXiv:2606. 19380v1 Announce Type: cross Abstract: Software engineering and deployment are increasingly being delegated to AI coding agents.