OpenAI Blog
Mar 11, 2025
Introducing AgentKit, new Evals, and RFT for agents
Today, we’re releasing new tools to help developers go from prototype to production faster: AgentKit, expanded evals capabilities, and reinforcement fine-tuning for agents.
Related stories
arXiv AI
Jun 8
Measuring Agents in Production
arXiv:2512. 04123v4 Announce Type: replace-cross Abstract: LLM-based agents already operate in production across many industries, yet we lack an understanding of what technical methods make deployments successful.
Hugging Face Trending Papers
Jul 15
AgentCompass: A Unified Evaluation Infrastructure for Agent Capabilities
As Large Language Models (LLMs) evolve into autonomous agents, the need for unified evaluation infrastructure becomes critical. However, current evaluation pipelines remain highly fragmented and tightly coupled, hindering reproducibility and causing redundant engineering.
OpenAI Blog
Apr 15
The next evolution of the Agents SDK
OpenAI updates the Agents SDK with native sandbox execution and a model-native harness, helping developers build secure, long-running agents across files and tools.
arXiv AI
Jul 16
AgentCompass: A Unified Evaluation Infrastructure for Agent Capabilities
arXiv:2607. 13705v1 Announce Type: new Abstract: As Large Language Models (LLMs) evolve into autonomous agents, the need for unified evaluation infrastructure becomes critical.
arXiv AI
Jul 9
Reliable and Developer-Aligned Evaluation of Agents for Software Engineering
arXiv:2607. 06713v1 Announce Type: cross Abstract: Large language models are rapidly moving towards closing the development cycle, transitioning from simple assistive companions to autonomous contributors deeply embedded into collaborative development environments.
arXiv AI
Jun 10
How can we assess human-agent interactions? Case studies in software agent design
arXiv:2510. 09801v3 Announce Type: replace Abstract: While benchmarks measure the accuracy of LLM-powered agents, they mostly assume full automation, failing to represent the collaborative nature of real-world use cases.
OpenAI Blog
Feb 11
Harness engineering: leveraging Codex in an agent-first world
By Ryan Lopopolo, Member of the Technical Staff
arXiv AI
Jul 7
AgentGym2: Benchmarking Large Language Model Agents in De-Idealized Real-World Environments
arXiv:2607. 05174v1 Announce Type: new Abstract: Language agents, i.
arXiv AI
Jun 29
Agentic Hardware Design as Repository-Level Code Evolution
arXiv:2606. 28279v1 Announce Type: cross Abstract: We present HORIZON, a self-evolving agent framework that treats hardware design as repository-level code evolution.
arXiv AI
Jul 9
AgentLens: Production-Assessed Trajectory Reviews for Coding Agent Evaluation
arXiv:2607. 06624v1 Announce Type: new Abstract: We present AgentLens, a production-assessed benchmark for interactive code agents.
arXiv AI
Jun 19
ScaleWoB: Guiding GUI Agents with Coding Agents via Large-Scale Environmental Synthesis
arXiv:2605. 25160v2 Announce Type: replace Abstract: GUI agents powered by large language models are advancing rapidly, creating urgent needs for evaluation and training based on realistic environments.