arXiv AI
Jul 14
How Do Practitioners Build SE Agents? Insights from a Mixed-Methods Study
arXiv:2607. 10856v1 Announce Type: cross Abstract: The rise of Software Engineering (SE) agents, i.
Measuring Agents in Production
arXiv:2512. 04123v4 Announce Type: replace-cross Abstract: LLM-based agents already operate in production across many industries, yet we lack an understanding of what technical methods make deployments successful.
Related stories
arXiv AI
Jun 10
How can we assess human-agent interactions? Case studies in software agent design
arXiv:2510. 09801v3 Announce Type: replace Abstract: While benchmarks measure the accuracy of LLM-powered agents, they mostly assume full automation, failing to represent the collaborative nature of real-world use cases.
arXiv AI
Jul 9
Reliable and Developer-Aligned Evaluation of Agents for Software Engineering
arXiv:2607. 06713v1 Announce Type: cross Abstract: Large language models are rapidly moving towards closing the development cycle, transitioning from simple assistive companions to autonomous contributors deeply embedded into collaborative development environments.
arXiv AI
Jul 9
AgentLens: Production-Assessed Trajectory Reviews for Coding Agent Evaluation
arXiv:2607. 06624v1 Announce Type: new Abstract: We present AgentLens, a production-assessed benchmark for interactive code agents.
Hugging Face Trending Papers
Jul 9
Tool-Making and Self-Evolving LLM Agents in Low-Latency Systems
Production LLM agents often waste latency and reliability by regenerating code for the same procedural steps on every request. We replace this inference-time coding loop with an agentic tool-making pipeline that compiles repeated SOP steps into validated, versioned tools before deployment.
arXiv Machine Learning
Jul 10
Tool-Making and Self-Evolving LLM Agents in Low-Latency Systems
arXiv:2607. 08010v1 Announce Type: cross Abstract: Production LLM agents often waste latency and reliability by regenerating code for the same procedural steps on every request.
Hugging Face Trending Papers
Jul 15
AgentCompass: A Unified Evaluation Infrastructure for Agent Capabilities
As Large Language Models (LLMs) evolve into autonomous agents, the need for unified evaluation infrastructure becomes critical. However, current evaluation pipelines remain highly fragmented and tightly coupled, hindering reproducibility and causing redundant engineering.
arXiv AI
Jul 3
A Unified Framework for the Evaluation of LLM Agentic Capabilities
arXiv:2605. 27898v2 Announce Type: replace Abstract: As LLMs are increasingly deployed as agents, reliable assessment of their agentic capabilities has become essential.
arXiv AI
Jun 26
AgentX: Towards Agent-Driven Self-Iteration of Industrial Recommender Systems
arXiv:2606. 26859v1 Announce Type: new Abstract: Recommendation algorithm iteration is moving from an artisanal, engineer-bound process toward an industrialized research loop, but this transition remains blocked by a structural execution bottleneck: the idea-to-launch cycle still depends on human engineers to generate hypotheses, modify production code, launch A/B experiments, and attribute online results.
arXiv AI
Jul 16
AgentCompass: A Unified Evaluation Infrastructure for Agent Capabilities
arXiv:2607. 13705v1 Announce Type: new Abstract: As Large Language Models (LLMs) evolve into autonomous agents, the need for unified evaluation infrastructure becomes critical.
OpenAI Blog
Oct 6, 2025
Introducing AgentKit, new Evals, and RFT for agents
Today, we’re releasing new tools to help developers go from prototype to production faster: AgentKit, expanded evals capabilities, and reinforcement fine-tuning for agents.
arXiv AI
Jul 7
AgentGym2: Benchmarking Large Language Model Agents in De-Idealized Real-World Environments
arXiv:2607. 05174v1 Announce Type: new Abstract: Language agents, i.