arXiv Machine Learning
Jun 16
MyPCBench: A Benchmark for Personally Intelligent Computer-Use Agents
arXiv:2606. 16748v1 Announce Type: new Abstract: Current benchmarks for computer-use agents evaluate models in impersonal environments.
MCP-Persona: Benchmarking LLM Agents on Real-World Personal Applications via Environment Simulation
arXiv:2606. 02470v1 Announce Type: new Abstract: The Model Context Protocol (MCP) has emerged as a transformative standard for connecting large language models (LLMs) with external data sources and tools, and has been rapidly adopted across personal applications and development platforms.
Related stories
arXiv AI
Jun 10
TinyTroupe: An LLM-powered Multiagent Persona Simulation Toolkit
arXiv:2507. 09788v3 Announce Type: replace-cross Abstract: Recent advances in Large Language Models (LLM) have led to a new class of autonomous agents, renewing and expanding interest in the area.
arXiv AI
Aug 12
UserToolBench: A User-Profile-Hidden Benchmark for Personalized Decision Making in Tool-Use LLMs
arXiv:2608. 10042v1 Announce Type: cross Abstract: Tool-use LLMs are increasingly asked to act on users' behalf, but existing benchmarks usually focus on profile recall, style imitation, generic tool use, or response-level personalization.
Hugging Face Trending Papers
Jun 29
MCP Server Architecture Patterns for LLM-Integrated Applications
The Model Context Protocol (MCP), introduced by Anthropic in November 2024, defines a standardized interface for connecting large language models (LLMs) to external tools, data sources, and services. Within months of release, hundreds of community-built MCP servers appeared on GitHub, but no software-maintenance literature has yet described how the ecosystem is being structured in production.
arXiv AI
Jun 30
MCP Server Architecture Patterns for LLM-Integrated Applications
arXiv:2606. 30317v1 Announce Type: cross Abstract: The Model Context Protocol (MCP), introduced by Anthropic in November 2024, defines a standardized interface for connecting large language models (LLMs) to external tools, data sources, and services.
arXiv Machine Learning
Jun 15
Persona-Pruner: Sculpting Lightweight Models for Role-Playing
arXiv:2606. 14695v1 Announce Type: new Abstract: Language Models (LMs) have shown remarkable potential as role-playing chatbots, delivering consistent, stylized interactions when given a specification of a character or user persona.
arXiv Machine Learning
Jun 3
$\Psi$-Bench: Evaluating Persona-Sensitive Influencing in Persuasive Dialogues
arXiv:2606. 02754v1 Announce Type: new Abstract: Personalization is a crucial capability of modern language agents.
arXiv Machine Learning
Jul 14
CTFusion: A CTF-based Benchmark for LLM Agent Evaluation
arXiv:2605. 11504v2 Announce Type: replace Abstract: Recent advances in Large Language Models (LLMs) have enabled agentic systems for complex, multi-step tasks; cybersecurity is emerging as a prominent application.
arXiv AI
Jul 17
MCPEvol-Bench: Benchmarking LLM Agent Performance Across Dynamic Evolutions of MCP Servers
arXiv:2607. 14642v1 Announce Type: new Abstract: As Model Context Protocol (MCP) servers emerge as the core infrastructure for connecting LLMs with external tools, existing benchmarks leverage real-world MCP servers to evaluate LLM agents' tool-using capabilities.
arXiv AI
Jun 10
How can we assess human-agent interactions? Case studies in software agent design
arXiv:2510. 09801v3 Announce Type: replace Abstract: While benchmarks measure the accuracy of LLM-powered agents, they mostly assume full automation, failing to represent the collaborative nature of real-world use cases.
arXiv AI
Aug 7
LUNAR: Benchmarking Personalized Large Language Models on UNiversal User BehAvioR Logs
arXiv:2608. 05246v1 Announce Type: new Abstract: Existing personalized LLM benchmarks primarily rely on textual personas or isolated behavioral signals, providing limited evaluation of cross-domain behavioral personalization, where responses must be grounded in heterogeneous daily-life activities.
Hugging Face Trending Papers
Jun 9
STAGE-Claw: Automated State-based Agent Benchmarking for Realistic Scenarios
Large language models are increasingly used to power personal agents for everyday applications, but evaluating these agents remains a challenge. Existing benchmarks still rely on sandboxed artifacts, static task design, and coarse scoring, which hinder scalability and limit progress toward reliable personal-agent evaluation.