ATLAS-Bench: An Agentic Tool-Use Benchmark for Local Area Search
Read the original on arXiv AI →ATLAS-Bench is a new benchmark comprising 1,200 local search queries across three regions, designed to evaluate large language model agents on their ability to satisfy mandatory requirements, preferences, and factual versus subjective judgments using place attributes, reviews, and user profiles. Each query distinguishes constraints from preferences and includes evidence and criteria for verification, with validation through iterative assertion checks and cross-model review. The benchmark is paired with ATLASBox, a standardized evaluation environment that provides search tools and a fixed POI knowledge base, allowing agents to recommend evidence-supported options or report unmet requirements, and the study reports that GPT-6-Luna achieves the highest constraint satisfaction and factual accuracy, though GLM-5.3 returns acceptable recommendations for more queries. whyItMatters":"The benchmark addresses gaps in existing local search evaluations by rigorously testing agents’ ability to meet mandatory constraints and preferences, thereby advancing the development of reliable LLM agents for everyday decision-making and planning."
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.