GeoFitBench-LLM: A Benchmark for Assessing Large Language Models' Capabilities in Geospatial Data Requirements Elicitation for Geo-Analytic Tasks

GeoFitBench-LLM is a benchmark designed to evaluate large language models (LLMs) on their ability to elicit geospatial data requirements for geo‑analytic tasks. It focuses on two reasoning dimensions—spatio‑temporal relational and semantic grounding—and uses a provider‑independent schema to encode ground‑truth requirements. Experiments on seven models show that while many can identify necessary resources, accurately specifying fitness‑for‑use remains challenging, with GPT‑5.4 Thinking recovering all resources in 66.67% of cases.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

• llms
• benchmarks