arXiv Computer Vision September 14, 2026 By Isaac Corley, Nils Lehmann, Caleb Robinson, Gabriel Tseng, Anthony Fuller, Hamed Alemohammad, Evan Shelhamer, Jennifer Marcus, Hannah Kerner

No One Knows the State of the Art in Geospatial Foundation Models

Read the original on arXiv Computer Vision →

The paper "No One Knows the State of the Art in Geospatial Foundation Models" critiques the current lack of standardization in geospatial foundation model (GFM) research, highlighting inconsistencies in evaluation, training, and model release practices across 152 papers. It reports significant discrepancies—46 cross-paper disagreements of at least 10 points for the same model and benchmark, 94 out of 126 papers using unique pretraining configurations, and 39% of papers releasing no model weights. The authors propose six concrete expectations, including named-license weight release, shared core evaluations, and a unified evaluation harness, to address these coordination failures and foster a clearer, comparable understanding of GFM progress.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

benchmarks

arXiv AI

Jul 22

Now We Know? A Systematic Comparison of TerraMind and THOR

arXiv:2607. 18504v1 Announce Type: cross Abstract: Benchmarks for Geospatial Foundation Models (GFMs) increasingly rank models by aggregate score, but such rankings obscure why models differ: how much of the gap is architecture, how much is decoder capacity, and how much is a use-case-specific artefact?

By Frederick Schindlegger, Kenzo Bounegta, Eva Gmelich Meijling, Johannes Jakubik, Arnt-B{\o}rre Salberg, Theodor Forgaard, Nicolas Longepe, Valerio Marsocci

computer-vision fine-tuning multimodal benchmarks

arXiv AI

Jun 12

GeoNatureAgent Benchmark: Benchmarking LLM Agents for Environmental Geospatial Analysis Across Frontier and Open-Weight Foundation Models

arXiv:2606. 12821v1 Announce Type: new Abstract: Environmental scientists spend disproportionate effort on data wrangling rather than analysis, and AI agents that automate geospatial workflows remain unvalidated: no benchmark evaluates agents operating through structured tool calling against real APIs.

By Gabriel Diaz-Ireland, Diego Prieto-Herr\'aez, Mario Garc\'ia Peces, Javier Vel\'azquez, Devika Jain

llms agents benchmarks

arXiv Computer Vision

Sep 10

Infra-Bench CLS: A Global, Open-Source Benchmark for Critical Infrastructure Classification with Earth Observation Foundation Models

arXiv:2609.09482v1 Announce Type: new Abstract: Critical infrastructure location data is often incomplete and unevenly distributed globally, especially in developing regions. Earth observation founda...

By Justin Guthrie, Edward Oughton, Konrad Wessels, Matthew Rice, Isaac Corley

fine-tuning benchmarks

arXiv Machine Learning

Jun 4

Geospatial Foundation Models to Enable Progress on Sustainable Development Goals

arXiv:2505. 24528v3 Announce Type: replace-cross Abstract: Foundation Models (FMs) are large-scale, pre-trained artificial intelligence (AI) systems that have revolutionized natural language processing and computer vision, and are now advancing geospatial analysis and Earth Observation (EO).

By Pedram Ghamisi, Weikang Yu, Xiaokang Zhang, Aldino Rizaldy, Jian Wang, Chufeng Zhou, Richard Gloaguen, Gustau Camps-Valls

computer-vision nlp benchmarks safety

arXiv Machine Learning

Aug 27

Planetary Prediction Engine: Autonomous Geospatial Prediction via Intelligent Data Selection and Foundation Model Embeddings

The Planetary Prediction Engine (PPE) is an autonomous AI system that transforms natural-language queries into end-to-end geospatial predictions. It automatically retrieves and fuses multimodal datasets from open-web and Earth observation sources, incorporates foundation model embeddings, and searches task‑specific model families with overfitting safeguards. Across multiple domains, PPE outperforms state‑of‑the‑art baselines, improving regression metrics for CDC health indicators, FEMA risk indices, and the Social Vulnerability Index, doubling accuracy for Nigerian food security indicators, and achieving higher recall in Ebola outbreak nowcasting.

By Evelyn Ma, Rama Kumar Pasumarthi, Kishwar Shafin, Mandar Sharma, Mimi Sun, Hamed Sadeghi, Dav M. Ebengo, Mbulayi Onesime, Rouslan Solomakhin, John Wamburu, William Ogallo, Aisha Walcott-Bryant, Sanxing Chen, Arbaaz Muslim, Yael Mayer, Ronald Ho, Roy Lee, Ruth Alcantara, Abdoulaye Diack, Monica Bharel, Lambert Rosique, Jeremy Amez-Droz, Christopher Haire, James Manyika, Yossi Matias, Niv Efron, Gautam Prasad, Shravya Shetty

rag agents multimodal benchmarks

arXiv Machine Learning

Jul 10

Scalable and Trustworthy Earth Observation Foundation Models

arXiv:2607. 07758v1 Announce Type: new Abstract: Foundation models (FMs) have transformed machine learning from isolated task-specific model development toward general-purpose models pretrained on broad data and adapted to multiple downstream tasks.

By Syed Usama Imtiaz, Mitra Nasr Azadani, Nasrin Alamdari

reinforcement-learning multimodal benchmarks

One email a morning, machine-written

Related stories

Now We Know? A Systematic Comparison of TerraMind and THOR

GeoNatureAgent Benchmark: Benchmarking LLM Agents for Environmental Geospatial Analysis Across Frontier and Open-Weight Foundation Models

Infra-Bench CLS: A Global, Open-Source Benchmark for Critical Infrastructure Classification with Earth Observation Foundation Models

Geospatial Foundation Models to Enable Progress on Sustainable Development Goals

Planetary Prediction Engine: Autonomous Geospatial Prediction via Intelligent Data Selection and Foundation Model Embeddings

Scalable and Trustworthy Earth Observation Foundation Models