Hugging Face Trending Papers
AtlasNLP: A Country-Aware Atlas of Dataset Representation in NLP
Read the original on Hugging Face Trending Papers →The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
Related stories
arXiv AI
Sep 1
AtlasNLP: A Country-Aware Atlas of Dataset Representation in NLP
arXiv:2608.30107v1 Announce Type: cross Abstract: Understanding which countries are represented in NLP datasets is essential for identifying gaps, targeting data collection, measuring progress, and i...
arXiv AI
Sep 25
From Policy Documents to Structured Survey Responses: Evaluating Large Language Models for Policy Monitoring
The paper explores using large language models (LLMs) as AI respondents to convert policy documents into structured survey responses. It introduces a long-context in‑context learning pipeline that maps policy text to predefined survey categories such as policy instruments, target groups, and thematic areas, and includes a secondary LLM validation step. Evaluation on a multi‑country dataset shows high agreement (84‑95%) with human responses for structured indicators, though free‑text fields differ, indicating that hybrid human‑AI workflows can enhance policy monitoring efficiency while still requiring human oversight.
Hugging Face Trending Papers
Oct 27
The State and Fate of Multilingual, Contextual Evaluation in the NLP World
The paper audits 51 multilingual evaluation benchmarks covering 242 datasets and 219 languages, focusing on coverage, representativeness, and rigor. It finds that while coverage is broad, many languages appear in only one benchmark and entire regions are underrepresented, with low-resource languages evaluated on far fewer tasks than high-resource ones. The study also highlights that most datasets rely on English translations, creating artifacts and limiting cultural relevance, and that benchmark contamination through translation leakage and corpus overlap undermines rigor. Recommendations call for natively constructed, culturally grounded, contamination-aware evaluation frameworks that truly serve the communities they aim to assess.
arXiv AI
Jul 8
Data Analysis in the Wild: Benchmarking Large Language Models Against Real-World Data Complexities
arXiv:2607. 06482v1 Announce Type: cross Abstract: Current benchmarks for evaluating Large Language Models (LLMs) in data analysis often fail to reflect real-world settings.
arXiv AI
Sep 12
Geospatial AI, Dataverse Metadata, and the Study of Place-Based Government
The paper presents a knowledge graph built from Harvard Dataverse’s public data, linking 102,650 datasets to 215,985 nodes and 528,003 edges that include keywords, publications, subjects, journals, and locations. About 43,991 datasets contain geospatial metadata, and 7,654 are identified as policy‑relevant, with elections and legislatures forming the largest cluster. The authors highlight the challenge of place resolution—disconnected nodes representing the same location—and propose the graph as a testbed for AI‑driven metadata enrichment and entity resolution, noting a bias toward American city‑level data.
arXiv AI
Oct 12
GeoFitBench-LLM: A Benchmark for Assessing Large Language Models' Capabilities in Geospatial Data Requirements Elicitation for Geo-Analytic Tasks
GeoFitBench-LLM is a benchmark designed to evaluate large language models (LLMs) on their ability to elicit geospatial data requirements for geo‑analytic tasks. It focuses on two reasoning dimensions—spatio‑temporal relational and semantic grounding—and uses a provider‑independent schema to encode ground‑truth requirements. Experiments on seven models show that while many can identify necessary resources, accurately specifying fitness‑for‑use remains challenging, with GPT‑5.4 Thinking recovering all resources in 66.67% of cases.