arXiv AI

Can Language Models Play the Wikipedia Game?

The paper "Can Language Models Play the Wikipedia Game?" investigates how language models perform on an information‑retrieval task where players navigate from one Wikipedia article to a target using only article links. The authors compare sentence‑embedding models and large language models across five Wikipedia languages and five academic‑literature datasets, building extensive local graphs of Wikipedia pages and paper citations with OpenAlex. They benchmark these models against human and TF‑IDF baselines, finding that no model significantly outperforms humans, though the best models (GPT‑o3 mini, GPT‑4o, QwQ:32B, and DeepSeek‑R1:70B) achieve comparable success, with performance varying by language and topic.

arXiv Computation and Language
Sep 14

Parameter-Efficient Retrievers for Polish and European Languages

The paper introduces a three‑stage training pipeline that builds compact, efficient dense retrievers without requiring ground‑truth relevance labels. Using cross‑lingual alignment, relational knowledge distillation, and contrastive fine‑tuning, the authors develop PolDense (six Polish models ranging from 17 M to 1 B parameters) and EuroDense (a 435 M‑parameter model covering nine European languages). Extensive evaluation on 41 Polish and 150 multilingual tasks shows that PolDense‑1B outperforms larger retrievers up to 9 B parameters, while EuroDense leads in task‑averaged and language‑averaged performance among models below 1 B parameters.

By S{\l}awomir Dadas, Rafa{\l} Po\'swiata, Ma{\l}gorzata Gr\k{e}bowiec, Micha{\l} Pere{\l}kiewicz
arXiv AI
Jul 31

SimpleWikiSearch: A Clean Offline Wikipedia Environment for Agentic Search

arXiv:2607. 26070v1 Announce Type: cross Abstract: Large language model (LLM)-based agentic search systems are often evaluated as if the underlying LLM were the only component that matters, yet their measured performance also depends on the surrounding search environment: the Wikipedia snapshot, preprocessing pipeline, chunking policy, retrieval backend, tool schema, observation format, and answer submission rule.

By Guanming Xiong, Penghui Zhang
arXiv Computation and Language
Sep 11

Think Before You Link: Rarity, Reasoning, and Retrieval in Multilingual Entity Linking

The paper introduces a new perspective on entity rarity in multimodal entity linking by using knowledge‑graph structural metrics instead of popularity metrics, revealing many rare entities previously overlooked. Experiments show that state‑of‑the‑art models suffer a 15.4–39.9% accuracy drop on these rare‑entity slices. The authors propose a training‑free framework that combines reasoning and retrieval with a vision‑language model, achieving a 6.9% overall accuracy gain and up to 23.3% improvement on rare entities, and release a new benchmark MERLIN‑Rare for focused evaluation.

By Parinthapat Pengpun, Simran Khanuja, Graham Neubig
arXiv Computation and Language
Oct 8

When Trivia Is Not Trivial: Everyday Knowledge Failures in Multilingual LLMs

The paper introduces TriviaRoomQA, a multilingual benchmark of 3,300 parallel multiple-choice questions across 288 topics, plus an additional 5,340 French-only questions. It evaluates 30 open-weight large language models (7–70 B parameters) from various regions, finding strong performance on knowledge-intensive subjects like history and geography but weaker results on everyday popular‑culture topics such as celebrities, music, movies, and news. The study also shows that model performance varies across languages for the same questions, indicating that factual knowledge access is not always language‑independent.

By Anna Mosolova, Djam\'e Seddah
arXiv AI
Oct 9

RIT-RAG: Navigating Document Corpora with Retrieval-Induced Trees

RIT‑RAG is a retrieval‑augmented generation framework that combines content retrieval with structural navigation of documents. It builds a tree for each document from its table of contents or sitemap, then uses retrieved chunk positions to induce manageable sub‑trees that an LLM agent can navigate, read selectively, and reformulate queries. Experiments on financial, scientific, and customer‑support benchmarks show RIT‑RAG outperforms vanilla, graph‑based, and agentic baselines, achieving up to 11.4‑point accuracy gains on the new EntQABench of 2.84 million technical‑documentation webpages.

By Meghanadh Pulivarthi, Swaraj Kumar Biswal, Kushagra Bhushan, Yatin Nandwani, Sachindra Joshi, Dinesh Raghu
arXiv AI
Jun 12

SkMTEB: Slovak Massive Text Embedding Benchmark and Model Adaptation

arXiv:2606. 13647v1 Announce Type: cross Abstract: We introduce SkMTEB, the first comprehensive MTEB-style text embedding benchmark for Slovak, a low-resource West Slavic language, comprising 31 datasets across 7 task types -- nearly 4$\times$ the depth of existing multilingual benchmark coverage for Slovak.

By Marek \v{S}uppa, Andrej Ridzik, Daniel Hl\'adek, Nat\'alia K\v{n}a\v{z}ekov\'a, Vikt\'oria Ondrejov\'a
arXiv AI
Sep 17

Knowledge-Graph Based Augmentation versus Retrieval Augmented Generation for Cultural-Related Question Answering

The paper compares Knowledge-Graph Based Augmentation (Graph-RAG) with Retrieval-Augmented Generation (RAG) for answering culturally specific questions. Using the LatamQA dataset, Graph-RAG, built automatically from Wikipedia via KGGen, matches RAG performance and reduces the base LLM’s error by 72% with a standard KG and 78% with a benchmark-aware variant. The approach also transfers zero‑shot to Portuguese, showing multilingual applicability.

By Pablo Poulenard, Yannis Karmim, Valentin Barri\`ere