arXiv AI

From Parameters to Answers: How LLMs Retrieve and Use Their Internal Knowledge

The paper investigates how large language models (LLMs) such as Qwen, Llama, and Gemma use internal knowledge when answering questions. By performing layer‑wise interventions on the hidden state after the question, the authors compare how different request directions (pair‑conditioned vs. global) and answer types (noun, adjective, code) influence the model’s routing of information. The study finds that the influence of request direction varies across models and layers, with some models showing a sustained routing effect while others do not, highlighting distinct patterns of early readability, causal steering, and later content dependence.

arXiv AI
Jun 8

The Masked Advantage: Uncovering Local-Language Access to Cultural Knowledge in LLMs

arXiv:2606. 07422v1 Announce Type: cross Abstract: Large language models are increasingly used to answer culturally grounded questions across languages, yet it remains unclear whether local cultural knowledge is better accessed through English or the local language.

By Yang Zhang, Xiao Fei, Amr Mohamed, Sarah Almeida Carneiro, Mersin Konomi, Mingmeng Geng, Ahmed Asaad, Guokan Shang, Michalis Vazirgiannis
arXiv AI
Sep 1

Look It Up: Analysing Internal Web Search Capabilities of Modern LLMs

The paper evaluates how modern large language models use internal web search to answer factual questions. Using 783 static queries and 288 dynamic queries, the authors find that enabling retrieval improves accuracy on static questions but hurts confidence calibration. On dynamic queries, models often retrieve but still achieve less than 70% accuracy, mainly due to poor query formulation and source selection, indicating that internal web search works better as a quick verification tool than a full information‑retrieval system.

By Sahil Kale
arXiv Machine Learning
Sep 2

How Do Language Models Choose Between Context and Memory?

The paper investigates how language models decide between contextual information and their internal memory when the two conflict. By estimating "authority directions" from agreement prompts and swapping these directions between matched prompts, the authors show that such interventions can reproduce 30–68% of the shift in source choice across Qwen, Llama, and OLMo models. Cross‑task experiments reveal that authority directions learned on one task transfer only modestly (≈9%) to another, indicating that authority computations are largely task‑specific.

By Benjamin Shih, John Winnicki, Arianna Cao
arXiv Computation and Language
Nov 4

Less but Sufficient: Relation Knowledge Selects Compact Knowledge-Graph Contexts, and What External Checks Can and Cannot Catch

The paper presents a method that uses relation knowledge—either manually crafted or learned from training data—to select a compact set of knowledge‑graph facts for large language models (LLMs). It introduces derivability, consistency, and validation checks to determine whether an answer can be derived from the selected facts, whether the facts conflict, and whether the derived answer should replace the LLM’s output. Experiments show that seven open LLMs perform better with the compact context than with relevance‑ranked facts, achieving higher exact‑match accuracy on ComplexWebQuestions and outperforming SubgraphRAG across 53 LLM, benchmark, and prompt settings.

By Seunghui Jwa, Minsu Oh, Sangwoo Hong, Yeo Chan Yoon
arXiv AI
Nov 5

Reasoning over Non-Parametric Memory in Multilingual LLMs: A Cognitive-Based Analysis of Relational Knowledge

The paper introduces “Rel-Bloom,” a multilingual diagnostic benchmark of 450 multiple-choice questions that tests language models’ ability to reason over relational facts supplied directly in prompts. It evaluates 43 models across six languages and two multilingual settings, revealing that accuracy drops sharply from Apply to Analyze levels, that Evaluate items remain a bottleneck even for high-performing models, and that difficulty rankings are consistent across languages. The study also finds that counterfactual contexts are generally harder than factual ones, and that removing entity identities can sometimes improve higher-level performance.

By Zohreh Abbasi Liasi, Batoul Hammoud, Anas Madkoor, Ehsaneddin Asgari
arXiv Computation and Language
Oct 26

Context or Knowledge: How LLMs Attribute Their Own Answers

arXiv:2610.32109v1 Announce Type: new Abstract: Language models can answer a question from a supplied document or from their parametric knowledge. When later asked where the answer came from, what de...

By Kirill Afendulev, Alexey Dontsov, Alla Chepurova, Elena Tutubalina, Anton Korznikov
arXiv AI
Oct 2

The Geometry of Contextual Relations: Language Models Address Facts by Order of Mention

The paper investigates how large language models (LLMs) encode and retrieve facts based on their order of mention in a context. By measuring changes in hidden states when shifting questions from one fact to another, the authors identify an "ordinal vector" that points to a fact’s position in the list rather than its name. Across several models (Qwen, Gemma, Llama), these ordinal addresses are ordered, steerable, low‑rank, and emerge in mid‑layers, suggesting that LLMs locate facts by their mention order, mirroring aspects of human recall.

By Yufa Zhou
arXiv AI
Oct 2

Where's Waldo? Query-language Preference under Cross-lingual Knowledge Disparities

The paper "Where's Waldo? Query-language Preference under Cross-lingual Knowledge Disparities" introduces the Waldo benchmark, a multilingual QA dataset built from Wikipedia that focuses on knowledge gaps and conflicts across languages. It evaluates eight models in five languages and finds that when a fact is missing in one language, models tend to use evidence from the other language, but when conflicting accounts exist, responses align strongly with the query language, leading to different answers for semantically identical questions. The study also explores mitigation strategies, including ablating attention heads and LoRA-based training, which can reduce the preference gap by up to 61.5%.

By Dayeon Ki, Ruochen Zhang, Silviu Cucerzan, Ryen W. White, Ning Gao