arXiv Computation and Language
Sep 7

Repeated Queries Exhaust an LLM's Brand Recommendations but Not Its Sources

The study examines how repeated identical buying questions affect the brand recommendations of large language models (LLMs) with and without web‑search retrieval. Across 300 question‑engine cells, five engines that did not use web search continued to add new, previously unseen brands up to run 15, while the single retrieval‑enabled engine’s list plateaued earlier. Domain citations continued to grow throughout the runs, indicating that LLMs keep accumulating source diversity even as brand lists stabilize.

By Dmitrij \.Zatuchin
arXiv AI
Oct 31

One Confidence Threshold, Unequal Coverage: Language Gaps in Typed Decision Interfaces and How Translating Inputs Reduces Them

The study evaluates whether a single confidence threshold in typed decision interfaces treats all languages equally. Using 1,000 items in 51 languages across three vendors, it finds that English consistently achieves higher coverage—clearing the 0.9 threshold 8.6–11.7 percentage points more often than other languages—despite similar accuracy. Even when thresholds are re‑weighted or calibrated per language, the gap persists, and translating inputs into English reduces but does not eliminate it. The authors recommend reporting per‑language coverage at the operating threshold.

By Junwei Yao, Yongxi Zhou, Zike Yang, Zhijun Wang, Jianan Liu, Yanxian Chen, Wenbo Ye
arXiv AI
Oct 2

Where's Waldo? Query-language Preference under Cross-lingual Knowledge Disparities

The paper "Where's Waldo? Query-language Preference under Cross-lingual Knowledge Disparities" introduces the Waldo benchmark, a multilingual QA dataset built from Wikipedia that focuses on knowledge gaps and conflicts across languages. It evaluates eight models in five languages and finds that when a fact is missing in one language, models tend to use evidence from the other language, but when conflicting accounts exist, responses align strongly with the query language, leading to different answers for semantically identical questions. The study also explores mitigation strategies, including ablating attention heads and LoRA-based training, which can reduce the preference gap by up to 61.5%.

By Dayeon Ki, Ruochen Zhang, Silviu Cucerzan, Ryen W. White, Ning Gao
arXiv AI
Sep 12

From Parameters to Answers: How LLMs Retrieve and Use Their Internal Knowledge

The paper investigates how large language models (LLMs) such as Qwen, Llama, and Gemma use internal knowledge when answering questions. By performing layer‑wise interventions on the hidden state after the question, the authors compare how different request directions (pair‑conditioned vs. global) and answer types (noun, adjective, code) influence the model’s routing of information. The study finds that the influence of request direction varies across models and layers, with some models showing a sustained routing effect while others do not, highlighting distinct patterns of early readability, causal steering, and later content dependence.

By Wenkang Wei, Yuan Fang, Renhe Jiang, Hong Cheng, Xingtong Yu