arXiv Computation and Language

The Language of the Question Selects the Market: Query Language and Exit IP as Separable Factors in Commercial Recommendations from a Generative Search Interface

arXiv Computation and Language
Sep 7

Repeated Queries Exhaust an LLM's Brand Recommendations but Not Its Sources

The study examines how repeated identical buying questions affect the brand recommendations of large language models (LLMs) with and without web‑search retrieval. Across 300 question‑engine cells, five engines that did not use web search continued to add new, previously unseen brands up to run 15, while the single retrieval‑enabled engine’s list plateaued earlier. Domain citations continued to grow throughout the runs, indicating that LLMs keep accumulating source diversity even as brand lists stabilize.

By Dmitrij \.Zatuchin
arXiv AI
Oct 31

One Confidence Threshold, Unequal Coverage: Language Gaps in Typed Decision Interfaces and How Translating Inputs Reduces Them

The study evaluates whether a single confidence threshold in typed decision interfaces treats all languages equally. Using 1,000 items in 51 languages across three vendors, it finds that English consistently achieves higher coverage—clearing the 0.9 threshold 8.6–11.7 percentage points more often than other languages—despite similar accuracy. Even when thresholds are re‑weighted or calibrated per language, the gap persists, and translating inputs into English reduces but does not eliminate it. The authors recommend reporting per‑language coverage at the operating threshold.

By Junwei Yao, Yongxi Zhou, Zike Yang, Zhijun Wang, Jianan Liu, Yanxian Chen, Wenbo Ye
arXiv AI
Oct 2

Where's Waldo? Query-language Preference under Cross-lingual Knowledge Disparities

The paper "Where's Waldo? Query-language Preference under Cross-lingual Knowledge Disparities" introduces the Waldo benchmark, a multilingual QA dataset built from Wikipedia that focuses on knowledge gaps and conflicts across languages. It evaluates eight models in five languages and finds that when a fact is missing in one language, models tend to use evidence from the other language, but when conflicting accounts exist, responses align strongly with the query language, leading to different answers for semantically identical questions. The study also explores mitigation strategies, including ablating attention heads and LoRA-based training, which can reduce the preference gap by up to 61.5%.

By Dayeon Ki, Ruochen Zhang, Silviu Cucerzan, Ryen W. White, Ning Gao
arXiv AI
Sep 12

From Parameters to Answers: How LLMs Retrieve and Use Their Internal Knowledge

The paper investigates how large language models (LLMs) such as Qwen, Llama, and Gemma use internal knowledge when answering questions. By performing layer‑wise interventions on the hidden state after the question, the authors compare how different request directions (pair‑conditioned vs. global) and answer types (noun, adjective, code) influence the model’s routing of information. The study finds that the influence of request direction varies across models and layers, with some models showing a sustained routing effect while others do not, highlighting distinct patterns of early readability, causal steering, and later content dependence.

By Wenkang Wei, Yuan Fang, Renhe Jiang, Hong Cheng, Xingtong Yu
Hugging Face Trending Papers
Aug 2

Language Equality has a Price: A Systematic Investigation of Multi-turn LLM Performance for EU-24+

We evaluate large language models (LLMs) as language agents playing goal-directed dialogue games in self-play across 30 languages: the 24 official EU languages plus six others. Unlike static or preference-based evaluation, this paradigm is multi-turn, reference-free and programmatically scored, and because the game mechanics are language-agnostic it extends to a new language by localising a fixed set of prompt and word-list files.

arXiv Computation and Language
Sep 17

"If I Had to Buy Just ONE: Galaxy S26 Ultra": Auditing AI-Generated Product Recommendations

arXiv:2609.18729v1 Announce Type: cross Abstract: Consumers increasingly use AI chatbots for advice on what to buy. With companies like OpenAI and Google monetising their AI through advertising, this...

By Lucas G. Uberti-Bona Marin, Thales Bertaglia, Giovanni Astante, Bram Rijsbosch, Gijs van Dijck, Anik\'o Hann\'ak, Gerasimos Spanakis, Konrad Kollnig
arXiv AI
Oct 7

Tool-calling retrieval versus vector RAG for a small Greek--English knowledge base: accuracy and robustness to how users type Greek

The study compares two methods for grounding assistants in a small Greek–English agricultural knowledge base: tool‑calling retrieval via a live data interface and vector retrieval‑augmented generation (RAG). Using the KyGround benchmark of 198 questions, vector RAG achieved 95.3% accuracy on canonical Greek questions, outperforming the tool agent’s 71.6% and revealing that the tool agent’s failures stem from literal searches that miss non‑verbatim matches. The results show that search tolerance to user typing variations—such as accents, capitalization, and Greeklish transliterations—is crucial for reliable community knowledge interfaces.

By Nikolaos D. Tantaroudas, Ilias Karachalios, Andrew J. McCracken
arXiv AI
6d ago

PolyRoute: A Twelve-Language and Code-Mixed Benchmark for Typed-Decision Routing, with Baselines and a Leaderboard

PolyRoute is a benchmark for typed-decision routing that includes 322 native-language messages across 12 languages and 9 writing systems, 156 code-mixed or romanised messages, and 967 optional native training rows. The dataset is annotated by LLMs (Qwen3.8-27B and gemma4-31b) and evaluates a three-way department-routing decision protocol. Leaderboard results show multilingual models outperform English-only baselines, with the best fine-tuned system achieving 96.6% accuracy.

By Mohammed Nayeem Khan, Sarun Kumar K S