arXiv Computation and Language
Retrieval Sensitivity to Identity Signals in Queries
Read the original on arXiv Computation and Language →The Flow has not summarised this story yet — read it at arXiv Computation and Language.
Related stories
arXiv Computation and Language
Sep 1
WildSEEK: Evaluating Language Models for Information-Seeking
WildSEEK is a new dataset of 3,000 real user information‑seeking queries, manually annotated for risk‑sensitive domains and whether the query is factoid or analytical. The accompanying evaluation framework tests LLM responses against four failure criteria—sycophantic behavior, overreliance, a default US‑centric perspective, and poor handling of vulnerable populations—finding higher failure rates for analytical queries. The authors also train classifiers on WildSEEK to analyze over 1.8 million realistic queries, revealing that more than a third are high‑risk and often analytical.
arXiv Computation and Language
3d ago
Framing the Narrative: Ideological Mimicry in Large Language Models
arXiv:2609.38256v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used to answer questions about politically contentious issues, yet evaluations typically treat a model's...
arXiv AI
Sep 3
PolERo: Studying Political Evasion in Romanian
PolERo presents a new dataset of 3,574 Romanian question‑answer pairs from presidential transcripts, annotated for political evasion using a two‑level taxonomy of response clarity and fine‑grained evasion strategies. The study evaluates various classification methods—including TF‑IDF baselines, fine‑tuned encoders, a sliding‑window encoder, and zero/few‑shot LLM prompting—under matched conditions. Cross‑lingual transfer experiments via joint bilingual training and machine‑translation augmentation reveal that fine‑tuned encoders perform competitively, transfer is asymmetric, and ambivalent evasion categories with pragmatic cues remain the most challenging across all models.
arXiv AI
Sep 1
How Identity and Opinion Shape Political Sycophancy in LLMs
arXiv:2608.29198v1 Announce Type: new Abstract: As Large Language Models (LLMs) increasingly encourage users to disclose personal profiles for tailored assistance, measuring their political alignment...
arXiv AI
Sep 17
From a River in Gilead to the Inference Distributions of Large Language Models: Covert Dialect Bias and Linguistic Profiling at Scale
The paper investigates covert dialect bias in large language models (LLMs) by analyzing how internal probability distributions associate different English varieties—Standard American English, African American Vernacular English, Nigerian Standard English, and Nigerian Pidgin—with housing-related adjectives. Using 260 meaning‑matched sentence quadruples and log‑probability scoring across ten open‑weight LLMs, the study finds that AAVE and NP are consistently linked to more negative adjectives than SAE, with NP experiencing the greatest penalty. The bias varies by context and stereotype cluster, and Nigerian Standard English shows a context‑dependent shift, being favored in formal tenant screening but penalized in more socially proximate scenarios.
arXiv Computation and Language
Sep 2
Sources of Truth: A Multi-Platform, Multilingual Audit of Citations in AI Mental Health Information Queries
arXiv:2609.00319v1 Announce Type: cross Abstract: Online health information seeking is shifting from keyword search, where users consider a ranked list of links, to conversational systems that compos...