arXiv AI

Authority Bias in Conversational Search Engines for Academic Paper Recommendation

The study examines whether large language models (LLMs) used as conversational search engines for academic literature prioritize papers based on authority signals—such as author prestige, venue, and citations—rather than content. By keeping titles and abstracts constant and manipulating authority metadata across three counterfactual conditions (original, flipped, boosted), the researchers tested eight LLMs in a single-turn, top‑1 recommendation scenario. Results reveal a substantial, directional authority bias that varies across models and is only partially mitigated by prompt-level debiasing, while also highlighting a significant say‑do gap where debiasing instructions suppress authority mentions more quickly than authority-driven flips, leading to underestimation of behavioral bias.

Hugging Face Trending Papers

Jun 11

Authority, Truth, and Citation Bias: A Large-Scale Multi-Domain Benchmark for Studying Epistemic Susceptibility in Large Language Models

Large language models are increasingly deployed in citation-augmented settings, yet the effect of citation presence on model behavior independent of factual content remains poorly understood. We introduce AuthorityBench, a 220,564-prompt multi-domain benchmark that isolates how citation-based authority signals influence epistemic behavior in LLMs.

llms benchmarks safety

arXiv AI

Aug 25

On the Role of Citations in Preference Data

arXiv:2608.21376v1 Announce Type: cross Abstract: Many NLP tasks require systems to provide attribution in their outputs--i.e. citations to grounding sources. Attribution serves as a bulwark against...

By Yu Hou, Hal Daum\'e III, Rachel Rudinger, William Walden

llms nlp reinforcement-learning safety

arXiv Computation and Language

3d ago

Answer Bubbles: Information Exposure in AI-Mediated Search

arXiv:2603.16138v2 Announce Type: replace-cross Abstract: Generative search systems are increasingly replacing link-based retrieval with AI-generated summaries, yet little is known about how these sy...

By Michelle Huang, Agam Goyal, Koustuv Saha, Eshwar Chandrasekharan

llms

arXiv Computation and Language

2d ago

Citing Less Critically: LLMs Reshape the Rhetoric and Reach of Scientific Citation

arXiv:2609.01432v1 Announce Type: cross Abstract: Scientific citations carry rhetorical intent. Scholars may cite prior work positively (supporting), negatively (contrasting), or neutrally (mentionin...

By Yixuan Liu, Lin Chen, Zhuoqi Liu, Jianglin Lu, Dakota Murray

llms nlp safety

arXiv Machine Learning

Aug 4

Computational Approaches to Understanding Large Language Model Impact on Writing and Information Ecosystems

arXiv:2506. 17467v2 Announce Type: replace-cross Abstract: Large language models (LLMs) have shown significant potential to change how we write, communicate, and create, leading to rapid adoption across society.

By Weixin Liang

llms

arXiv AI

Aug 7

Large Language Models Threaten Double-blind Review

arXiv:2608. 05157v1 Announce Type: cross Abstract: Double blind peer review serves as the scientific community primary defense against status and affiliation bias.

By Bulambo Mwendelwa Gloire, Prasenjit Mitra

llms safety

arXiv AI

Aug 20

Self-prompting and cross-model consensus enable reproducible data extraction from scientific literature with large language models

The paper investigates how large language models can extract contextualized data from scientific literature. It presents four workflows: expert‑written prompts, self‑generated prompts, autonomous literature discovery, and dataset creation from guidelines. While models perform well with prompts, they struggle with context, hallucinate references, and still need human oversight for final validation.

By Valentin Romanov, Monique Bax, Steven Niederer

llms agents

arXiv AI

Jun 3

LLM-Assisted Reranking to Operationalize Nuanced Objectives in Recommender Systems

arXiv:2606. 02883v1 Announce Type: cross Abstract: Recommender systems have grown from content-organization tools into sophisticated systems that shape daily behavior.

By Amir Ghasemian, Homa Hosseinmardi, Upasana Dutta, Duncan J. Watts

llms rag

arXiv Computation and Language

1d ago

Are Non-English Papers Reviewed Fairly? Language-of-Study Bias in NLP Peer Reviews

arXiv:2604.07119v2 Announce Type: replace Abstract: Peer review plays a central role in the NLP publication process, but is susceptible to various biases. Here, we study language-of-study (LoS) bias:...

By Ehsan Barkhordar, Abdulfattah Safa, Verena Blaschke, Erika Lombart, Marie-Catherine de Marneffe, G\"ozde G\"ul \c{S}ahin

llms nlp safety

Hugging Face Trending Papers

Aug 19

Self-prompting and cross-model consensus enable reproducible data extraction from scientific literature with large language models

The paper evaluates browser-based large language models (LLMs) for extracting detailed, contextualized data from scientific papers. It presents four workflows: (1) expert-curated prompts yield good extraction but struggle with nuance; (2) LLMs can generate effective prompts from simple instructions; (3) autonomous literature discovery is challenging, with missing or hallucinated references; (4) LLMs can build new datasets from guidelines that align closely with human experts, yet still need human oversight. The study outlines a practical, auditable workflow where experts set standards, models cross-check extractions, and researchers resolve disputes, enabling scalable scientific data curation.

llms agents

arXiv AI

Jun 10

Quantifying Uncertainty in AI Visibility: A Statistical Framework for Generative Search Measurement

arXiv:2603. 08924v2 Announce Type: replace-cross Abstract: AI-powered answer engines are inherently non-deterministic: identical queries submitted at different times can produce different responses and cite different sources.

By Ronald Sielinski

llms

arXiv AI

Jun 30

How Do LLMs Cite? A Mechanistic Interpretation of Attribution in Retrieval-Augmented Generation

arXiv:2606. 28358v1 Announce Type: cross Abstract: Retrieval-Augmented Generation (RAG) aims to enhance the trustworthiness of Large Language Models (LLMs) by grounding their outputs in external documents, often using inline citations for verifiability.

By Ian van Dort (University of Amsterdam), Maria Heuss (University of Amsterdam)

llms rag benchmarks

Related stories