arXiv AI

WORTH Benchmark: Evidence Discovery under Not-Fully-Cooperative Information -- A Two-Episode Pilot

arXiv AI
3d ago

Trapped in Your Story: Evaluating Narrative-Induced Sycophancy in Large Language Models

Large language models (LLMs) tend to exhibit sycophancy by aligning with users’ expressed views. This study introduces a benchmark of 5,500 evaluation instances built from 500 debatable questions across ten domains, where the user stance is fixed but the justification is systematically varied across eleven conditions, including neutral, personal experience, causal framing, and worldview-based synthesis. Seven LLMs were evaluated, revealing that narrative structure can significantly influence agreement, with causal framing and bullet presentation boosting sycophancy while personal experience can reduce it.

By Runtai Yang, Dandan Zhu, Kaiwei Zhang, Guangtao Zhai
arXiv Computation and Language
Aug 27

ClueWeaver: Reward-Guided Dual-Agent Evidence Reasoning for Compact LLMs on Literary Long Narratives

ClueWeaver is a dual-agent framework designed to enable compact, locally deployable language models to answer questions about long literary narratives. The Finder agent retrieves passages that contain answer-critical clues, while the Interpreter agent derives the answer from those passages, generates rationales with paragraph-ID citations, and performs self-calibration for high-risk questions. Both agents are trained with reward-guided reinforcement learning to prioritize evidence retention, correctness, grounding, and concise explanations, resulting in improved performance and inspectability over end-to-end prompting.

By Jihao Zhu, Zhiwei Yang, Wenxiao Zhang, Junqian Zhao, Qi You, Fangqi Wang, Zheyuan Deng, Hanzhe Yang, Yu Liu, Jin B. Hong
arXiv AI
Jun 6

Narrative Knowledge Weaver: Narrative-Centric Retrieval-Augmented Reasoning for Long-Form Text Understanding

arXiv:2606. 05724v1 Announce Type: cross Abstract: Long-form narrative QA requires reasoning over evolving story worlds rather than isolated passages: answers may depend on earlier goals, changing character states, social relations, causal triggers, temporal position, and later consequences.

By Qiuyu Tian, Fengyi Chen, Yiding Li, Youyong Kong, Fan Guo, Yuyao Li, Jinjing Shen, Zhijing Xie, Yiyun Luo, Xin Zhang, Yingce Xia, Zequn Liu