arXiv AI

Provenance-Aware Belief Updating in Human-AI Feedback Loops

The paper introduces Latent Quotient Evidence Training (LQET), a relational objective that uses report lineage to guide language models in updating beliefs based on evidence provenance rather than report frequency. Experiments on a binary benchmark show LQET reduces new‑evidence update RMSE by 41‑44% and log‑odds drift by 89‑91% compared to task‑only training, and improves performance in a closed‑loop simulation where model confidence controls report re‑inclusion. By grounding belief updates in supplied provenance, LQET enables dynamic human‑AI systems where influence follows new evidence instead of repeated mentions.

arXiv AI
Sep 17

GraphEcho: Structural Redundancy and Evidence Provenance in LLM Graph Agents

GraphEcho is a benchmark that examines how large language model agents navigate graph paths and handle evidence redundancy. It tests whether agents treat repeated encounters as additional corroboration by varying path counts and evidential origins while keeping evidence content constant. The study finds that redundant paths increase repeated walks, and that provenance-aware post‑training can reduce revisits but may limit source diversity, revealing a gap between efficient exploration and effective evidence use.

By Sikun Wang, Yixi Zhou, Lei Fan, Fan Zhang
arXiv AI
Sep 10

Evaluating and Improving Evidence-Grounded Fact-Checking in LLMs via Multi-Round Evidence Ablation

The paper introduces Fact-Ablated Evaluation (FAE), a framework that iteratively removes cited evidence to test whether large language models (LLMs) adjust their fact‑checking predictions accordingly. Experiments reveal that many off‑the‑shelf LLMs rely more on internal knowledge than on the provided evidence. To address this, the authors propose REAL, a training method that uses counterfactual evidence supervision to encourage LLMs to base veracity judgments on evidence, achieving better evidence‑dependent performance across four datasets.

By Xingyu Deng, Mingzi Cao, Nikolaos Aletras, Xi Wang, Mark Stevenson
arXiv AI
Oct 28

Clearing the FoG: Training KG Agents to Retrieve the Evidence Behind Their Answers

The paper introduces FoG, a framework that trains knowledge graph (KG) agents to retrieve the evidence behind their answers. FoG combines evidence‑grounded synthesis, a KG interaction agent, and an evidence‑coverage process reward (ECPR) to ensure that every answer is supported by retrieved triples. Experiments on three KGQA benchmarks show that 3B–4B open‑source agents trained with FoG outperform larger baselines in both answer accuracy and evidence coverage, even with zero‑shot cross‑KG transfer.

By Zongyuan Sun, Shuai Zhao, Yingjun Du, Ke Shu, Ziyue Zhu, Haoran Luo
arXiv AI
Jun 30

Evidence-Informed LLM Beliefs for Continual Scientific Discovery

arXiv:2606. 29182v1 Announce Type: new Abstract: Open-ended scientific discovery with large language models (LLMs) increasingly operates as a long-horizon loop of hypothesis search and verification, where a reward signal guides which hypotheses to test next.

By Dhruv Agarwal, Reece Adamson, Andrew McCallum, Peter Clark, Ashish Sabharwal, Bodhisattwa Prasad Majumder
arXiv AI
Oct 19

CastFlow: Bridging Evidence Acquisition and Accurate Prediction in Agentic Time Series Forecasting

CastFlow is a role‑specialized agentic forecasting framework that combines general‑purpose tool reasoning with specialized temporal forecasting. It uses an Experience Memory to guide a frozen LLM in acquiring evidence via a multi‑view toolkit, then a lightweight specialized LLM refines predictions through evidence‑grounded reasoning. The two‑stage training strategy—supervised fine‑tuning followed by reinforcement learning with verifiable rewards—optimizes the specialized model based on final forecasting outcomes, achieving superior performance on nine out of ten real‑world benchmarks.

By Bokai Pan, Mingyue Cheng, Xiaoyu Tao, Zhiding Liu, Shuo Yu, Yuchong Wu, Qi Liu, Defu Lian, Enhong Chen
arXiv Computation and Language
Aug 21

When Text and Numbers Disagree: Evidence Arbitration in Large Language Models

arXiv:2608. 20116v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used in settings where textual summaries, numerical observations, and external tool outputs may provide conflicting evidence.

By Mattia Carletti, Edward Phillips, Fredrik K. Gustafsson, Patitapaban Palo, Lei Clifton, Danielle Belgrave, Xiao Gu, David A. Clifton