arXiv Computation and Language

Althea: The Fact-Checking--Metalearning Tradeoff in AI-Assisted Verification

Althea is a retrieval‑augmented system that supports user‑driven claim evaluation, matching standard pipelines on AVeriTeC while improving discrimination between supported and refuted claims. In a longitudinal survey experiment with 961 participants, two AI‑assisted treatments—Exploratory (guided reasoning) and Summary (synthesized verdicts)—initially boosted accuracy and confidence, but these gains faded after the system was removed, leaving no advantage over unrelated news. In contrast, a Self‑search baseline, which lacks a fading procedure, maintained a significant advantage, highlighting a fact‑checking–metalearning tradeoff where methods that improve immediate accuracy may not foster durable literacy gains.

arXiv Machine Learning
Sep 22

EAVer: Long-Form Factuality Verification as an End-to-End Agentic Policy

arXiv:2609.22223v1 Announce Type: cross Abstract: Long-form factuality verification is commonly implemented as a static decompose-search-verify pipeline, with separately prompted modules processing c...

By Kening Zheng, Aoying Zheng, Zhigang Chang, Yazhi Guo, Miaotian Guo, Qingwei Zong, Xianhai Xie, Weiqiang Jin, Chengze Li, Hanrong Zhang, Jie Yang, Wei-Chieh Huang, Lingzhe Zhang, Liancheng Fang, Xin Zou, Hanqian Li, Jiahao Huo, Yibo Yan, Zizhuang Deng, Lei Miao, Wei Guo, Haihong Tang, Bo Zheng, Philip S. Yu
arXiv AI
Aug 17

The Metacognitive Bottleneck: Japanese Riddles Reveal Fundamental Limits of Machine Insight and Self-Evaluation in Reasoning AI

arXiv:2509. 14704v3 Announce Type: replace Abstract: Benchmark saturation and training-data contamination increasingly obscure whether reported gains in large language models (LLMs) reflect genuine advances in reasoning or familiarity with recurring patterns in benchmark problems.

By Masaharu Mizumoto, Dat Nguyen, Zhiheng Han, Xingfu Li, Yo Nakawake, Le Minh Nguyen
arXiv AI
Aug 20

The Lifecycle of LLM-as-a-Judge for Large-Scale Recommendation Explanations

The paper introduces a lifecycle framework for LLM-as-a-Judge systems used to evaluate recommendation explanations at Netflix. It outlines four phases—Birth, Training, Deployment, and Monitoring—detailing how each stage addresses specific technical and operational challenges. The authors report that after five weeks of A/B testing, judge-aligned explanations increased novel content viewing and successful browse-to-play sessions without quality takedowns.

By Emma Yanyang Kong, JJ Tan, Ishan Gupta, Lars Olds, Claire Campbell, David Fagnan, Veli Balin, Rohan Gosain, Louis Garcia, Minsu Jang
arXiv AI
Aug 26

Generating Biomedical Fact-Checking Reports with RL-Enhanced Agentic Search

The paper introduces BioCheck Agent, an LLM-based system that generates structured biomedical fact‑checking reports using agentic search and a reinforcement‑learning framework called EG‑GRPO. Unlike prior methods that output only supported or refuted labels, BioCheck Agent synthesizes conclusions with retrieved evidence from PubMed, employing advanced Boolean search operators. Experiments show that, compared to the base Qwen3.5‑4B model, BioCheck Agent improves label prediction accuracy on SciFact by 9.95 %, raises evidence quality by 3.7 %, and reduces hallucinations by 19.63 %.

By Jiongxiao Wang, Dingli Ma, Chaoqun Ni
arXiv AI
Sep 10

Evaluating and Improving Evidence-Grounded Fact-Checking in LLMs via Multi-Round Evidence Ablation

The paper introduces Fact-Ablated Evaluation (FAE), a framework that iteratively removes cited evidence to test whether large language models (LLMs) adjust their fact‑checking predictions accordingly. Experiments reveal that many off‑the‑shelf LLMs rely more on internal knowledge than on the provided evidence. To address this, the authors propose REAL, a training method that uses counterfactual evidence supervision to encourage LLMs to base veracity judgments on evidence, achieving better evidence‑dependent performance across four datasets.

By Xingyu Deng, Mingzi Cao, Nikolaos Aletras, Xi Wang, Mark Stevenson