arXiv AI
Calibrating Small Language Models for Claim Check-Worthiness Detection
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Machine Learning
Aug 27
How Robust Are Automated Fact-Checking Systems? A Cross-Benchmark Evaluation
The paper evaluates the robustness of automated fact‑checking systems by cross‑benchmarking nine models—including random baselines, fine‑tuned transformers, zero‑shot LLMs, and top AVeriTeC 2025 systems—across four datasets from scientific, open‑web, and climate domains. It finds that fine‑tuned models outperform zero‑shot LLMs on ClimateCheck, that system rankings vary strongly with domain and metric, and that replacing retrieved evidence with gold annotations boosts veracity accuracy by 14–22 points, underscoring retrieval as the main bottleneck. The authors provide code, pre‑processed datasets, and results to enable reproducible research.
arXiv AI
Jun 11
Soft-Prompt Tuning for Fair and Efficient LLM Benchmark Evaluation
arXiv:2606. 12117v1 Announce Type: cross Abstract: Benchmark scores often misrepresent a large language model's (LLM's) knowledge, because they rely, e.
arXiv Computation and Language
Aug 25
Claim-Level Confidence Calibration for Reliable Decision Making with Large Language Models
arXiv:2608.22483v1 Announce Type: new Abstract: Large Language Models (LLMs) increasingly support decision-making in high-stakes domains, but they often hallucinate and express confidence that is mis...
arXiv Machine Learning
Aug 18
Limits to scalable evaluation at the frontier: LLM as Judge won't beat twice the data
arXiv:2410. 13341v4 Announce Type: replace Abstract: High quality annotations are increasingly a bottleneck in the explosively growing machine learning ecosystem.
Hugging Face Trending Papers
Aug 26
How Robust Are Automated Fact-Checking Systems? A Cross-Benchmark Evaluation
Automated fact-checking (AFC) systems retrieve evidence and predict claim veracity, yet evaluations omit simple baselines, systems are developed for a single benchmark and cannot be trusted to general...
arXiv AI
Aug 11
PluginEval: A Diagnostic Benchmark for Fine-Grained Error Attribution in Function Calling
arXiv:2608. 08700v1 Announce Type: new Abstract: Reliable evaluation of tool routing is critical as Large Language Models increasingly operate as autonomous agents.