arXiv AI

When LLMs Read Tables Carelessly: Measuring and Reducing Data Referencing Errors

arXiv:2606. 32029v1 Announce Type: cross Abstract: While large language models (LLMs) perform well on table tasks, they still make data referencing errors (DREs), i.

arXiv AI

Aug 6

When Large Language Models Know the Table: A Framework for Assessing Data Contamination in Tabular Datasets

arXiv:2510. 20351v3 Announce Type: replace-cross Abstract: Large language models (LLMs) are increasingly exposed to data contamination, i.

By Matteo Silvestri, Fabiano Veglianti, Flavio Giorgi, Fabrizio Silvestri, Gabriele Tolomei

llms

arXiv AI

Jun 18

TopBench: A Benchmark for Implicit Predictive Reasoning in Tabular Question Answering

arXiv:2604. 28076v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) have advanced Table Question Answering, where most queries can be answered by extracting information or simple aggregation.

By An-Yang Ji, Jun-Peng Jiang, De-Chuan Zhan, Han-Jia Ye

llms agents nlp benchmarks

arXiv AI

Jul 22

TReB: A Comprehensive Benchmark for Evaluating Table Reasoning Capabilities of Large Language Models

arXiv:2506. 18421v3 Announce Type: replace-cross Abstract: The majority of data in businesses and industries is stored in tables, databases, and data warehouses.

By Ce Li, Xiaofan Liu, Zhiyan Song, Ce Chi, Boshen Shi, Chen Zhao, Guanguang Chang, Zhendong Wang, Kexin Yang, Xing Wang, Chao Deng, Junlan Feng

llms benchmarks

arXiv AI

1d ago

Efficient Table QA via TableGrid Navigation and Progressive Inference Prompting

arXiv:2605. 20254v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) have shown promising results on NLP tasks, however, their performance on tabular data still needs research attention, because Table Question-Answering (TQA) requires precise cell retrieval and multi-step structured reasoning.

By Amritansh Maurya, Navjot Singh, Mohammed Javed, Omar Moured

llms nlp fine-tuning benchmarks

arXiv AI

Jun 18

DRIFT: Refining Instruction Data via On-Policy Data Attribution

arXiv:2606. 18307v1 Announce Type: cross Abstract: Optimizing the training data distribution for Supervised Fine-Tuning (SFT) dictates the capability of Large Language Models (LLMs).

By Zefan Wang, Lincheng Li, Tianyu Yu, Yuan Yao

llms fine-tuning safety

arXiv AI

Jun 3

Fixing FOLIO and MALLS: Verified Annotations and an LLM-assisted Framework to Focus Human Relabeling

arXiv:2606. 02837v1 Announce Type: cross Abstract: Accurate translation from Natural Language to First-Order Logic (NL-to-FOL) underpins neurosymbolic AI systems and Natural Language Inference (NLI), making the quality of NL-to-FOL benchmarks essential -- yet these datasets have never been rigorously audited.

By Andrea Brunello, Cristian Curaba, Luca Geatti, Michele Mignani, Angelo Montanari, Nicola Saccomanno

llms benchmarks

arXiv AI

Jul 21

BERT-as-a-Judge: A Robust Alternative to Lexical Methods for Efficient Reference-Based LLM Evaluation

arXiv:2604. 09497v2 Announce Type: replace-cross Abstract: Accurate evaluation is central to the large language model (LLM) ecosystem, guiding model selection and downstream adoption across diverse use cases.

By Hippolyte Gisserot-Boukhlef, Nicolas Boizard, Emmanuel Malherbe, C\'eline Hudelot, Pierre Colombo

llms

arXiv Machine Learning

Aug 3

TELLER: Dual-Path Iterative Preference Optimization for Table Entity Linking

arXiv:2607. 28680v1 Announce Type: cross Abstract: Entity linking in tables matches short and ambiguous cell mentions to their corresponding knowledge-base entities.

By Yixin Peng, Kehao Li, Stefan Decker

fine-tuning safety

arXiv AI

Jul 23

Auto-Fill: Learning to Predict Missing Values Accurately with Specialist Language Models

arXiv:2607. 19847v1 Announce Type: cross Abstract: Predicting missing cell values in tabular data is a fundamental problem in data cleaning.

By Yurong Liu, Yeye He, Haoyu Dong, Junjie Xing, Shi Han, Dongmei Zhang, Surajit Chaudhuri

llms benchmarks

arXiv AI

Jun 26

\textsc{DiARC}: Distinguishing Positive and Negative Samples Helps Improving ARC-like Reasoning Ability of Large Language Models

arXiv:2606. 26530v1 Announce Type: cross Abstract: The Abstraction and Reasoning Corpus (ARC;~\citealp{chollet2019measure}) contains tasks that require summarizing patterns from limited grid samples and predicting output grids.

By Yuxuan Yang, Feiyang Li, Yile Wang

llms fine-tuning benchmarks safety

arXiv AI

2d ago

The Metacognitive Bottleneck: Japanese Riddles Reveal Fundamental Limits of Machine Insight and Self-Evaluation in Reasoning AI

arXiv:2509. 14704v3 Announce Type: replace Abstract: Benchmark saturation and training-data contamination increasingly obscure whether reported gains in large language models (LLMs) reflect genuine advances in reasoning or familiarity with recurring patterns in benchmark problems.

By Masaharu Mizumoto, Dat Nguyen, Zhiheng Han, Xingfu Li, Yo Nakawake, Le Minh Nguyen

llms benchmarks

arXiv AI

Jun 10

Flaws in the LLM Automation Narrative

arXiv:2606. 11166v1 Announce Type: cross Abstract: Large Language Models (LLMs) are increasingly described as performing at the level of human experts on knowledge economy tasks.

By George Perrett, Javae Elliott, Jennifer Hill, Marc Scott

llms benchmarks

Related stories