The paper investigates how the linear serialization of tables influences the ability of instruction‑tuned language models to perform table lookup tasks. By comparing row‑major and column‑major formats on synthetic 256‑cell tables of varying aspect ratios, the authors find that the optimal serialization depends on table shape: row‑major works best for tall tables, while column‑major excels on wide tables. Error analysis reveals a complementary pattern—row‑major preserves the correct row but often misses the target column, whereas column‑major preserves the correct column but sometimes loses the row—leading the authors to introduce the concept of relational locality. "whyItMatters":"The study shows that failures attributed to table width are actually driven by the chosen serialization format, highlighting the importance of representation choices for language model performance on tabular data."
arXiv:2606. 09323v1 Announce Type: new Abstract: Tabular encoders are usually evaluated inside task-specific end-to-end pipelines, so models from different training paradigms are difficult to compare directly even when they operate on similar tabular signals.
arXiv:2610.29188v1 Announce Type: new Abstract: Tables in real-world documents, including scientific papers, financial reports, and statistical summaries, frequently use merged cells, multi-level hea...
arXiv:2605.12292v2 Announce Type: replace Abstract: Benchmarking tabular learning has revealed the benefit of dedicated architectures, pushing the state of the art. But real-world tables often contai...
The paper investigates the so‑called ‘structure tax’ that is often attributed to constraining large language model outputs to structured formats like JSON or XML. By evaluating various models, datasets, and schema designs, the authors find that the accuracy loss depends more on how the schema is organized than on the presence of structure itself. Specifically, reasoning‑first field ordering can match or surpass free‑form accuracy, whereas answer‑first ordering leads to significant drops, especially in smaller models. The study also shows that well‑designed schemas improve confidence calibration and the separability of correct versus incorrect representations in transformer layers.
The paper introduces a reproducible benchmark for evaluating attention mechanisms in tabular foundation models, focusing on the distinct row and column attention patterns that differ from language model attention. It compares several backends—Torch SDPA, FlashAttention variants, vLLM, and SageAttention—across realistic tabular shapes on A100, H100, and B200 GPUs, revealing that optimal backend choice varies by attention type, hardware, and model specifics. The study finds FlashAttention generally performs best, but CuDNN can outperform it for column attention on longer sequences, while SageAttention excels for large row sequences beyond 16k rows.
arXiv:2608. 02412v1 Announce Type: new Abstract: Large language models (LLMs) have become the default tool for a remarkable range of tasks, yet they have had conspicuously little success at one of the most common machine learning workloads: predictive analytics over tabular data.
The paper investigates reducing memory usage of n‑gram embedding tables in open‑weight language models by storing them in low precision. Experiments on nanochat, Llama, and GPT‑2 families show that four‑bit tables cut table memory by about 75% while retaining over 99% of the validation‑loss improvement. The study also demonstrates that re‑allocating the saved memory to larger tables can surpass the performance of full‑precision tables, and that similar savings are achievable in large models like DeepSeek‑V4.1‑Flash and Qwen3.8‑Flash‑Next with minimal impact on perplexity and downstream tasks.
TILE (Table-Interfaced LLM Embeddings) bridges large language models (LLMs) and table foundation models (TFMs) to improve tabular learning with string columns. It introduces four key techniques—row serialization, intermediate-layer extraction, columnar pooling, and by-pass features—to extract effective representations from LLMs for tabular data. When combined with TFMs, TILE achieves state‑of‑the‑art performance, outperforming the strongest baseline on 90% of splits across 100 datasets and ranking first on STRABLE’s 108 datasets.
arXiv:2607. 19257v1 Announce Type: cross Abstract: Practitioners make three prompt-design decisions with almost no controlled evidence behind them: how to format instructions and context (markdown, plain text, prose, or tabular), how many simultaneous instructions a system prompt can carry before compliance degrades, and how much context a model can hold before recall and honesty degrade.
arXiv:2606. 09578v1 Announce Type: new Abstract: Large Language Models (LLMs) and Vision-Language Models (VLMs) are increasingly evaluated on table reasoning tasks, but the role of table representation remains under-explored.
Table understanding is a core task in document intelligence, encompassing two key subtasks: table reconstruction and table visual question answering (TabVQA). While recent approaches predominantly rel...
arXiv:2607. 26000v1 Announce Type: cross Abstract: Tabular Foundation Models (TFMs) have emerged as novel approaches for tabular predictive tasks, demonstrating competitive predictive performance to ensemble tree-based models.