arXiv AI
Jul 21
Kernelized Linear Attention: Breaking the Capacity Wall with Symmetric Cones
arXiv:2607. 17419v1 Announce Type: cross Abstract: Linear attention promises constant-time recurrent inference but degrades sharply on associative recall.
LimiX-2M: Mitigating Low-Rank Collapse and Attention Bottlenecks in Tabular Foundation Models
arXiv:2606. 04485v1 Announce Type: new Abstract: Tabular foundation models (TFMs) increasingly rival tree ensembles, but their performance is often compute-inefficient: with standard affine scalar tokenization, each feature injects value variation through an essentially one-dimensional channel, and feature IDs/positional signals cannot increase within-feature value degrees of freedom, yielding weak early-layer value sensitivity and redundant hidden states.
Related stories
arXiv Machine Learning
Jun 8
TabSwift: An Efficient Tabular Foundation Model with Row-Wise Attention
arXiv:2606. 07345v1 Announce Type: new Abstract: Tabular foundation models, exemplified by TabPFN, perform prediction via in-context learning, inferring test labels directly from labeled training examples.
arXiv Machine Learning
Jul 31
WIDE: Boosting Adaptive LLM Inference via Token-level Dynamic Width Pruning
arXiv:2607. 28418v1 Announce Type: cross Abstract: Pruning is a promising approach for improving the efficiency of LLMs.
arXiv AI
Jun 12
MiniMax Sparse Attention
arXiv:2606. 13392v1 Announce Type: new Abstract: Ultra-long-context capability is becoming indispensable for frontier LLMs: agentic workflows, repository-scale code reasoning, and persistent memory all require the model to jointly attend over hundreds of thousands to millions of tokens, yet the quadratic cost of softmax attention makes this untenable at deployment scale.
arXiv Machine Learning
1d ago
Localized TabICLv2: Scaling Tabular In-Context Learning through k-NN
arXiv:2608. 16429v1 Announce Type: new Abstract: Foundational models for tabular data have made significant progress in recent years, with TabICLv2 reporting state-of-the-art performance on several tabular classification tasks.
arXiv Machine Learning
Jul 20
VarRate: Training-Free Variable-Rate KV Cache Compression for Long-Context LLMs
arXiv:2607. 15498v1 Announce Type: cross Abstract: The key-value (KV) cache is the main memory bottleneck in long-context large language model (LLM) inference.
arXiv AI
Jul 16
Adaptive Filtering of the KV Cache: Diagnosing and Correcting Structural-Role Bias in LLM Inference
arXiv:2607. 13205v1 Announce Type: cross Abstract: Attention-based KV cache eviction (H2O and its descendants) compresses the memory-constrained state of a long-context model by ranking tokens on accumulated attention mass, treated here as signal energy, and keeping the heaviest.
arXiv Machine Learning
Jun 2
TabPrep: Closing the Feature Engineering Gap in Tabular Benchmarks
arXiv:2606. 02384v1 Announce Type: new Abstract: Progress in tabular machine learning has largely focused on increasingly sophisticated model architectures.
arXiv Machine Learning
5d ago
TabH2O: A Unified Foundation Model for Tabular Prediction
arXiv:2605. 18383v2 Announce Type: replace Abstract: We present TabH2O, a foundation model for tabular data that performs classification and regression in a single forward pass via in-context learning.
arXiv AI
Jun 9
TRL-Bench: Standardizing Cross-Paradigm Representation-Level Evaluation of Tabular Encoders
arXiv:2606. 09323v1 Announce Type: new Abstract: Tabular encoders are usually evaluated inside task-specific end-to-end pipelines, so models from different training paradigms are difficult to compare directly even when they operate on similar tabular signals.
arXiv Machine Learning
Jun 5
GOTabPFN: From Feature Ordering to Compact Tokenization for Tabular Foundation Models on High-Dimensional Data
arXiv:2606. 05441v1 Announce Type: new Abstract: We investigate how to make small tabular foundation models effective for High-Dimensional, Low-Sample Size (HDLSS) tabular prediction without retraining large backbones.
arXiv Machine Learning
6d ago
FLARE++: Low-rank attention with dynamic attention routing
arXiv:2608. 11519v1 Announce Type: new Abstract: Full self-attention is a strong token mixer for PDE surrogates on irregular domains, but its quadratic cost limits its use on high-resolution problems.