arXiv Machine Learning June 4, 2026 By Yuanrui Wang, Xingxuan Zhang, Han Yu, Mingchao Ming, Gang Ren, Hao Yuan, Li Mao, Yunjia Zhang, Chun Yuan, Peng Cui

LimiX-2M: Mitigating Low-Rank Collapse and Attention Bottlenecks in Tabular Foundation Models

Read the original on arXiv Machine Learning →

arXiv:2606. 04485v1 Announce Type: new Abstract: Tabular foundation models (TFMs) increasingly rival tree ensembles, but their performance is often compute-inefficient: with standard affine scalar tokenization, each feature injects value variation through an essentially one-dimensional channel, and feature IDs/positional signals cannot increase within-feature value degrees of freedom, yielding weak early-layer value sensitivity and redundant hidden states.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv AI

Jul 21

Kernelized Linear Attention: Breaking the Capacity Wall with Symmetric Cones

arXiv:2607. 17419v1 Announce Type: cross Abstract: Linear attention promises constant-time recurrent inference but degrades sharply on associative recall.

By Ayoub Ghriss, Sourav Chakraborty

llms rag efficiency benchmarks

arXiv Machine Learning

Jun 8

TabSwift: An Efficient Tabular Foundation Model with Row-Wise Attention

arXiv:2606. 07345v1 Announce Type: new Abstract: Tabular foundation models, exemplified by TabPFN, perform prediction via in-context learning, inferring test labels directly from labeled training examples.

By Si-Yang Liu, Han-Jia Ye

llms

arXiv Machine Learning

Jul 31

WIDE: Boosting Adaptive LLM Inference via Token-level Dynamic Width Pruning

arXiv:2607. 28418v1 Announce Type: cross Abstract: Pruning is a promising approach for improving the efficiency of LLMs.

By Haozhe Hu, Hao Wu, Peiran Yin, Chao Han, Yunpu Ma, Xiaoyu Shen

llms nlp efficiency benchmarks

arXiv AI

Jun 12

MiniMax Sparse Attention

arXiv:2606. 13392v1 Announce Type: new Abstract: Ultra-long-context capability is becoming indispensable for frontier LLMs: agentic workflows, repository-scale code reasoning, and persistent memory all require the model to jointly attend over hundreds of thousands to millions of tokens, yet the quadratic cost of softmax attention makes this untenable at deployment scale.

By Xunhao Lai, Weiqi Xu, Yufeng Yang, Qiaorui Chen, Yang Xu, Lunbin Zeng, Xiaolong Li, Haohai Sun, Haichao Zhu, Vito Zhang, Pengyu Zhao

llms agents efficiency multimodal

arXiv Machine Learning

1d ago

Localized TabICLv2: Scaling Tabular In-Context Learning through k-NN

arXiv:2608. 16429v1 Announce Type: new Abstract: Foundational models for tabular data have made significant progress in recent years, with TabICLv2 reporting state-of-the-art performance on several tabular classification tasks.

By Beimnet Bekele Guta

llms fine-tuning benchmarks

arXiv Machine Learning

Jul 20

VarRate: Training-Free Variable-Rate KV Cache Compression for Long-Context LLMs

arXiv:2607. 15498v1 Announce Type: cross Abstract: The key-value (KV) cache is the main memory bottleneck in long-context large language model (LLM) inference.

By Shahrzad Esmat, Dhawal Shah, Ali Jannesari

llms efficiency benchmarks

Related stories

Kernelized Linear Attention: Breaking the Capacity Wall with Symmetric Cones

TabSwift: An Efficient Tabular Foundation Model with Row-Wise Attention

WIDE: Boosting Adaptive LLM Inference via Token-level Dynamic Width Pruning

MiniMax Sparse Attention

Localized TabICLv2: Scaling Tabular In-Context Learning through k-NN

VarRate: Training-Free Variable-Rate KV Cache Compression for Long-Context LLMs