← Back to all news
arXiv Machine Learning By Ali Eslamian, Qiang Cheng

TabNSM: Neural Sparse Mixer for Tabular Regression

Read the original on arXiv Machine Learning →

TabNSM is a scalable regression framework for large-scale, high-dimensional tabular data that builds on sparse-attention and mixer architectures. Its core component, the Adaptive Sparse Interaction Module (ASIM), combines foreground feature discovery, sparse local interaction encoding, and Feature-Token Mixing to achieve near-linear complexity. For regression, TabNSM adds a Multi-Stage Regression Head, GridLoss (an ordinal-aware soft-binning objective), and RISE (a difficulty-aware sampling strategy), achieving strong predictive performance and practical scalability across nine real-world benchmarks, especially on high-dimensional and heterogeneous datasets.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv AI
4d ago

LoopICL: Looping a single transformer block to solve tabular tasks

LoopICL is a transformer architecture that loops a single block to address tabular tasks. It separates parameter count from computational depth by using a cell stream for per‑cell features and a row stream for in‑context examples, refined via within‑column and cross‑column attention. During pre‑training, varying loop counts and a learned exit‑gate allow the model to adjust inference depth at test time, achieving competitive performance with TabICLv2 while using about 90% fewer parameters.

By Amir Rezaei Balef, Katharina Eggensperger
llms
More like this →
arXiv Machine Learning
Sep 2

Solving In-Table Prediction Problems by Deep Neural Networks with Performance Evaluation Using Synthetic Data

The paper introduces In-Table Prediction (ITB), a self‑supervised task where deep neural networks learn to predict any column in a table from the remaining columns. It proposes a novel neural layer to handle missing continuous values, generates synthetic datasets with controlled column relationships, and evaluates three architectures—MLP, ResNet, and Transformer—showing that attention‑based Transformers perform best when ample training data and large embeddings are used. The study is limited to synthetic, small‑column tables and is presented as an initial investigation rather than a comprehensive real‑world analysis.

By Xiao Zhao, Daniela Oelke
llmsrag
More like this →