TabClustPFN is a Prior-data Fitted Network designed for clustering tabular data. It jointly addresses the lack of supervision, unknown cluster cardinality, and label switching by integrating a hybrid pretraining prior, a decoupled partition‑inference and cardinality‑inference architecture, and a permutation‑invariant SoftARI training objective. On a curated benchmark of 44 real‑world tabular datasets, it outperforms classical, deep, and amortized baselines while maintaining efficient runtime.
arXiv:2509. 25289v4 Announce Type: replace-cross Abstract: Identifying an effective clustering algorithm for a given dataset remains a fundamental unsupervised learning issue.
arXiv:2606. 11473v1 Announce Type: cross Abstract: Prior-fitted networks (PFNs) are a promising class of tabular foundation models that perform in-context learning, whereby the entire labelled training set is supplied as context, and predictions for test queries are produced in a single forward pass.
arXiv:2608. 00346v1 Announce Type: new Abstract: Data imbalance poses a major challenge in supervised classification, where the majority-class bias contributes to false negatives and overestimates classification accuracy.
TabMF is a framework that repurposes pretrained tabular foundation models (TFMs) as encoders for supervised datasets, adding a dataset-level readout and fine‑tuning via contrastive post‑training on synthetic views. The authors instantiate TabMF on TabICLv2 and TabPFN‑2.5 and evaluate the resulting representations on six tasks involving unseen real datasets. The approach achieves the best aggregate performance among both learned and classical dataset representations, excelling in dataset retrieval, compact benchmark construction, and distribution‑shift severity assessment.
arXiv:2609.36648v1 Announce Type: new Abstract: Vision-language pre-training has reshaped image clustering, giving rise to language-assisted image clustering (LaIC), which leverages textual semantics...
ClusterAttention is a training‑free technique that speeds up bidirectional attention by recursively clustering keys and queries into fixed‑size, power‑of‑two blocks, enabling block‑sparse attention to match dense attention latency on GPUs. The method derives error bounds for sparse attention, showing tighter clusters can reduce error when compensated via centroids, and demonstrates significant speedups—up to six‑fold on large tabular data and 1.8× on video generation—while preserving over 99% of dense accuracy.
The paper explores how prior-data fitted networks (PFNs) can be adapted for anomaly detection in tabular data. Using frozen TabPFN features, the authors show that scoring samples by nearest-neighbor distance in feature space yields strong results, and they identify optimal layers and extraction procedures. They further improve performance by fine‑tuning the model on a reference set, achieving higher AUROC on the ADBench benchmark with both a fine‑tuning‑free (ZEN) and a fine‑tuned (FOCUS) approach.
The paper introduces OFAG, a foundation model designed for attributed graph clustering that can be trained once and applied to diverse graphs without graph‑specific tuning. It learns a reusable clustering strategy from synthetic graphs and uses a dimension‑agnostic encoder to handle varying feature spaces, producing clustering‑friendly node representations in a single forward pass. Across ten datasets, a frozen OFAG model outperforms baselines in both speed and clustering quality, and the authors provide code and pretrained checkpoints for easy adoption.
TabICLv2 is a new state‑of‑the‑art tabular foundation model that outperforms existing methods on regression and classification tasks. It relies on a synthetic data generation engine for diverse pretraining, architectural innovations such as a scalable softmax attention, and optimized training protocols that replace AdamW with the Muon optimizer. On the TabArena and TALENT benchmarks, TabICLv2 surpasses the current best model, RealTabPFN‑2.5, without any tuning, while also being faster and capable of handling million‑scale datasets with limited GPU memory.
The paper demonstrates that a tabular foundation model can achieve strong generalization using only a single real table for self‑supervised pre‑training, challenging the belief that large synthetic or real datasets are necessary. By systematically pre‑training and evaluating across diverse benchmarks, the authors show that the number and quality of tasks that can be derived from a dataset are critical for downstream performance. This finding suggests that carefully constructed task sets from limited data can enable effective transfer learning in tabular models.
OrionMoE-v1.0 is a compact tabular foundation model that uses a dual‑axis Transformer and a sparse mixture‑of‑experts architecture to perform in‑context learning on labeled support tables. It selectively activates experts along the feature axis, guided by both individual feature representations and overall dataset context, enabling adaptive computation across diverse tabular problems. Pretrained on synthetic classification tasks, it achieves competitive performance on multiple benchmarks, especially on small‑context, high‑dimensional, and class‑imbalanced datasets.
The paper introduces FedDCN, a federated deep clustering network that jointly optimizes reconstruction and clustering losses for high‑dimensional, heterogeneous data. It addresses challenges of non‑IID client data by generating synthetic augmentations and applying geometric regularization to align latent spaces. Experiments show the method’s effectiveness under both IID and non‑IID settings, and the authors outline future research directions.