The paper investigates whether Prior‑Data Fitted Networks (PFNs) encode spectral information about their input context. Across seven PFNs—including pretrained tabular foundation models and a decision‑tree prior model—a linear probe on the residual stream recovers the context’s frequency spectrum with high accuracy (R² ≥ 0.95), driven by a single principal direction. Activation and subspace patching show that the network exploits this spectral structure in a low‑dimensional subspace, and a Filter Bank Decoder can transform frozen PFN representations into an explicit stationary kernel that enables competitive Gaussian process regression at far lower cost.
arXiv:2610.02798v1 Announce Type: new Abstract: The Muon optimizer applies spectral orthogonalization to matrix-valued updates and has shown strong performance in large-scale neural network training,...
arXiv:2606. 05957v1 Announce Type: new Abstract: Singular learning theory and information geometry have studied the same parameter spaces in mostly separate vocabularies: the former computes Bayesian invariants in resolved coordinates, the latter works in original coordinates under a non-degeneracy assumption that overparameterised models routinely violate.
arXiv:2609.36883v1 Announce Type: new Abstract: Tabular foundation models (TFMs) are increasingly popular because they deliver strong predictions on new datasets through in-context learning, without...
arXiv:2606. 10324v1 Announce Type: new Abstract: The analogy between deep neural network forward passes and renormalization group (RG) flows has been repeatedly noted in the literature, but existing treatments remain qualitative: depth is described as a coarse-graining scale, attention is likened to a partition function, and representations are said to flow toward fixed points.
arXiv:2610.07804v1 Announce Type: new Abstract: Prior Fitted Networks (PFNs) such as TabPFN now rival established statistical procedures across prediction and estimation tasks. A natural explanation...
arXiv:2605.12904v2 Announce Type: replace Abstract: Tabular foundation models (TFMs) have emerged as a powerful paradigm for in-context learning on structured data, enabling direct prediction on new...
arXiv:2607. 07500v1 Announce Type: cross Abstract: Time series classification (TSC) is dominated by a two-stage paradigm: train a feature encoder -- either from scratch on the target dataset or via pretraining on large corpora -- and then fit a task-specific classifier on top.
The paper introduces three algorithms—NTK-KIP, MetaQuill, and MetaQuill-KIP—to enhance neural fields (NFs) for reconstructing signals from sparse data. NTK-KIP distills a support set of coordinates to enable non‑linear inpainting with a finite NTK, while MetaQuill meta‑learns a shared initialization that allows quick adaptation to new scenes via a small weight offset. MetaQuill-KIP combines both approaches, achieving high‑quality, semantically plausible reconstructions with lightweight per‑instance adaptation, outperforming diffusion‑style baselines that rely on large pretrained generative models.
The analogy between deep neural network forward passes and renormalization group (RG) flows has been repeatedly noted in the literature, but existing treatments remain qualitative: depth is described as a coarse-graining scale, attention is likened to a partition function, and representations are said to flow toward fixed points. No existing work has defined a measurable RG order parameter, tested it under controlled variation of the input distribution, or made quantitative predictions that are empirically verified.
arXiv:2609.38049v1 Announce Type: new Abstract: Generative models for function-valued data, such as time series and solutions of partial differential equations, must learn distributions over infinite...
AdaKerNet is a task‑adaptive neural kernel decoder that operates on frozen multimodal representations from large foundation models, without requiring access to the models’ parameters. It learns Lipschitz‑controlled multimodal features, a reference kernel providing a soft structural prior, and a lightweight nonlinear predictor that deforms this structure. Experiments on four multimodal large language models and diverse input modalities show consistent improvements over baseline decoders, achieving up to 41% error reduction in scarce‑label settings.
Prior Fitted Networks (PFNs) such as TabPFN now rival established statistical procedures across prediction and estimation tasks. A natural explanation is that PFNs have the property of statistical ada...