arXiv AI
The Illusion of Replacement: Rethinking Specialized Machine Learning Models in the Foundation Model Era
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Jun 11
GILT: An LLM-Free, Tuning-Free Graph Foundational Model for In-Context Learning
arXiv:2510. 04567v3 Announce Type: replace-cross Abstract: Graph Neural Networks (GNNs) are powerful tools for processing relational data but often struggle to generalize to unseen graphs, giving rise to the development of Graph Foundational Models (GFMs).
arXiv AI
Oct 16
Language Models Struggle to Use Representations Learned In-Context
The article examines whether large language models (LLMs) can effectively use representations they learn in-context to perform downstream tasks. Experiments with open‑weight LLMs on next‑token prediction and a new adaptive world modeling task reveal that these models struggle to deploy novel semantics defined in-context, even when such semantics are encoded in their latent representations. The study also shows that state‑of‑the‑art closed‑source reasoning models similarly fail to reliably leverage in‑context patterns, highlighting a gap in flexible representation usage.
arXiv AI
Oct 15
A Scaled-Up Empirical Study of Syntactic Language Models
arXiv:2610.16837v1 Announce Type: cross Abstract: Syntactic language models (SLMs) that generate sentences along with their syntactic tree structures have shown promise in both syntactic generalizati...
arXiv AI
Sep 15
Mimir: Large-scale Multilingual Concept Modeling
arXiv:2605.25263v2 Announce Type: replace-cross Abstract: Current language modeling approaches are built around tokens. Text corpora are split into tokens, and models are trained by performing comput...
arXiv Computation and Language
Oct 22
StructLens: A Structural Lens for Language Models via Maximum Spanning Trees
StructLens is a framework that analyzes language model representations by constructing maximum spanning trees from semantic representations in residual streams, inspired by dependency parsing. It reveals that contiguous tokens are close in representation space, with middle layers showing the strongest local-span organization. The study also shows that smaller local units emerge earlier during pre‑training, while larger units appear later, offering insights into how token representations are organized across layers and training stages.
arXiv AI
Aug 21
When Machines Speak: A Unified Generative Framework for Integrating Machine-Native Symbols into Pretrained Large Language Models
arXiv:2608. 19529v1 Announce Type: cross Abstract: Many real-world AI systems represent entities, behaviors, and structured information using discrete machine-native symbols rather than natural language.