The Flow has not summarised this story yet — read it at arXiv Machine Learning.
The paper argues that many AI tasks can be split into a deterministic core and a fuzzy residue, and that decomposing tasks rather than expanding models yields a property called parity by construction—where model‑size no longer affects performance on the deterministic sub‑tasks. Across nine models ranging from 494 M to Opus‑class size, the authors show that once the deterministic context is resolved, operation selection and selector resolution scores become identical, while a non‑decomposable task (labelling) remains sensitive to model size. They present a general method for identifying decomposition points, constructing deterministic surfaces and grammars, and ensuring safe residues, and demonstrate the approach in physics, motion retargeting, object‑model editing, and a Web‑CLI tool. whyItMatters":"The study demonstrates that task decomposition can eliminate the need for model scaling or fine‑tuning, offering a systematic design discipline applicable across diverse AI domains."
A recurring design choice in building AI systems is whether, when a task is too hard for a model, to expand the model (make it larger, better-prompted, fine-tuned), or to decompose the task (extract t...
The paper investigates when in‑context learning (ICL) can succeed even if the instruction or target task is not covered by a model’s pretraining data. It introduces a theoretical framework that distinguishes two types of distribution shift—unsupported instructions and unsupported target tasks—and shows how demonstrations can mitigate these shifts. The authors further extend the analysis to long‑horizon tasks that can be decomposed into pretraining‑supported subtasks, providing error bounds that explain how ICL can handle tasks beyond the model’s original support.
The paper investigates how Tabular Foundation Models (TFMs) can achieve strong transfer learning by self‑supervised pre‑training on a single real table. It finds that a table’s usefulness is largely determined by its number of features rather than instances, and that fine‑grained column‑level preprocessing improves downstream performance while dataset‑level filtering does not. The authors propose that tabular in‑context generalization is primarily retrieval‑based, with models learning to identify and aggregate relevant examples from the provided context.
arXiv:2506. 20699v2 Announce Type: replace Abstract: Learning in non-stationary and multi-context environments requires more than ordinary within-task generalization.
arXiv:2606. 18627v1 Announce Type: new Abstract: Model merging has emerged as a training-free alternative to multi-task learning, aiming to combine multiple task-specific fine-tuned models into a single multi-task model.