Chain of Operators (CHOP) is a framework that enables frozen scientific foundation models to tackle out‑of‑distribution (OOD) tasks without any parameter updates. By leveraging In‑Context Operator Networks (ICON), CHOP breaks down unfamiliar problems into sequences of explicit mathematical operations and model calls, effectively translating OOD queries into the model’s learned regime. Experiments on PDE benchmarks and air‑quality forecasting show that CHOP consistently reduces inference errors while maintaining full interpretability and cross‑equation generalization.
By Minghui Yang, Chenghan Wu, Ling Guo, Liu Yang
The paper proposes that large language models (LLMs) encode high‑level concepts as linear directions within their activation space and that they can use subspaces and vector algebra to perform tasks. By analyzing functional modules and residual streams during in‑context learning (ICL), the authors find that LLMs can create evidence‑accumulating subspaces and solve ICL tasks through simple algebraic operations within those subspaces.
By Jung H. Lee, Sujith Vijayan
arXiv:2609.22143v1 Announce Type: cross
Abstract: Machine learning learns functions: prompt to response, image to caption. What these functions are mathematically remains hard to say. We present a me...
By Afjal Chowdhury, James Chen, Alan Edelman
arXiv:2610.01054v1 Announce Type: cross
Abstract: In-context learning (ICL) enables language models to perform new tasks from demonstrations without weight updates. However, every ICL inference requi...
By Guangzhi Xiong, Zhenghao He, Bohan Liu, Sanchit Sinha, Wenqian Ye, Aidong Zhang
The paper investigates how large language models learn new tasks in-context, comparing rule-based instruction following to example-based few-shot prompting across five diverse tasks. Results show that models generally learn more reliably from rule descriptions than from examples alone, and adding more examples does not consistently improve performance. Instruction tuning further enhances rule-based learning while preserving example-based capabilities, with rule advantages being strongest for algebraic tasks and weaker for tasks requiring distributional sensitivity or parametric knowledge.
By Xiang Fu, Seungmin Cho, Yukyung Lee, Najoung Kim
arXiv:2505. 11766v4 Announce Type: replace Abstract: Neural Operators (NOs) are powerful architectures for learning mappings between function spaces.
By Haoze Song, Zhihao Li, Xiaobo Zhang, Zecheng Gan, Zhilu Lai, Wei Wang
arXiv:2606. 31903v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) increasingly process long visual-token sequences, increasing the overall inference computation.
By Zhaoyang Luo, Runmin Dong, Miao Yang, Fan Wei, Yushan Lai, Bin Luo, Haohuan Fu
arXiv:2605. 28854v2 Announce Type: replace-cross Abstract: Large language models (LLMs) exhibit remarkable flexibility in adapting to novel tasks from in-context examples without parameter updates, a capability known as in-context learning (ICL).
By Hua-Dong Xiong, Li Ji-An, Robert C. Wilson, Kwonjoon Lee, Xue-Xin Wei
Neural operators learn mappings between function spaces, but are typically developed with dense input-output training fields and fully observed inputs at inference. Many scientific problems require instead predicting solution fields from sparse, irregular, or partial observations under uncertainty.
arXiv:2510. 02528v2 Announce Type: replace Abstract: Large Multimodal Models (LMMs) demonstrate impressive in-context learning abilities from few multimodal demonstrations, yet the internal mechanisms supporting such task learning remain opaque.
By Shuhao Fu, Esther Goldberg, Ying Nian Wu, Hongjing Lu
arXiv:2503. 05598v2 Announce Type: replace-cross Abstract: This review examines neural operator architectures for learning solution operators of parametric partial differential equations (PDEs), with an emphasis on conceptual clarity and practical implementation.
By Prashant K. Jha
The paper introduces the Feature Interaction Modeling Operator (FM-Operator), a point‑wise query neural operator that explicitly designs feature construction and multiplicative interactions between sensor observations and query coordinates. By reinterpreting the DeepONet aggregation as a diagonally constrained multiplicative interaction, FM-Operator restructures both feature construction and interaction to enable richer information exchange while maintaining point‑wise evaluation. Experiments on several PDE benchmarks show that FM-Operator consistently outperforms vanilla DeepONet and improves upon the Shift‑DeepONet baseline, indicating that tailored representation construction and interaction can enhance query‑based neural operator performance.
By Quan Gu, Xiaoduo Li, Hongxia Liu