arXiv AI

A Graph Signal Processing Perspective on Numerical Sequence Representations in LLM In-Context Learning

arXiv:2608. 03015v1 Announce Type: cross Abstract: Pretrained large language models (LLMs) have demonstrated in-context learning (ICL) capabilities for numerical inference over sequences serialized as text.

arXiv AI
Jul 28

Extracting Algorithms in Pre-trained LLMs: A Case on Hidden Markov Models

arXiv:2607. 22646v1 Announce Type: new Abstract: Large language models (LLMs) display a striking ability to predict next observations from Hidden Markov Models (HMMs) via in-context learning (ICL), but the algorithm underlying this capability remains undetermined: prior work has proposed several candidates without consensus, and none has been grounded in the model's internal activations.

By Yijia Dai, Zhaolin Gao, Yahya Sattar, Jennifer J. Sun, Sarah Dean