arXiv Machine Learning
Large Language Models Develop Belief State Geometry In-Context
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
Hugging Face Trending Papers
2d ago
Large Language Models Develop Belief State Geometry In-Context
Large language models (LLMs) trained on next-token prediction exhibit remarkable in-context learning (ICL) abilities, yet the representations that support ICL remain poorly understood. We consider suc...
arXiv AI
Jul 28
Extracting Algorithms in Pre-trained LLMs: A Case on Hidden Markov Models
arXiv:2607. 22646v1 Announce Type: new Abstract: Large language models (LLMs) display a striking ability to predict next observations from Hidden Markov Models (HMMs) via in-context learning (ICL), but the algorithm underlying this capability remains undetermined: prior work has proposed several candidates without consensus, and none has been grounded in the model's internal activations.
arXiv AI
Sep 1
DIP: Dynamic In-Context Planner For Diffusion Language Models
arXiv:2601.03199v2 Announce Type: replace-cross Abstract: Diffusion language models (DLMs) have shown strong potential for general natural language tasks with in-context examples. Existing In-Context...
arXiv Machine Learning
Jun 25
How Does the Pretraining Distribution Shape In-Context Learning? A Fundamental Trade-Off
arXiv:2510. 01163v2 Announce Type: replace Abstract: The factors driving the performance of in-context learning (ICL) in large language models (LLMs) remain poorly understood despite ICL's surprising effectiveness, enabling models to adapt to new tasks from only a handful of examples.
arXiv Machine Learning
1d ago
Long-Context Demonstration Selection Using State Space Models
arXiv:2609.17888v1 Announce Type: new Abstract: We study the problem of demonstration selection, which involves selecting a subset of examples for prepending to a query to a language model. This prob...
arXiv Machine Learning
Aug 11
Large language models reorganize representational geometry during in-context learning
arXiv:2605. 28854v2 Announce Type: replace-cross Abstract: Large language models (LLMs) exhibit remarkable flexibility in adapting to novel tasks from in-context examples without parameter updates, a capability known as in-context learning (ICL).