arXiv Machine Learning June 18, 2026 By Sultan Daniels, Dylan Davis, Dhruv Gautam, Wentinn Liao, Gireeja Ranade, Anant Sahai

Decomposing Prediction Mechanisms for In-Context Recall

Read the original on arXiv Machine Learning →

arXiv:2507. 01414v2 Announce Type: replace Abstract: We introduce a new family of toy problems that combine features of linear-regression-style continuous in-context learning (ICL) with discrete associative recall.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI

4d ago

Beyond Semantics: How Temporal Biases Shape Retrieval in Transformer and State-Space Models

arXiv:2510.22752v2 Announce Type: replace-cross Abstract: In-context learning is governed by both temporal and semantic relationships, shaping how Large Language Models (LLMs) retrieve contextual inf...

By Anooshka Bajaj, Deven Mahesh Mistry, Sahaj Singh Maini, Yash Aggarwal, Zoran Tiganj

llms benchmarks safety

arXiv AI

Jul 28

Extracting Algorithms in Pre-trained LLMs: A Case on Hidden Markov Models

arXiv:2607. 22646v1 Announce Type: new Abstract: Large language models (LLMs) display a striking ability to predict next observations from Hidden Markov Models (HMMs) via in-context learning (ICL), but the algorithm underlying this capability remains undetermined: prior work has proposed several candidates without consensus, and none has been grounded in the model's internal activations.

By Yijia Dai, Zhaolin Gao, Yahya Sattar, Jennifer J. Sun, Sarah Dean

llms

arXiv Machine Learning

Aug 11

Large language models reorganize representational geometry during in-context learning

arXiv:2605. 28854v2 Announce Type: replace-cross Abstract: Large language models (LLMs) exhibit remarkable flexibility in adapting to novel tasks from in-context examples without parameter updates, a capability known as in-context learning (ICL).

By Hua-Dong Xiong, Li Ji-An, Robert C. Wilson, Kwonjoon Lee, Xue-Xin Wei

llms

arXiv AI

Jun 8

Data-Efficient Autoregressive-to-Diffusion Language Models via On-Policy Distillation

arXiv:2606. 06712v1 Announce Type: cross Abstract: We study the transformation of autoregressive models (ARLMs) into diffusion language models (DLMs).

By Xingyu Su, Jacob Helwig, Shubham Parashar, Atharv Chagi, Lakshmi Jotsna, Degui Zhi, James Caverlee, Dileep Kalathil, Shuiwang Ji

llms diffusion efficiency

arXiv Machine Learning

Jun 16

Next-Latent Prediction Transformers Learn Compact World Models

arXiv:2511. 05963v4 Announce Type: replace Abstract: Transformers replace recurrence with a memory that grows with sequence length and self-attention that enables ad-hoc lookups over past tokens.

By Jayden Teoh, Manan Tomar, Kwangjun Ahn, Edward S. Hu, Tim Pearce, Pratyusha Sharma, Akshay Krishnamurthy, Riashat Islam, Alex Lamb, John Langford

llms diffusion benchmarks safety

arXiv Machine Learning

Jun 9

PriFT: Prior-Support Guided Supervised Fine-Tuning

arXiv:2606. 09396v1 Announce Type: cross Abstract: Supervised fine-tuning (SFT) is an efficient approach for downstream task adaptation and often serves as the initialization stage for reinforcement learning (RL), but it can show weaker generalization than RL.

By Ke Wang, Shuangqi Li, Mathieu Salzmann, Pascal Frossard

nlp reinforcement-learning fine-tuning benchmarks

One email a morning, machine-written

Related stories

Beyond Semantics: How Temporal Biases Shape Retrieval in Transformer and State-Space Models

Extracting Algorithms in Pre-trained LLMs: A Case on Hidden Markov Models

Large language models reorganize representational geometry during in-context learning

Data-Efficient Autoregressive-to-Diffusion Language Models via On-Policy Distillation

Next-Latent Prediction Transformers Learn Compact World Models

PriFT: Prior-Support Guided Supervised Fine-Tuning