arXiv Machine Learning
Jul 1
Probing Memorization of Tabular In-Context Learning
arXiv:2606. 31208v1 Announce Type: new Abstract: Large tabular models (LTMs), i.
Categorical Prior Lock-in: Why In-Context Learning Fails for Structured Data
arXiv:2606. 11961v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used as conditional generators for structured data, relying on in-context learning (ICL) to adapt to new distributions without parameter updates.
Related stories
arXiv AI
Jul 23
Lifted Representation Hypothesis in Language Models
arXiv:2607. 19360v1 Announce Type: new Abstract: Large language models (LLMs) often answer queries by mapping individual observations to more general rule-like structures.
arXiv Machine Learning
Jun 25
How Does the Pretraining Distribution Shape In-Context Learning? A Fundamental Trade-Off
arXiv:2510. 01163v2 Announce Type: replace Abstract: The factors driving the performance of in-context learning (ICL) in large language models (LLMs) remain poorly understood despite ICL's surprising effectiveness, enabling models to adapt to new tasks from only a handful of examples.
arXiv Machine Learning
Jun 5
GenFT: A Generative Parameter-Efficient Fine-Tuning Method for Pretrained Foundation Models
arXiv:2506. 11042v2 Announce Type: replace Abstract: Parameter-efficient fine-tuning (PEFT) has emerged as a resource-efficient strategy for adapting Pretrained Foundation Models (PFMs) by learning a small number of task-specific updates $\Delta W$.
arXiv AI
Jun 9
Emergence of Context Characteristics Sensitivity in Large Language Models
arXiv:2606. 09525v1 Announce Type: cross Abstract: During instruction fine-tuning (IFT), large language models (LLMs) learn to follow instructions by using the provided context to answer a query.
arXiv Machine Learning
Aug 11
Large language models reorganize representational geometry during in-context learning
arXiv:2605. 28854v2 Announce Type: replace-cross Abstract: Large language models (LLMs) exhibit remarkable flexibility in adapting to novel tasks from in-context examples without parameter updates, a capability known as in-context learning (ICL).
arXiv AI
Jul 14
Extending LLM Context via Associative Recurrent Memory
arXiv:2607. 11614v1 Announce Type: cross Abstract: Extending the context length of large language models (LLMs) is critical for many real-world applications, yet standard transformers remain constrained by quadratic compute and linear memory scaling.
arXiv Machine Learning
Jun 2
Understanding LoRA as Knowledge Memory: An Empirical Analysis
arXiv:2603. 01097v3 Announce Type: replace Abstract: Continuous knowledge updating for pre-trained large language models (LLMs) is increasingly necessary yet remains challenging.
arXiv AI
Jun 30
The Hidden Cost of Structured Generation in LLMs: Draft-Conditioned Constrained Decoding
arXiv:2603. 03305v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are increasingly used to generate executable outputs, JSON objects, and API calls, where a single syntax error can make the output unusable.
arXiv Machine Learning
Jul 7
LP-SFT: Local-Preserving Supervised Fine-Tuning via Multimodal Entropy Structure
arXiv:2607. 04733v1 Announce Type: cross Abstract: Supervised fine-tuning (SFT) is the standard approach for adapting pretrained language models to downstream domains, yet it often improves target-domain behavior at the cost of degrading pre-existing capabilities.
arXiv Machine Learning
Jul 15
RAFP: Identifying LLM Lineages via Rare-Region Fingerprints
arXiv:2505. 12682v2 Announce Type: replace Abstract: Large language models (LLMs) are increasingly released under restricted licenses, creating a growing need for robust model ownership verification.
arXiv AI
Jun 12
Meta-Learning Transformers to Improve In-Context Generalization
arXiv:2507. 05019v2 Announce Type: replace-cross Abstract: In-context learning enables transformer models to generalize to new tasks based solely on input prompts, without any need for weight updates.