SetFit: Efficient Few-Shot Learning Without Prompts

Few-shot learning in practice: GPT-Neo and the 🤗 Accelerated Inference API

Language models are few-shot learners

Are We Really Doing Few-Shot Learning? A Critical Examination of Pre-Training Assumptions

The paper critically evaluates common few‑shot learning protocols that rely on pre‑training a model on a large auxiliary set with classes disjoint from the target but drawn from the same visual domain. By comparing no pre‑training, class‑disjoint in‑domain pre‑training, supervised out‑of‑domain pre‑training, and label‑free out‑of‑domain pre‑training across eight datasets and three architectures, the authors find that in‑domain pre‑training yields a 33.41‑point average improvement, while out‑of‑domain pre‑training offers a 23.75‑point gain, revealing a 9.66‑point optimistic bias due to domain overlap. They also demonstrate that a label‑free augmentation strategy can match supervised out‑of‑domain performance and propose a descriptor‑based source‑selection method that closely approximates oracle selection, underscoring the need to move beyond in‑domain pre‑training as the default evaluation protocol.

Training-free LLM Verification via Recycling Few-shot Examples

arXiv:2506.17251v3 Announce Type: replace-cross Abstract: Although large language models (LLMs) have achieved remarkable performance, the inherent stochasticity of their reasoning processes and varyi...

Re-Evaluating Continual Learning with Few-Shot Adaptation

arXiv:2606. 03843v1 Announce Type: cross Abstract: Continual learning methods aim to maximize the stability and plasticity of machine learning models that are trained on a sequence of tasks.

One-shot imitation learning

Guided Prompt Evolution for Vision-Language Models Adaptation

The paper introduces EvoPrompt, a framework for adapting vision‑language models to new tasks with limited data while preventing catastrophic forgetting. EvoPrompt uses a Modality‑Shared Prompt Projector to create hierarchical prompts and an evolutionary training strategy that separates low‑rank updates into directional and magnitude components, preserving learned semantic directions. Experiments show that EvoPrompt achieves state‑of‑the‑art few‑shot performance while maintaining the original zero‑shot capabilities of the pre‑trained models.

From Prompts to Trees: Effective LLM-Guided Tree Generation for Few-Shot Tabular Classification

While Large Language Models (LLMs) possess rich world knowledge and impressive generalization capabilities, their direct application to tabular data classification is hindered by high inference costs...

ARASH: Adaptive Retrieval And Shot Selection for Tabular Prediction

ARASH is a method that improves the efficiency of Tabular Foundation Models by selecting optimal few-shot prompts based on local neighborhood analysis within the training set. It reduces the prompt length and memory usage of TabPFN by 1261.5× and 2.56×, respectively, while maintaining comparable accuracy. This approach addresses the challenge of identifying relevant rows for in-context learning in tabular data.

Few-Medoids: An Embarrassingly Simple Coreset Selection Method for Few-Shot Knowledge Distillation

arXiv:2607. 05891v1 Announce Type: cross Abstract: Coreset selection aims to identify a small and highly representative subset of a massive dataset for efficient model training.

Fine-Tuning Without Forgetting In-Context Learning: A Theoretical Analysis of Linear Attention Models

arXiv:2602. 23197v2 Announce Type: replace-cross Abstract: Transformer-based large language models exhibit in-context learning, enabling adaptation to downstream tasks via few-shot prompting with demonstrations.

How Much Prompt Is Enough? A Blackbox Minimization of Few-Shots in LLMs

The paper introduces ramework, a blackbox prompt‑minimization framework that identifies the minimal subset of few‑shot prompts necessary for large language models (LLMs). In a case study, the framework reduces few‑shot exemplars by an average of 65.3% in character count while maintaining full propositional output fidelity, revealing that models tend to keep logical identifiers and constraint declarations while discarding natural language prose. The analysis further distinguishes between universal encoder and decoder models, offering insights into prompt compression and structural analysis.