The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
The paper presents a learning theory for sequence models that explains when they will rely on in-context learning versus specialization. Using a solvable topic model where documents are drawn from a finite mixture of latent topics, the authors show that an in-context predictor emerges early, while a specialized predictor that stores the topic mixture only appears after the training data exceeds the model’s parameter count. Experiments with transformers confirm the predicted sample‑complexity thresholds and crossover point without any fitted parameters.
The paper introduces MARETopic, a training‑free framework that identifies topics by selecting rank‑based prototype documents from pretrained embeddings. By projecting embeddings onto a low‑dimensional manifold and building ranked neighborhood lists, a greedy algorithm picks exactly K exemplar texts whose neighborhoods cover the corpus. Two variants—MARETopic_Corr, which uses a query‑performance predictor and rank correlation, and MARETopic_Diff, which employs a rank‑based diffusion matrix—achieve higher purity and NMI on benchmark datasets and run significantly faster, while also improving topic coherence and vocabulary diversity through a novel Maximal Marginal Relevance step.
SentenceETM is a neural topic model that assigns a topic to each sentence by jointly using lexical and contextual evidence. It represents each topic with a vocabulary distribution via word embeddings and a Gaussian over sentence embeddings in a topic‑specific low‑dimensional subspace. The model exactly marginalizes the sentence topic, combining word and embedding evidence, and uses amortized variational inference for document proportions while re‑estimating Gaussian parameters in closed form.
The paper introduces SeLATM, a framework that improves topic modeling by generating topics at the segment level and refining them through agentic feedback loops. This approach addresses limitations of LLM-based topic assignment methods, such as the inability to produce topic distributions, overly broad or narrow topics, and high resource consumption. Experiments on multiple datasets show that SeLATM reduces LLM resource usage while maintaining superior performance.
arXiv:2602. 17907v2 Announce Type: replace-cross Abstract: Traditional neural topic models are typically optimized by reconstructing the document's Bag-of-Words (BoW) representations, overlooking contextual information and struggling with data sparsity.
The paper investigates whether few-shot in-context learning (ICL) emerges similarly across different data modalities. Using a controlled cross-modality framework, the authors test the Convergent Emergence Hypothesis, which posits that tasks benefiting from ICL in one modality will also benefit in others. They find that paired-mapping ICL appears in six modalities—language, genome, integer sequences, time series, images, and proteins—outperforming baselines and showing correlated task effects in five of them, supporting the hypothesis in some but not all cases.