Hugging Face Trending Papers
Every Part Has Its Role: Part-of-Speech Effects in LLM Post-Training
Read the original on Hugging Face Trending Papers →The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
Related stories
arXiv Computation and Language
2d ago
Every Part Has Its Role: Part-of-Speech Effects in LLM Post-Training
arXiv:2610.29843v1 Announce Type: new Abstract: Large language models are typically trained with next-token prediction (NTP), which rewards predicting specific tokens rather than explicitly accountin...
Hugging Face Trending Papers
2d ago
Verbalizing Activation Differences with Natural Language Autoencoders
Activation verbalizers translate the internal representations of large language models into human-readable text. The recently proposed natural language autoencoders (NLAs) learn this by reconstructing...
arXiv Computation and Language
1d ago
Verbalizing Activation Differences with Natural Language Autoencoders
The paper investigates natural language autoencoders (NLAs) as tools for verbalizing activation differences in large language models. It examines whether NLAs capture compositional structure, generalize to fine‑tuned models, recover fine‑tuning objectives, and can be specialized for model diffing. Experiments across four models show that NLAs largely succeed in these tasks, though they do not yet match state‑of‑the‑art diffing methods on emergent misalignment detection.
arXiv Computation and Language
Sep 17
Modelling Adjectival Modification Effects on Semantic Plausibility
The paper investigates how adjectival modifiers affect the semantic plausibility of events, using the Adept benchmark of 16,000 English sentence pairs that differ by a single adjective. Experiments show that sentence transformers, despite being conceptually suited to the task, underperform compared to models like RoBERTa. The authors provide an error analysis and discuss the implications of their findings for future work on balancing training and test data.
arXiv Machine Learning
Jun 2
A Pre-Training Analogue of Grokking in Language Models: Tracing Delayed Grammatical Generalization
arXiv:2606. 00230v1 Announce Type: new Abstract: Grokking, the phenomenon in which neural networks generalize long after fitting their training data, has been studied in supervised settings on many epochs.
arXiv AI
Sep 28
Achieving Tokenizer Flexibility in Language Models through Heuristic Adaptation and Supertoken Learning
The paper introduces TokenAdapt, a model‑agnostic tokenizer transplantation method that uses a hybrid heuristic to initialize new token embeddings, and a novel pre‑tokenization learning approach for multi‑word Supertokens to improve compression. TokenAdapt combines local subword decomposition and global semantic similarity to preserve semantics while reducing retraining needs. Empirical results show that TokenAdapt outperforms existing baselines such as Transtokenizer and ReTok, achieving lower perplexity ratios and significant compression gains.