arXiv Machine Learning
Logits are All We Need to Adapt Closed Models
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv AI
Jul 28
Frustratingly Simple Black-Box Adaptation of Language Models via Logit Bias
arXiv:2607. 22837v1 Announce Type: cross Abstract: Many organizations aim to adapt language models for internal use, both to improve performance on domain-specific tasks and to address privacy concerns around sensitive data.
arXiv Machine Learning
Jul 21
Token-Level Off-Policy Learning for Faithful Generation Under Distribution Shift
arXiv:2607. 17524v1 Announce Type: cross Abstract: We propose Token-Level Off-Policy Labeling (TOPL), an off-policy training paradigm that reframes post-training as a token-level correctness prediction task.
arXiv Computation and Language
Sep 22
Want Better Synthetic Data? Steer It: Activation Steering for Low-Resource Language Generation
arXiv:2606.18389v2 Announce Type: replace Abstract: Large language models (LLMs) have become an effective tool for synthetic data generation, including for low-resource languages, where generated dat...
Hugging Face Trending Papers
Jul 20
Token-Level Off-Policy Learning for Faithful Generation Under Distribution Shift
We propose Token-Level Off-Policy Labeling (TOPL), an off-policy training paradigm that reframes post-training as a token-level correctness prediction task. Our key intuition is that by training the model to distinguish good and bad tokens in a response, we naturally guide the model towards generating good tokens, while avoiding the pitfalls that come with directly training the model to generate off-policy tokens.
arXiv Machine Learning
Aug 13
Weightless Fine-Tuning: Personalizing LLMs via Logit-Space Transport
arXiv:2608. 11342v1 Announce Type: new Abstract: Supervised fine-tuning (SFT) is a standard approach for adapting LLMs to a target distribution, but in settings such as personalization, where each author requires separate weight access, optimization, storage, and retraining, its costs become prohibitive.
arXiv Computation and Language
Oct 16
No Single Tokenizer Feature Reliably Predicts Downstream Language Model Performance
The paper investigates whether intrinsic tokenizer features can predict the performance of downstream language models. By training about 200 models across nine languages and evaluating numerous tokenizer metrics, the authors find that none of the existing metrics consistently forecasts model performance across different families, tasks, and languages. They propose a new linguistically grounded metric that captures aspects of tokenizer quality missed by prior measures, though its predictive power may still vary.