Hugging Face Blog
Leveraging Pre-trained Language Model Checkpoints for Encoder-Decoder Models
Read the original on Hugging Face Blog →The Flow has not summarised this story yet — read it at Hugging Face Blog.
Related stories
arXiv AI
Aug 17
CForce: Boosting Parallel Decoding for dLLMs via Consistency Forcing
arXiv:2608. 13925v1 Announce Type: cross Abstract: Diffusion large language models (dLLMs) accelerate language generation by predicting multiple masks in a single forward pass.
arXiv AI
Sep 10
Good Pretraining, Bad SFT: Checkpoint Quality Across the Training Stack
arXiv:2609.08966v1 Announce Type: new Abstract: Language-model checkpoints are commonly selected by pretraining loss or benchmark scores, assuming that the highest-scoring checkpoint will remain the...
arXiv AI
Aug 11
One Adapter Pair per Model: A Universal Activation Interface for Language Models
arXiv:2608. 09521v1 Announce Type: new Abstract: Activation-based tools are usually tied to one model's native hidden space, requiring probes, sparse autoencoders, and natural-language interpreters to be rebuilt or rediscovered for each new language model.
arXiv Machine Learning
Jul 13
Complexity-Guided Component-wise Initialization for Language Model Pretraining
arXiv:2607. 09204v1 Announce Type: cross Abstract: Pretrained language models often exhibit structured weight spectra, suggesting that training may repeatedly produce similar layerwise and component-wise organization.
arXiv Computation and Language
Aug 28
Dependency-Aware Revocable Decoding for Efficient Diffusion Large Language Model Inference
The paper introduces Dependency-Aware Revocable Decoding (DARD), a training‑free framework for diffusion large language models that separates tokens into masked, candidate, and unmasked states. DARD verifies candidate tokens using a selective context that excludes less reliable tokens and adaptively regulates their influence on subsequent decoding. Experiments on 12 textual and multimodal benchmarks across three open‑source dLLMs show that DARD improves the speed‑quality Pareto frontier, achieving a 2.71× speedup and a 4.35‑point CIDEr gain over Saber on Flickr30K.
arXiv AI
Jun 2
EuroBERT: Scaling Multilingual Encoders for European Languages
arXiv:2503. 05500v3 Announce Type: replace-cross Abstract: General-purpose multilingual vector representations, used in retrieval, regression and classification, are traditionally obtained from bidirectional encoder models.