Falcon 2: An 11B parameter pretrained language model and VLM, trained on over 5000B tokens and 11 languages

Falcon-Edge: A series of powerful, universal, fine-tunable 1.58bit language models.

Length-MAX Tokenizer for Language Models

arXiv:2511. 20849v2 Announce Type: replace-cross Abstract: We introduce a new tokenizer for language models that minimizes the average tokens per character, thereby reducing the number of tokens needed to represent text during training and to generate text during inference.

Falcon-H1: A Family of Hybrid-Head Language Models Redefining Efficiency and Performance

Matryoshka Language Model Suites

arXiv:2608. 09703v1 Announce Type: new Abstract: Training a language model suite classically requires training each model separately and serving them independently.

Prototype Language Models

arXiv:2607. 00510v1 Announce Type: new Abstract: Knowing which training examples drive outputs is fundamental to auditing, correcting, and understanding language models, yet for modern LLMs this remains expensive, approximate, and largely post-hoc.

CLAP: Direct VLM-to-VLA Adaptation via Language-Action Grounding

arXiv:2607. 08974v1 Announce Type: cross Abstract: Vision-language-action models (VLAs) inherit semantic capabilities from pretrained VLMs, yet large-scale post-training on robot data and architectural modifications can reshape the backbone so extensively that it becomes difficult to isolate what the VLM contributes to control.

TokEval: A Tokenizer Evaluation Suite

arXiv:2608. 18062v1 Announce Type: cross Abstract: Language model tokenizers are typically selected with minimal evaluation, despite the fact that their design choices directly impact model capabilities.

LK Losses: Direct Acceptance Rate Optimization for Speculative Decoding

arXiv:2602. 23881v2 Announce Type: replace Abstract: Speculative decoding accelerates autoregressive large language model (LLM) inference by using a lightweight draft model to propose candidate tokens that are then verified in parallel by the target model.

A Sovereign, Open-Source Foundation Model for German and English

arXiv:2607. 09424v1 Announce Type: cross Abstract: We present Soofi S 30B-A3B, a sovereign, open-source Mixture-of-Experts (MoE) hybrid Mamba Transformer foundation model for German and English.

K-Forcing: Joint Next-K-Token Decoding via Push-Forward Language Modeling

arXiv:2606. 10820v1 Announce Type: cross Abstract: Autoregressive (AR) language modeling is the dominant paradigm for text generation, yet its sequential token-by-token decoding makes inference memory-bound and inefficient.

Learning When to Attend: Conditional Memory Access for Long-Context LLMs

arXiv:2603. 17484v2 Announce Type: replace-cross Abstract: Language models struggle to generalize beyond pretraining context lengths, limiting long-horizon reasoning and retrieval.

TokEval: A Tokenizer Evaluation Suite

Language model tokenizers are typically selected with minimal evaluation, despite the fact that their design choices directly impact model capabilities. This can be partly attributed to a limited understanding of which tokenizer properties affect which aspects of downstream performance.