Benchmarking Text Generation Inference

Introducing multi-backends (TRT-LLM, vLLM) support for Text Generation Inference

Faster Text Generation with Self-Speculative Decoding

Assisted Generation: a new direction toward low-latency text generation

The Effect of Text Chunk Size on Retrieval-Augmented Generation Performance

arXiv:2607. 24767v1 Announce Type: cross Abstract: Retrieval-Augmented Generation (RAG) systems have emerged as a powerful process for allowing large language models (LLMs) to retrieve relevant information to use as source material during text generation.

Forking Fast: Efficiently Estimating Uncertainty Dynamics in Text Generation

arXiv:2608. 19611v1 Announce Type: cross Abstract: LLM reasoning is stochastic, and so understanding a model requires grappling with the distribution of reasoning chains that it might produce for a given question, i.

DiffusionGemma: 4x faster text generation