RW-TTT: Batched Serving for Request-Owned Test-Time Training State
Read the original on arXiv Machine Learning →RW‑TTT addresses the challenge of serving large language models (LLMs) that perform test‑time training (TTT) by reading and updating request‑owned state, such as fast weights or streaming learner state. The method tags each decoding step with its owner, version, and READ/WRITE effect, batches only compatible phases, and commits updates solely to the owning request, thereby avoiding state corruption. On a single GPU with eight fast‑weight InPlace‑TTT streams, RW‑TTT achieves 274.61 aggregate tokens per second—9.31× faster than sequential serving and 3.44× faster than per‑stream replicas—while maintaining correct behavior on the RULER long‑context benchmark.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.