Push-based KV transfer to improve TTFT and TPOT for disaggregated inference in vLLM

Published 23 October 2026

Moving KV transfer coordination off the critical path cuts time to first token by roughly 3x and improves time per output token by up to 30%.