TokenGate: Adapting LLMs for Fast and Memory-Efficient Prefill with Learned Token Pruning

Published 30 October 2026

TokenGate helps language models process long inputs by learning which tokens to keep and prune. It delivers 1.4–11.3× faster first-token generation and uses 1.5–3.4× less peak memory, without reducing accuracy.