#token-pruning

TokenGate: A New Approach to Accelerating LLM Prefill

Stanford and UC Berkeley researchers developed TokenGate, a trainable token pruning component for LLMs that accelerates long-context prefill by up to 11.3x.