FlashMap is a new speculative decoding drafter designed to accelerate large language model inference. By mapping Gaussian noise to a token block in a single step, the system allows a target model to verify K proposed tokens in one parallel pass. This approach maintains the output quality of the original target model while achieving a speedup of 6.14 times.
The architecture utilizes a 6-block diffusion transformer with an embedding dimension of 2048 and 16 attention heads. It processes hidden states from layers 1, 9, 17, 25, and 33 of the target model. The system was trained on 500,000 samples from the Open-PerfectBlend dataset, requiring approximately 4,600 GPU hours on H200 hardware.
Researchers evaluated the method across 9 benchmarks covering math, code, and chat tasks at temperatures 0 and 1. Tests showed that drafting 4 noise samples as a tree increased average speed by 11 percent. The implementation is available as a package requiring torch and transformers version 4.56 or higher, supporting execution on a single GPU.
