You are designing speculative decoding for a 70B chat m...

When designing for a 70B model on limited hardware, the primary constraint is the KV cache and weight memory footprint. A standalone 7B draft model often consumes too much VRAM. EAGLE is the preferred choice because it uses a lightweight, co-located autoregressive module that is distilled from the target model. This approach maximizes the acceptance rate while staying within the strict memory budget of two H100 GPUs.