Autoregressive generation is memory-bottlenecked at batch size 1. What are the fundamental architectural alternatives?

Autoregressive (AR) generation is fundamentally limited by memory bandwidth because each token generation requires loading the entire model weight set. Diffusion models offer an alternative by generating all tokens in parallel through an iterative denoising process. This shifts the bottleneck from memory bandwidth to compute capacity. While diffusion models provide significant throughput advantages for long outputs, they generally lag in quality for recall-heavy tasks and require a trade-off in architectural complexity.