How does the capacity planning model change when the se...

Capacity planning shifts significantly when the goal is throughput-per-dollar. AR throughput is limited by memory bandwidth, becoming independent of batch size once memory is saturated. Conversely, diffusion throughput scales with compute and improves as output length L increases relative to refinement steps K. The cost-per-token crossover point is at L = K. Therefore, planning should involve provisioning diffusion GPUs for the long-output tail of the distribution and AR GPUs for the short-output head to maximize cost efficiency.