When would you choose W8A8 over W8A16 for production, a...

W8A8 quantization is beneficial when the workload is compute-bound, which occurs at higher batch sizes (typically above 295 for common projection sizes). Below this threshold, the kernel remains memory-bound, meaning activation quantization adds unnecessary complexity and accuracy risk without providing the throughput benefits of INT8 tensor cores. The primary risk with W8A8 is outlier-channel clipping. Engineers must monitor activation distributions; if more than 0.1% of values are clipped, per-channel activation scales are required to maintain model accuracy.