FlashAttention-3 adds warp-specialization and uses H100...

In FlashAttention-2, the online-softmax update in registers can stall the next GEMM from issuing. FlashAttention-3 uses H100-specific hardware features: the Tensor Memory Accelerator (TMA) for asynchronous data movement and the WGMMA instruction for asynchronous matrix multiplication. This allows one warp group to perform the softmax update and prefetch the next KV tile via TMA while another warp group executes the GEMM. A100 lacks these specific hardware primitives, making such efficient pipelining impossible without explicit barriers that negate the latency benefits.