Your Triton softmax runs at 1.9 ms but torch.compile ac...

A naive softmax implementation often uses two passes: one to find the maximum and another to compute the exponential and normalization. This results in redundant HBM reads. To close the performance gap, implement an online softmax. This technique tracks the running maximum (m) and running sum (d) as scalar accumulators across a loop over column tiles. At each new tile, you apply the correction term (emprev-mnew) to the accumulated sum before adding new terms, effectively reducing the operation to a single pass and significantly lowering HBM bandwidth requirements.