Your custom CUDA GeLU runs at 1.8 ms but PyTorch’s runs...

The performance gap is primarily due to two factors. First, using float32 instead of bfloat16 doubles the bytes moved per element; switching to bfloat16 halves HBM traffic and bandwidth costs. Second, the kernel likely uses scalar loads instead of vectorized loads. PyTorch’s kernel uses float4 memory instructions to load 4 values per thread, fully utilizing the 128-byte HBM cache line. To close the gap, modify the kernel to advance by BLOCK_SIZE × 4, use float4 loads, and process 4 elements per thread.