A kernel loads a 2D tensor and runs at 3% of peak HBM b...

Use Nsight Compute to examine the sector/request ratio. If the ratio is near 32, the access pattern is non-coalesced, likely because threads are striding across rows of a row-major tensor. To fix this, you can either stage the data through shared memory (loading a tile in a coalesced manner and then accessing it within shared memory) or restructure the algorithm to ensure the tensor is accessed in row-major order.