You benchmarked allreduce and got 100 GB/s on a 4-GPU H...

When observed bandwidth is significantly lower than peak, first verify that NCCL is utilizing NVLink rather than falling back to PCIe by checking NCCL_DEBUG=INFO. Second, ensure the tensor size is sufficiently large (typically >10 MB), as small tensors are dominated by latency rather than bandwidth. Third, confirm that the benchmark includes a proper warm-up phase and correct synchronization. Finally, investigate whether SHARP (Scalable Hierarchical Aggregation and Reduction Protocol) is enabled, as it can significantly boost allreduce performance.