179 questions
No questions match those filters.
Why does nanoGPT’s throughput increase by 25% after inc...
This is one of the questions in the full AI/ML interview bank. Pro unlocks all 1789 questions; Premium includes the same bank plus the highest daily Practice limit.
See plansThe vocabulary embedding and unembedding projections are large matmuls where one dimension is the vocabulary size. When this dimension is not a multiple of the hardware tile size (e.g., 64 or 128), the final tile is sparsely populated. This forces the SM to execute a full tile’s worth of instructions despite doing only a fraction of the useful work. By padding the vocabulary size to the nearest multiple of 64, the alignment is improved, eliminating this waste and significantly boosting throughput across the entire forward and backward pass.