179 questions
No questions match those filters.
← All questions
Inference & ServingMedium· Tensor Parallelism· Memory Management· Optimization
Tensor parallelism divides your parameter memory by t....
This is one of the questions in the full AI/ML interview bank. Pro unlocks all 1789 questions; Premium includes the same bank plus the highest daily Practice limit.
See plansIn vanilla TP, matmul activations (FFN intermediate, attention projections) divide by t, but pointwise activations stay full-size. Sequence parallelism addresses this by replacing the row-parallel AllReduce with ReduceScatter and AllGather, allowing GPU k to hold only s/t rows through pointwise ops, effectively dividing memory for those layers by t without changing total communication costs.