179 questions
No questions match those filters.
← All questions
System DesignVery Hard· Parallelism· Infrastructure
How do you plan the aspect ratio and parallelism for a...
This is one of the questions in the full AI/ML interview bank. Pro unlocks all 1789 questions; Premium includes the same bank plus the highest daily Practice limit.
See plansGiven the high-speed NVLink within nodes and slower inter-node InfiniBand, you should avoid tensor parallelism across nodes. Instead, use 8-way tensor parallelism within each 8-GPU node. Use pipeline parallelism to distribute the 80 layers across 4 nodes (20 layers per node). This results in a total of 32 H100 GPUs, which is a suitable allocation for a 70B model, keeping the aspect ratio within a validated range.