179 questions
No questions match those filters.
How would you distribute a 70B-parameter model across a...
This is one of the questions in the full AI/ML interview bank. Pro unlocks all 1789 questions; Premium includes the same bank plus the highest daily Practice limit.
See plansDistributing a 70B model requires a strategy that respects the physical hardware constraints. For a 64-GPU cluster, one must leverage high-speed interconnects like NVLink for intra-node communication and InfiniBand for inter-node communication.
The chosen parallelism strategy must be evaluated against memory and bandwidth limitations. This involves estimating the overhead caused by pipeline bubbles or communication latency, and ensuring that the batch size is effectively managed as a resource to maximize throughput without exceeding memory limits.