Your prototype performs 50k embeddings in 3ms, but the product requires 20 million. What breaks and how do you change the approach?

Scaling to 20 million embeddings increases the memory footprint significantly. Because the arithmetic intensity is only 0.5 FLOP per byte, the bottleneck becomes memory bandwidth rather than raw compute. A simple scan is no longer viable, so you must transition to approximate search methods, ensuring you can justify when a scan might still be appropriate for smaller subsets.