← All questions
Inference & ServingHard· Compute-bound· Recomputation· Latency

Describe a case where recomputation-for-speed makes the...

If an operation is compute-bound (with an arithmetic intensity above the ridge point), the compute units are already fully saturated. Adding recomputation FLOPs forces the pipeline to wait, directly increasing the kernel’s latency. For example, applying recomputation inside a large tiled matmul adds unnecessary FLOPs to an already saturated pipeline, increasing total execution time. Always measure arithmetic intensity before deciding if recomputation is ‘free’ or ‘costly’.

Code playground (Python)