Why does the decay phase recover so much loss so quickl...

The stable phase maintains a high learning rate, which keeps the model in a noisy equilibrium between gradient signal and step size, preventing it from getting trapped in any single basin. The decay phase is critical because it gradually reduces the step size, allowing the optimizer to effectively navigate toward the bottom of the loss basin. This annealing process is the primary driver of final loss improvement, which is why early termination of cosine schedules often results in significantly worse model performance.