What is the difference between Adam and AdamW, and why...

The distinction lies in how the weight decay term interacts with the adaptive learning rate. In Adam with L2 regularization, the decay term enters the gradient and is then scaled by the adaptive factor 1/√vi, which varies per parameter based on gradient history. This breaks the intended conditioning. AdamW decouples the weight decay from the gradient update, applying it directly to the parameter. This ensures uniform decay, which leads to measurably different and superior training dynamics at large scales.