How do you detect a regression in production before use...

To detect regressions proactively, you should track a quality heatmap where the rolling mean score per intent cluster is monitored using exponential smoothing. Set an alert threshold for when any cluster drops more than 0.3 points within a 6-hour window. Additionally, monitor latency and refusal rates, as safety over-refusals are frequently early indicators of model regressions occurring during a new rollout.