How do you diagnose a drop in recall@20 after shipping...

The discrepancy occurs because the offline evaluation environment is not simulating the production filtering logic. A hard predicate mathematically bounds recall at c * a (where c is coverage and a is accuracy). By making the predicate null-tolerant, you can run an experiment that reveals whether the tagger is failing to identify the correct document type or if the filter is simply being too aggressive, allowing you to isolate the root cause.