← All questions
Inference & ServingHard· Model Routing· Calibration· Error Costs

Your three-way complexity classifier is 65% accurate. D...

You must evaluate the cost of different error directions. Over-routing a simple query to an expensive path wastes latency but still returns a correct answer. Under-routing a complex query to a simple path returns a wrong answer that downstream components cannot recover. Therefore, you should calibrate asymmetrically—routing up unless the classifier is confident—even if it means accepting worse expected latency. Acknowledge that because the classifier decides before retrieval, it lacks the evidence to be truly uncertain, making its confidence unreliable on out-of-distribution queries.

Code playground (Python)