← All questions
EvaluationHard· Ranking Metrics· Classification· Relevance Modeling

Your five-way relevance classifier hits 82% held-out ac...

A strong answer separates the metric from the model immediately: accuracy is order-blind and dominated by the class-0 mass, and arg max quantizes a hundred candidates onto five values, so the ranking at the head is decided by the candidate list rather than the model. It then names the one-line fix - rank by the sum of j * p_j - and quantifies it as recovering the ordering inside the tied bucket at zero inference cost. A weak answer treats the gap as a capacity problem and proposes more data, a bigger encoder, or more training epochs; none of those touch the defect, because the classifier was already right about the classes.

Code playground (Python)