Two engineers disagree on whether to buy a reranker or...

The adjudication process should focus on identifying the ‘shape’ of the failure. Rerankers are designed to move L1 and L2 precision, whereas graph indices are machinery for multi-hop aggregation. If the failures are primarily at the L3 or L4 level, both proposed solutions are likely wrong, and the investment should instead go toward corpus improvements, such as better documentation or a case store with outcome labels. The staff-level approach is to label 200 failures by level to determine which proposal actually addresses the root cause.