← All questions
RAGHard· System Design· Evaluation· Inference & Serving

Platform wants to ship GNN-RAG for a two-orders-of-magn...

The latency win and the accuracy regression are not in tension; they measure different aspects of the system. The regression is the expected cost of swapping a symbolic retriever for a learned one on the low-training-coverage multi-hop tail. The adjudication is to implement routing: send high-confidence, in-distribution traffic to GNN-RAG, and keep the symbolic route as a fallback for low-confidence or out-of-distribution queries. This allows you to split the eval results by route, resolving the disagreement into two correct, distinct performance metrics.

Code playground (Python)