← All questions
EvaluationMedium· Benchmarks· Data Generation· Retrieval

A model performs differently on Natural Questions (NQ)...

A benchmark’s collection protocol determines which retriever wins. SQuAD’s surface-level similarity makes it a ‘lexical matching’ task, while NQ’s paraphrase-heavy nature makes it a ‘semantic retrieval’ task. A benchmark that does not match how your users write queries will recommend the wrong retriever.

Code playground (Python)