← All questions
Retrieval & EmbeddingsHard· Lexical Retrieval· Ranking· Normalization

Your lexical retriever returns section headings and tab...

When headings or captions dominate, the mechanism is usually a ratio-based normalization where the denominator is minimized by the short length of the snippet, artificially inflating the score. To fix this, you can implement a minimum chunk length at indexing time, switch to a model with a tunable length term, or apply pivoted normalization. It is important to measure the impact of these changes rather than simply adjusting parameters like k without addressing the underlying length bias.

Code playground (Python)