179 questions
No questions match those filters.
← All questions
RAGMedium· FiD· Context Window· Attention Mechanism
Why doesn’t Fusion-in-Decoder (FiD) just concatenate th...
This is one of the questions in the full AI/ML interview bank. Pro unlocks all 1789 questions; Premium includes the same bank plus the highest daily Practice limit.
See plansA weak answer focuses only on context window limits. A strong answer addresses the cost structure: self-attention over concatenated passages is 4(kn)^2d per layer, compared to 4kn^2d for independent encoding, representing a ratio of k. Independent encoding does not remove cross-passage attention; it simply relocates it to the decoder, where the interaction costs O(kn) per decoded token.