179 questions
No questions match those filters.
← All questions
RAGVery Hard· Context Windows· Reranking· Inference Cost· Latency
Context windows are a million tokens now. The platform...
This is one of the questions in the full AI/ML interview bank. Pro unlocks all 1789 questions; Premium includes the same bank plus the highest daily Practice limit.
See plansArbitrate this dispute using arithmetic rather than abstract principles. The platform team is correct that a million-token context window can technically accommodate 200 chunks of 256 tokens (totaling 51,200 tokens). However, they are incorrect regarding the financial impact. Because prefill latency and cost scale quadratically with context length, the computational expense of passing that much data through the model far outweighs the cost of a dedicated reranker, which remains a rounding error in comparison.