179 questions
No questions match those filters.
Walk me through how you would design a context-extensio...
This is one of the questions in the full AI/ML interview bank. Pro unlocks all 1789 questions; Premium includes the same bank plus the highest daily Practice limit.
See plansA successful context-extension design requires a graduated schedule that moves beyond simple architectural changes. It is essential to incorporate document-contiguous packing and a diverse corpus that includes books, math, arXiv papers, legal filings, and long-form technical documentation. Relying solely on one domain, such as fiction, will result in a model that struggles with the structural dependencies found in technical or legal writing. Architectural adjustments, such as RoPE scaling, are necessary to handle the increased token capacity.