179 questions
No questions match those filters.
← All questions
LLM FoundationsHard· Position Encodings· Transformers
Explain the sinusoidal encoding formula and why 10000 i...
This is one of the questions in the full AI/ML interview bank. Pro unlocks all 1789 questions; Premium includes the same bank plus the highest daily Practice limit.
See plansThe sinusoidal encoding uses a geometric series of wavelengths to represent positions. The base of 10000 is chosen to cover up to approximately 63,000 tokens. High-frequency pairs are effective for short-range discrimination, while low-frequency pairs manage long-range dependencies. Crucially, the rotation-matrix property allows the model to express relative offsets linearly.