179 questions
No questions match those filters.
← All questions
LLM FoundationsHard· Induction Heads· In-Context Learning
Your model correctly quotes a part number that cannot b...
This is one of the questions in the full AI/ML interview bank. Pro unlocks all 1789 questions; Premium includes the same bank plus the highest daily Practice limit.
See plansInduction heads enable this behavior by identifying the pattern [A][B]…[A] and predicting [B]. A one-layer head can score on (ti, tj), but because the position to attend to is identified by tj-1, a previous-token head must first shift that information forward. The resulting composed matrix is near-identity over the vocabulary, allowing the model to generalize to unseen strings.