← All questions
Training & Fine-tuningHard· RAFT· Hyperparameters· Evaluation

What do you set P to, and how would you know you got it...

P should not be a static number but a reflection of the model’s prior over context sufficiency, anchored to the retrieval system’s recall@k. To find the right value, perform a sweep and select based on the combined accuracy metric A = rg + (1-r)b, where g is accuracy with golden context and b is accuracy with bad context. This requires evaluation splits separated by retrieval success; otherwise, the terms cannot be estimated, and the sweep remains unselectable.

Code playground (Python)