Should we convert an eight-shot in-context learning blo...

Converting a demonstration block to a LoRA adapter is technically flawed because the update is additive, rank-bounded, and applied at every layer, but it is also query-conditioned. Because the demonstrations are retrieved per query, there is no static delta to bake. Instead of training, you should use prefix KV caching to remove the prefill latency at zero training risk. Fine-tuning should only be considered based on the step budget, not cost savings.