If you had to use expert-choice for some reason, how wo...

If expert-choice must be used, the most common approach is to accept the train-test distribution shift by switching to token-choice routing during inference. To reduce this shift, one could constrain the training objective to ensure every token receives at least K expert invocations, effectively solving a minimum-coverage assignment problem. However, this requires a more expensive constrained solver and still results in inference behavior that the model did not encounter during training. Ultimately, expert-choice is generally avoided for single-query inference tasks.