I swap two rows of the input table and your model’s answer changes. Is that a bug in the model or in your evaluation?

A relation is a set of tuples, meaning row order should be invariant. The model’s sensitivity to row swaps stems from the additive row-ID embedding, which injects order-dependent signals into the input tokens. The evaluation fails to catch this because aggregate accuracy metrics compare marginals and effectively hide the instability. Simply fine-tuning on shuffled data is a poor fix, as it does not address the underlying architectural reliance on order.