If a malicious actor fine-tunes an open-weight model on...

Current research demonstrates that standard RLHF-based alignment can be significantly degraded within 100 to 500 fine-tuning steps when a model is trained to comply with harmful prompts. This is a known and unresolved limitation of current alignment techniques. Consequently, for open-weight models, capability audits are essential because they provide a safety assessment that exists independently of the model’s current refusal rate.