We need to adapt a large model to a highly technical do...

Be careful about what LoRA actually buys you here. Low-rank adaptation is excellent at instruction tuning — shifting format, register, rule-following behavior. It’s much weaker at injecting genuinely new, high-rank domain knowledge, because a low-rank update is mathematically limited in how much it can shift the model’s underlying representation.

The risk with defaulting to standard LoRA on a highly technical domain is a model that writes like a specialist — the right tone, the right jargon — without actually knowing the specialism. That’s a worse failure mode than an obviously-wrong answer, because it’s more convincing.

For genuinely deep domain knowledge under GPU constraints, I’d reach for: higher rank and more adapted layers than a default instruction-tune config, continual pre-training on domain text if the budget allows it, or — often the most practical answer — retrieval to supply the substance while LoRA supplies the behavior and format on top.