How do you design ML model serving for a multi-tenant S...

Multi-tenant ML serving fails quietly, one layer at a time, so isolation has to be designed into each layer independently rather than assumed to follow from getting one of them right.

• Data isolation. Every query carries and is scoped by tenant_id, and that scoping is enforced with row-level security at the database layer — not just as a WHERE clause in application code, which a bug or a missed code path can silently bypass.
• Model isolation, chosen deliberately. Whether tenants get individually trained models or share one model with tenant-specific features is a real architectural trade-off, not a default: dedicated models fit tenants with heavy customization needs or contractual data-separation requirements, but training and maintaining one model per tenant stops scaling well past a modest tenant count. A shared model with tenant features baked in serves the majority of tenants more sustainably.
• Compute isolation. Separate Kubernetes namespaces or resource quotas per tenant (or per tenant tier) prevent the “noisy neighbor” problem — one tenant’s traffic spike degrading latency for every other tenant sharing the same serving infrastructure. Rate limiting helps but doesn’t fully solve this without a hard resource boundary underneath it; a premium tier commonly gets dedicated pods for exactly this reason.
• Feature isolation. One tenant’s data must never leak into another tenant’s training set, including indirectly — a shared feature pipeline that pools data across tenants for “efficiency” is a common way this happens by accident.
• Audit trails. Every prediction gets logged with both tenant_id and user_id. This isn’t just good debugging hygiene — GDPR- and HIPAA-scale compliance commitments typically require an audit trail that’s queryable per tenant specifically, not only in aggregate.

The pattern worth stating explicitly in an interview: these five strategies are needed together, not as alternatives — skipping any one of them reopens exactly the isolation gap the others were meant to close.