Multi-tenant SaaS — One System Serving 10,000 Mutually Distrustful TenantsTenant Isolation, Data Partitioning, Quotas & Metered Billing

Scenario + Requirements

Design a B2B collaboration SaaS (Notion-like / Figma-like) with 10,000 enterprise tenants, 5M users total, too big for a single database. Tenants range from 5-person teams to 50,000-seat customers — three orders of magnitude apart. The central tension: one shared infrastructure serves everyone for cost efficiency, yet no two tenants trust each other — tenant A must never read B's data, and A's bulk import must never starve B's online requests.

• Isolation: strong data isolation (compliance: GDPR, SOC2, some customers demand data residency in a specific region); performance isolation (noisy neighbors must not bleed across).
• Scale: peak write ~50k QPS, per-tenant data 1MB~2TB, extremely skewed distribution (top 1% of tenants hold ~50% of data).
• SLO: read p99 < 100ms; large customers want 99.95% availability; onboarding a new tenant < 1 minute.
• Billing: hybrid per-seat + usage-based (API calls, storage), zero tolerance for billing errors.

Four things this issue: which isolation model, how to partition data, how to tame noisy neighbors, and how to meter usage into money.

High-level Architecture

Component roles: the gateway resolves tenant_id from subdomain / JWT and makes it a first-class citizen across the whole call path; the Tenant Catalog is the control plane storing each tenant's shard placement, isolation tier, quota, and billing plan; the data plane routes by catalog — small tenants mix into shared shards (pool), large customers move to a dedicated database (silo). Usage events flow asynchronously off to the side, into billing.

Key Technical Points

1. Isolation Model: Silo vs Pool vs Bridge — a continuum of security vs cost

Core trade-off: each notch of isolation buys security/compliance but costs efficiency and operational scalability.

[Principle] Multi-tenant isolation is not a boolean but a spectrum. AWS's vocabulary splits it three ways: Silo (each tenant a dedicated stack/DB, physical isolation), Pool (all tenants share resources, isolated by application logic), Bridge (mixed: silo what must be isolated, pool the rest). The key insight is that the isolation boundary can be chosen per layer — compute pooled, storage siloed for large customers — and that is entirely valid.

2. Data Partitioning: shard by tenant_id + row-level isolation — skew is enemy #1

Core trade-off: shared schema saves cost but isolation rides entirely on code discipline; schema/DB-per-tenant isolates hard but doesn't scale operationally.

[Principle] Under pool, three granularities: ① shared table + tenant_id column (cheapest, backed by Postgres RLS to catch missing-WHERE leaks); ② schema-per-tenant; ③ database-per-tenant (= silo). SaaS data naturally shards on tenant_id — same-tenant data lands on the same physical shard, the vast majority of queries carry tenant_id and produce no cross-shard fan-out. That is SaaS's huge advantage over generic sharding.

The hard part is skew: random hash(tenant_id)→shard, but one giant tenant can blow up a single shard. The fix is to isolate hot tenants onto their own dedicated shard.

# Postgres RLS: defense-in-depth, backstops even a missing WHERE tenant_id
ALTER TABLE docs ENABLE ROW LEVEL SECURITY;
CREATE POLICY tenant_iso ON docs
  USING (tenant_id = current_setting('app.tenant_id')::uuid);

# Inject tenant context at request start (with pooled conns, set within a txn)
SET LOCAL app.tenant_id = '...';   # all subsequent queries auto-filtered by RLS

# Routing: hash by default, hot tenants overridden via catalog
def route(tenant_id):
    t = catalog.get(tenant_id)
    if t.dedicated_shard:        # large customer isolated
        return t.dedicated_shard
    return shards[hash(tenant_id) % len(shards)]

3. Noisy Neighbors & Resource Quotas — fairness is pool's Achilles heel

Core trade-off: hard isolation (dedicated resources) prevents contagion but wastes capacity; soft quotas (shared + fair scheduling) save money but require active governance.

[Principle] Under pool, one tenant's million-row bulk import can devour the connection pool / CPU and starve another tenant's online requests — the noisy neighbor. Three lines of defense: ① ingress rate limiting (per-tenant token bucket, tiered by billing plan); ② resource quotas (max connections, query timeout, storage cap per tenant); ③ fair scheduling (per-tenant queues with round-robin, not a shared FIFO that lets a big tenant jump the line). The key: quotas must be per-tenant, not global — a global limit cannot stop a single tenant from blowing things up.

# Per-tenant token bucket + fair queue (no single tenant monopolizes workers)
def admit(tenant_id, cost):
    bucket = buckets[tenant_id]            # capacity set by billing plan
    if not bucket.try_consume(cost):
        raise RateLimited(tenant_id)        # 429 with Retry-After
    enqueue(tenant_queues[tenant_id])       # per-tenant queue

def dispatch():                             # workers round-robin, no preemption
    for q in round_robin(tenant_queues):    # even a big tenant gets only a fair share
        if (job := q.poll()): run(job)

4. Usage Metering & Billing Integration — a one-cent billing error is an incident

Core trade-off: real-time accurate metering is expensive; batched approximation is cheaper but hard to reconcile. Billing demands exactly-once semantics.

[Principle] In hybrid billing (seat + usage), the hard part is usage: every API call / storage delta is a usage event that must be aggregated into the bill without loss or duplication. The pipeline: app instrumentation → event queue → aggregation (by tenant×metric×time window) → push to Stripe metered billing. Key designs: each usage event carries an idempotency key (prevents retries from double-charging); aggregation uses at-least-once delivery + downstream idempotent dedup; usage data is the financial source of truth — persist it so it can be recomputed (replayable for billing disputes).

# Idempotent usage event: idempotency_key prevents retries from double-billing
event = {
  "idempotency_key": f"{tenant}:{request_id}",   # unique
  "tenant_id": tenant, "metric": "api_call",
  "qty": 1, "ts": now,
}
emit(usage_topic, event)        # at-least-once delivery

# Aggregate: sum by (tenant, metric, hour), dedup on idempotency_key
INSERT INTO usage_agg(tenant_id, metric, window, qty)
SELECT tenant_id, metric, date_trunc('hour', ts), sum(qty)
FROM (SELECT DISTINCT ON (idempotency_key) * FROM raw_events) d
GROUP BY 1,2,3;

Evolution & Optimization

• Start pool, evolve to bridge: all-pool early for speed; when enterprise customers arrive, isolate them to silo (lift-and-shift the single tenant's data to a dedicated DB).
• Skew governance: monitor per-shard tenant size, auto-isolate hot tenants; reshard via "dual-write + verify + cutover" (cf. Notion/Figma).
• Data residency: compliance customers demand EU/US data not leaving region — add a region dimension to the catalog and route per-tenant to the corresponding regional cluster (multi-region control plane).
• Per-tenant observability: tag all metrics/logs/traces with tenant_id, or you cannot answer "which tenant is causing this" during an incident. Cardinality explodes, so be careful with high-cardinality labels.

Pitfalls + Interview Questions

Likely follow-ups:

1. How do you choose silo / pool / bridge? What signal triggers moving a tenant from pool to silo?
2. How does the pool model prevent cross-tenant data leaks at both the code layer and the DB layer?
3. A giant tenant occupies 80% of a single shard — what do you do? How do you reshard with zero downtime?
4. Design per-tenant rate limiting where free vs enterprise have different quotas, and a hot tenant's burst doesn't hurt others.
5. How does a usage-based billing pipeline guarantee exactly-once bills and stay replayable for disputes?

Deep-dive Resources

• AWS SaaS Tenant Isolation Strategies whitepaper: authoritative definitions and implementation patterns for silo/pool/bridge.
• Notion's The Great Re-shard: sharding by workspace + zero-downtime capacity expansion, in practice.
• Figma Databases Team: decoupling logical/physical sharding + DBProxy routing.
• AWS: Multi-tenant data isolation with Postgres RLS: the row-level security pattern for tenant isolation.
• Designing Data-Intensive Applications (Kleppmann), partitioning & consistency chapters: the fundamentals of partition-key choice, skew, and rebalancing.

Going Deeper (click to expand)