call(back)

Design a Control Plane for Cloud Clusters

Design the control plane that manages a fleet of cloud clusters: users declare clusters and their sizes, the system provisions hosts, keeps each cluster at its desired size, and detects and replaces unhealthy hosts.

Cover the desired-state model and its store, how the actual state is observed, how the two are reconciled, how host health is judged — and in particular how you separate the host is dead from the control plane cannot reach it — and what the system does when the control plane itself is partitioned or down.

Hints

Hint 1

Desired state lives in a consistent store; controllers run a reconcile loop that reads desired and actual state and issues the actions that close the gap. Agents on hosts heartbeat and report.

Hint 2

A failure detector marks a host suspect after missed heartbeats and dead after a longer window; the trade-off is detection speed against false positives.

Hint 3

Fence a host — revoke its credentials, detach its volumes, mark it in the store — before launching its replacement, or a host that was only unreachable comes back and runs beside its successor.

Asked at