Design a Centralized Logging System
Design the logging platform for a large company: thousands of services on tens of thousands of hosts emit logs, and engineers search them to debug incidents, follow a request by trace id, and drive alerts.
Assume ~2M log events per second at peak, ~1 KB each. Recent logs (7–30 days) must be searchable within seconds of being written; older logs are kept a year for audit at much lower cost. Emitting a log line must never slow down or take down the service that emits it, and one noisy service must not starve everyone else.
Cover: collection on the host, the ingestion path, how producers are decoupled from consumers, partitioning, storage tiers and indexing strategy, the query path, delivery guarantees, multi-tenancy and sensitive data, and how you monitor the logging system itself.
Hints
Hint 1
Do the arithmetic: 2M events/s × 1 KB is ~2 GB/s, roughly 170 TB a day before compression. That number rules out indexing every token of every line forever and forces tiers.
Hint 2
Put a durable log (Kafka or similar) between collection and everything else. It absorbs bursts, lets indexers fall behind without losing data, and lets you add consumers — alerting, archival, security — without touching producers.
Hint 3
Indexing is the cost lever. Compare a full inverted index on hot data with a label index plus compressed chunks that are scanned in parallel; most queries filter by service, time range and trace id first.
Asked at
Sign in to talk to the interviewer — ask what's in scope, what scale to design for, or for a nudge when you're stuck. Conversations save to your account.