call(back)

Design an Accurate Ad Click Aggregator

Design the system that counts ad clicks. Advertisers want per-ad click counts by minute on a dashboard within about a minute, and the same counts drive billing, so accuracy is the primary requirement: a click must be counted once — not zero times, not twice.

Assume 10k clicks per second on average with 10× peaks, tens of millions of ads, and clients that retry, double-click, go offline and send events late or out of order. Queries are: clicks for ad X between two times at minute granularity, and the top ads in a window.

Cover: how a click is captured so it can be verified and deduplicated; ingestion; the aggregation model (event time, windows, lateness); how you get effectively-exactly-once counts through retries and restarts; hot ads; the store that serves queries; how streaming numbers are reconciled with a batch recount; and what you monitor to prove the numbers are right.

Hints

Hint 1

Give every click an identity you can trust: serve the ad with a signed impression token and route the click through your redirect endpoint, so a click is verifiable and a duplicate is recognisable by its impression id.

Hint 2

Exactly-once is a property of the whole pipeline: idempotent producers, event-time windows with a watermark, checkpointed state, and a sink that upserts by (ad, minute) instead of incrementing.

Hint 3

Keep the raw events. A batch job that recomputes from immutable raw data and overwrites the streaming aggregates is what makes billing defensible — and what lets you fix a bug after the fact.

Asked at