call(back)

Design an Audit Log Service

Design a service that records audit events — who did what, to which resource, when — for many accounts, and answers queries scoped to one account: by time range, actor, resource or action.

Cover the write path at high volume, the storage layout and how long events are kept, the read path for low-latency account-scoped queries, and how the log resists tampering. Say how much indexing you are willing to pay for on ingest and why.

Hints

Hint 1

Partition by account and then by time: every query is account-scoped, so an account's events sit together and a time range is a contiguous scan.

Hint 2

Keep recent events in a hot store with secondary indexes for the common filters, and age closed time buckets into object storage in a columnar format with per-file min/max metadata.

Hint 3

A hash chain per account — each event carries the hash of the previous — makes deletion or alteration detectable; anchor the chain's head periodically to a write-once location.

Asked at