call(back)

Design a Cloud Queue Service

Start from a plain in-memory Queue class with enqueue and dequeue, and turn it into a cloud service that many clients use at once. Then make it correct when a client, a server or the network fails in the middle of an enqueue or a dequeue.

Define the API, including what a client receives from dequeue and how it signals that it has finished with a message; the durability and replication behind an acknowledged enqueue; the visibility rules for a message that has been handed out; and the delivery guarantee you offer and why.

Hints

Hint 1

API: enqueue(queue, message, idempotency_key), dequeue(queue) returning a message plus a receipt, and ack(receipt). Persist and replicate before acknowledging an enqueue.

Hint 2

A dequeue hides the message for a visibility timeout rather than deleting it; no ack before the timeout means redelivery.

Hint 3

At-least-once with consumer-side dedup versus at-most-once: a dequeue whose response is lost looks exactly like a crashed consumer, so redelivery is the only safe answer, and consumers must be idempotent.

Asked at