call(back)

Design a File Sync Service

Design a Dropbox-style service: a folder that stays in sync across a user's devices, with version history and the ability to share folders and links with other people.

Assume hundreds of millions of users, files from a few bytes to tens of gigabytes, devices that edit offline, and networks where re-uploading a whole large file for a one-line change is unacceptable.

Cover: how files are broken up, stored and deduplicated; the metadata model and how devices learn what changed; the upload and download flows and the moment a new version exists; conflicts between devices; version history and garbage collection; sharing and permissions; security; and how it scales.

Hints

Hint 1

Split every file into blocks of a few megabytes and address each block by its hash. A file version is just an ordered list of block hashes, so an edit uploads only the blocks that changed.

Hint 2

Metadata is a journal per namespace: every commit gets the next sequence number, devices keep a cursor, and a long-poll or push tells them to fetch. A shared folder is its own namespace mounted into each member's tree.

Hint 3

A commit names the revision it was based on. If that is no longer the latest, do not overwrite — keep both as a conflicted copy.

Asked at