Design a distributed message queue
Build a durable, high-throughput message broker that many producers and consumers share, like Kafka.
Functional requirements
- Producers publish messages to topics
- Many independent consumers read at their own pace
- Messages are retained for a configurable time
Non-functional requirements
- No acknowledged message is ever lost
- Millions of messages per second
- Messages with the same key stay in order
Scale
- 10M messages per second
- 100 TB retained
- Thousands of consumers
How you’ll be scored
10 checks covering your diagram and your notes, and any valid approach counts. Hints appear for anything you miss. Common mistakes, such as single points of failure or components you never explain, cost points. How grading works
What interviewers look for
- Clarify before you draw. State what is in and out of scope. For “Design a distributed message queue”, that starts with: Producers publish messages to topics; Many independent consumers read at their own pace; Messages are retained for a configurable time. Interviewers notice candidates who pin the requirements down first.
- Turn the scale into numbers. The scale (10M messages per second; 100 TB retained; Thousands of consumers) should become requests per second, storage and bandwidth, and those figures should drive your choices. Estimates with units carry more weight than adjectives.
- Show how each quality is met. Your design should visibly deliver: No acknowledged message is ever lost; Millions of messages per second; Messages with the same key stay in order. Saying that it does isn’t enough; point to the part of the design that makes it true.
- Expect follow-ups on the hard parts. Interviewers usually probe logs, replication and partitioning here. This is a hard problem, so interviewers expect depth: what breaks first at this scale, how each failure is contained, and which trade-offs you accepted on purpose.
- Explain failure and trade-offs. Say what happens when a component fails, where a single point of failure would hurt, and what you chose not to build. Naming a trade-off is a stronger answer than claiming there isn’t one.