How to decouple and scale ingestion of 100,000 messages per second?

SAA-C03 By islever Editorial Last updated

Design scalable and loosely coupled architectures.

Short answer

D Publish messages to an Amazon SNS topic with multiple Amazon SQS subscriptions so each consumer processes independently from its own queue.

Question

A company has an application that ingests incoming messages. Dozens of other applications and microservices then quickly consume these messages. The number of messages varies drastically and sometimes increases suddenly to 100,000 each second. The company wants to decouple the solution and increase scalability.
Which solution meets these requirements?
  1. Persist the messages to Amazon Kinesis Data Analytics. Configure the consumer applications to read and process the messages.
  2. Deploy the ingestion application on Amazon EC2 instances in an Auto Scaling group to scale the number of EC2 instances based on CPU metrics.
  3. Write the messages to Amazon Kinesis Data Streams with a single shard. Use an AWS Lambda function to preprocess messages and store them in Amazon DynamoDB. Configure the consumer applications to read from DynamoDB to process the messages.
  4. Publish the messages to an Amazon Simple Notification Service (Amazon SNS) topic with multiple Amazon Simple Queue Service (Amazon SOS) subscriptions. Configure the consumer applications to process the messages from the queues. Correct Answer

Key Takeaways

This question tests whether you can distinguish true pub/sub decoupling with SQS buffering from analytics or single-shard streaming designs; the trap is assuming Kinesis Data Analytics or one Kinesis shard can absorb sudden 100K/sec bursts.

The scenario requires a fully decoupled, elastically scalable message ingestion path for bursty traffic that can suddenly reach 100,000 messages per second. Publishing to an Amazon SNS topic with multiple Amazon SQS subscriptions lets dozens of consumers process independently, making D the correct answer.

Common mistake: Many learners choose Kinesis Data Streams because it is a streaming service, but option C uses a single shard that cannot handle the required throughput and forces consumers to poll DynamoDB instead of using a durable queue per consumer.

Expert Analysis

Why the Answer Is Correct

Option D uses Amazon SNS fan-out to multiple Amazon SQS subscriptions, which cleanly decouples the ingestion application from dozens of consumer applications and microservices. Each consumer can process from its own SQS queue independently, so a slow or failed consumer does not block the others, and SQS buffers sudden spikes instead of forcing synchronous processing. Standard SQS queues and SNS topics are designed for high-throughput, distributed workloads, making this the right architecture for traffic that can jump to 100,000 messages per second. The design also supports retries, dead-letter queues, and independent scaling per consumer, which directly addresses the scalability requirement.

Why the Other Options Are Wrong

Option A misuses Amazon Kinesis Data Analytics, which is an analytics service that reads from streaming sources rather than a general-purpose message distribution layer for dozens of applications. Option B scales EC2 instances based on CPU but does not decouple producers from consumers, and Auto Scaling reacts too slowly for sudden 100,000-messages-per-second bursts. Option C is limited by a single Kinesis shard, whose throughput is far below the stated burst, and having consumers poll DynamoDB adds latency and unnecessary read pressure. None of these alternatives provides the durable, per-consumer buffering and fan-out model that SNS with SQS subscriptions provides.

Community Comment Notes

PhucVuu correctly warned not to confuse Kinesis Data Analytics with Kinesis Data Streams, noting that analytics services are for analysis rather than delivering messages to consumer applications. Channn focused on the key requirement with the short note “Key words: decouple,” which aligns with the SNS-to-SQS fan-out design. Buruguduystunstugudunstu pointed out that option C would make consumers continuously poll DynamoDB, which could impact performance, reinforcing why D is cleaner. Some learners raised SNS throughput concerns, but the standard SNS-to-SQS fan-out pattern is the intended scalable decoupling solution for this scenario.

Official Reference

These links open the official vendor documentation in a new tab.

Exam Strategy

When a question says “decouple” and mentions many consumers or microservices, look for fan-out with one SQS queue per consumer. Throughput limits should steer you away from single-shard Kinesis and toward SNS with SQS subscriptions.

Frequently Asked Questions

Why is Kinesis Data Streams with a single shard wrong for 100,000 messages per second?

A single Kinesis shard supports roughly 1 MB/s or 1,000 records/s, so it cannot absorb a sudden 100K/sec burst. Option D scales through SNS fan-out to multiple SQS queues.

Does SNS fan-out to SQS still fit if dozens of microservices consume the same messages?

Yes. Each microservice gets its own SQS subscription and queue, so it consumes independently, buffers bursts, and can scale or retry without affecting other consumers.

More SAA-C03 FAQ →

Discussion 155 contributions

Compiled from public community threads and reader submissions, lightly edited for clarity. Handles and vote tallies are shown as recorded, and comments reflect the views of individual contributors.

From public community threads 30 of 155

  1. PhucVuu 👍 31 Answered D
    Keywords: - The number of messages varies drastically - Sometimes increases suddenly to 100,000 each second A: Incorrect - Don't confuse between Kinesis Data Analytics and Kinesis Data Stream =)) Kinesis Data Analytics will get the data from Kinesis Data Stream or Kinesis Data FireHose or MSK (Managed Stream for apache Kafka) for analytic purpose. It can not consume message and send to applications. B: Incorrect - Base on the keywords -> Auto Scaling group not scale well because it need time to check the CPU metric and need time to start up the EC2 and the messages varies drastically. Example: we have to scale from 10 to 100 EC2. Our servers may be down a while when it was scaling. C: Incorrect - Kinesis Data Streams can handle this case but we should increase the more shards but not single shard. D: Correct: We can handle high workload well with fan-out pattern SNS + multiple SQS -> This is good for use case: - The number of messages varies drastically - Sometimes increases suddenly to 100,000 each second
  2. SilentMilli 👍 23
    By default, an SQS queue can handle a maximum of 3,000 messages per second. However, you can request higher throughput by contacting AWS Support. AWS can increase the message throughput for your queue beyond the default limits in increments of 300 messages per second, up to a maximum of 10,000 messages per second. It's important to note that the maximum number of messages per second that a queue can handle is not the same as the maximum number of requests per second that the SQS API can handle. The SQS API is designed to handle a high volume of requests per second, so it can be used to send messages to your queue at a rate that exceeds the maximum message throughput of the queue.
  3. daizy 👍 23
    D. Publish the messages to an Amazon Simple Notification Service (Amazon SNS) topic with multiple Amazon Simple Queue Service (Amazon SQS) subscriptions. Configure the consumer applications to process the messages from the queues. This solution uses Amazon SNS and SQS to publish and subscribe to messages respectively, which decouples the system and enables scalability by allowing multiple consumer applications to process the messages in parallel. Additionally, using Amazon SQS with multiple subscriptions can provide increased resiliency by allowing multiple copies of the same message to be processed in parallel.
  4. career360guru 👍 9 Answered A
    A is correct SNS can not handle 100K per second message throughput. As per the documentation it can only handle upto 300 msgs/sec.
  5. Gatt 👍 6 Answered A
    Why D? Won't the 100,000 messages per second requirements make SNS a poor choice? Since AWS states that SNS has a per region limit of API requests per second and the maximum is 30,000. But then, C is also wrong, since 1 shard can only support a maximum input of 1000. B is wrong since you're not decoupling and once it increases to 100,000 ASG won't keep up, as it take too long to adapt. A could be right? If you already have a stream in place, you can then feed it onto Kinesis Analytics and apply whatever procedures you were doing initially with your applications.
Show 25 more comments
  1. Ello2023 👍 6
    This answer is also valid, however do you think it would benefit to pay more by being with Kineses which is real time (Unnecessary) and writing an SQL custom code to process this when you can simply have SNS with SQS which process more data, sever-less, cheaper.
  2. channn 👍 6 Answered D
    Key words: decouple
  3. Buruguduystunstugudunstuy 👍 5
    Option C involves writing the messages to Amazon Kinesis Data Streams with a single shard and using an AWS Lambda function to preprocess the messages and store them in DynamoDB. This option would not be the most efficient solution, as it would require the consumer applications to continuously poll DynamoDB for new messages, which could impact their performance. Overall, Option D is the most effective solution for decoupling the solution and increasing scalability.
  4. PaoloRoma 👍 5 Answered A
    A) decouples and scales - reference to Kinesis Data Analytics implies using Kinesis Data Streams to me B) scales but it is not clear how it decouples C) decouples, but does not scale: single shard and single lambda D) decouples, but does not scale: single shard and single lambda
  5. cookieMr 👍 4 Answered D
    Option A: It is more suitable for real-time analytics and processing of streaming data rather than decoupling and scaling message ingestion and consumption. Option B: It may help with scalability to some extent, but it doesn't provide decoupling. Option C: It is a valid option, but it lacks the decoupling aspect. In this approach, the consumer applications would still need to read directly from DynamoDB, creating tight coupling between the ingestion and consumption processes. Option D: It is the recommended solution for decoupling and scalability. The ingestion application can publish messages to an SNS topic, and multiple consumer apps can subscribe to the relevant SQS queues. SNS ensures that each message is delivered to all subscribed queues, allowing the consuming apps to independently process the messages at their own pace and scale horizontally as needed. This provides loose coupling, scalability, and fault tolerance, as the queues can handle message spikes and manage the consumption rate based on the consumer's processing capabilities.
  6. Andreshere 👍 4 Answered D
    The key word for this question is “decouple”. When we want to decouple an application (with microservices), we want to factor it as much as possible. One pattern that allows that decoupling is the fan-out pattern, which sets an SNS topic with many SQS queues that process the messages. So, the correct answer is D. A. Using Amazon KDA is not suitable because the lack of consumers and customers. It is better to use Amazon Kinesis Data Streams, which processes real-time data from many different sources (consumers). Additionally, persisting does not make sense for the statement requirements. B. EC2 instances are not feasible for supporting high workloads with unpredictable traffic spikes. Moreover, it is expensive and inefficient because EC2 has not mechanisms to avoid traffic spikes and decoupling. C. Even though using Amazon KDS is okay, it cannot handle with traffic spikes with that number of shards. Additionally, this solution is operationally complex since we use DynamoDB, which can have a bottleneck.
  7. Buruguduystunstugudunstuy 👍 4
    CORRECT ANSWER The correct solution that meets these requirements is Option D: Publish the messages to an Amazon Simple Notification Service (Amazon SNS) topic with multiple Amazon Simple Queue Service (Amazon SQS) subscriptions. Configure the consumer applications to process the messages from the queues. Option D involves using Amazon Simple Notification Service (SNS) and Amazon Simple Queue Service (SQS) to decouple the solution and increase scalability. SNS is a fully managed, publish-subscribe messaging service that allows you to send messages to multiple recipients simultaneously. SQS is a fully managed, distributed message queue service that enables you to store, process, and transmit messages between microservices, distributed systems, and serverless applications.
  8. Buruguduystunstugudunstuy 👍 4
    To implement this solution, you would first publish the incoming messages to an SNS topic. You could then create multiple SQS subscriptions to the SNS topic, and configure the consumer applications to process the messages from the queues. This approach allows you to decouple the ingestion application from the consumer applications, and it allows you to scale the number of consumer applications independently of the ingestion application.
  9. iCcma 👍 4
    friends certainly D makes a lot of sense, so even with the answer I gave, I'm curious, here I leave a link with a debate with very good arguments. I want to change my answer to D
  10. gilang_from_indonesia 👍 4
    It should be D. Key words: Persist, decouple
  11. D2w 👍 4 Answered D
    Decouple is your keyword for SQS
  12. Stevey 👍 3
    D. is the answer. The question states that there are dozens of other applications and microservices that consume these messages and that the volume of messages can vary drastically and increase suddenly. Therefore, you need a solution that can handle a high volume of messages, distribute them to multiple consumers, and scale quickly. SNS with SQS provides these capabilities. Publishing messages to an SNS topic with multiple SQS subscriptions is a common AWS pattern for achieving both decoupling and scalability in message-driven systems. SNS allows messages to be fanned out to multiple subscribers, which in this case would be SQS queues. Each consumer application could then process messages from its SQS queue at its own pace, providing scalability and ensuring that all messages are processed by all consumer applications. A. Amazon Kinesis Data Analytics is primarily used for real-time analysis of streaming data. It's not designed to distribute messages to multiple consumers.
  13. IdanAWS 👍 3 Answered D
    D. Here’s why option D is the correct choice: Amazon SNS: Amazon SNS is a fully managed pub/sub messaging service that enables message publishing and subscription to topics. It provides fast and flexible communication between publishers and subscribers. Amazon SQS: Amazon SQS is a fully managed message queuing service that decouples the components of a distributed application. It offers reliable and scalable queues for storing messages and enables applications to process them asynchronously. By publishing the messages to an Amazon SNS topic and using Amazon SQS subscriptions, the solution achieves decoupling and scalability. Multiple applications and microservices can subscribe to the topic and receive messages through their individual SQS queues. This allows for parallel processing and enables the system to handle varying message volumes, including spikes of up to 100,000 messages per second.
  14. hrushikeshrelekar 👍 3
    Option D Amazon SNS allows you to publish messages to a topic, which can then fan out those messages to multiple subscribers. By using Amazon SQS as a subscriber to the SNS topic, you can handle the message load in a decoupled and scalable way. SQS can store messages until the consuming application is ready to process them, helping to smooth out the variance in message load. This approach allows the company to effectively decouple the message producing applications from the consuming applications, and it can easily scale to handle the high load of messages. The number of messages (100,000 each second) might require careful configuration and sharding of SQS queues or use of FIFO queues to ensure that they can handle the load. Options A, B, and C have their own limitations:
  15. SilentMilli 👍 3 Answered C
    By writing the incoming messages to Kinesis Data Streams and using a Lambda function to preprocess and store the messages in DynamoDB, the company can decouple the ingestion application from the consumer applications and increase scalability. Kinesis Data Streams can automatically scale to handle the sudden increases in the number of messages, and the Lambda function can be triggered to process the messages as they arrive. The consumer applications can then read and process the messages from DynamoDB.
  16. shinejh0528 👍 3
    oh... I confused between Kinesis Data Analytics and Kinesis Data Stream as you mentioned... I solved several this type of questions, but SNS is always about 'notification', so i choose A. but i think Kinesis Data Analytics is just wrong, so D is most correct answer.
  17. Six_Fingered_Jose 👍 3 Answered D
    I go for D because decoupling = SQS in most cases, and they did mention SNS into multiple SQS so with multiple SQS you can handle more than 100k requests
  18. TUANHA2312 👍 3 Answered D
    keypoint is: The company wants to decouple the solution and increase scalability.
  19. NikaCZ 👍 3
    multiple subscribers at the same time = SNS. decoupling = SQS
  20. Teerx 👍 3
    Who decided the correct answer is A ?? the admin ??
  21. SilentMilli 👍 3
    After much thought, I think D is the right answer.
  22. Mourner 👍 3 Answered D
    Company needs to decouple and improve scalabilty.
  23. BoboChow 👍 3 Answered D
    This a senario of Fan Out Pattern
  24. mikey2000 👍 3
    Why is A the correct answer?
  25. 9014 👍 3
    of course, the answer is D

Reader comments & corrections

No reader corrections yet — spotted an error or have a note? Share it below.

Log in to comment, report an error, or add a note about this question.

Scheduled scaling for a predictable morning EC2 traffic spike

S-grade

Improving RDS Performance with Provisioned IOPS SSD

S-grade

S3 Multi-Region Access Point and Replication Strategy

S-grade

Securing NLB with Security Groups for Minimal Change

S-grade

Practice All SAA-C03 Questions

Access 120 questions with complete answers and detailed explanations.

View Full SAA-C03 Practice Test →

← Back to SAA-C03 Study Guide