Designing Asynchronous ML Microservices with ECS and SQS

SAA-C03 By islever Editorial Last updated Oct 13, 2026

Design scalable and loosely coupled architectures.

Short answer

D Direct requests to SQS and deploy ML models as Amazon ECS services that read from the queue and auto-scale based on queue size.

Question

A company is developing machine learning (ML) models on AWS. The company is developing the ML models as independent microservices. The microservices fetch approximately 1 GB of model data from Amazon S3 at startup and load the data into memory. Users access the ML models through an asynchronous API. Users can send a request or a batch of requests. The company provides the ML models to hundreds of users. The usage patterns for the models are irregular. Some models are not used for days or weeks. Other models receive batches of thousands of requests at a time. Which solution will meet these requirements?

Direct the requests from the API to a Network Load Balancer (NLB). Deploy the ML models as AWS Lambda functions that the NLB will invoke. Use auto scaling to scale the Lambda functions based on the traffic that the NLB receives.
Direct the requests from the API to an Application Load Balancer (ALB). Deploy the ML models as Amazon Elastic Container Service (Amazon ECS) services that the ALB will invoke. Use auto scaling to scale the ECS cluster instances based on the traffic that the ALB receives.
Direct the requests from the API into an Amazon Simple Queue Service (Amazon SQS) queue. Deploy the ML models as AWS Lambda functions that SQS events will invoke. Use auto scaling to increase the number of vCPUs for the Lambda functions based on the size of the SQS queue.
Direct the requests from the API into an Amazon Simple Queue Service (Amazon SQS) queue. Deploy the ML models as Amazon Elastic Container Service (Amazon ECS) services that read from the queue. Use auto scaling for Amazon ECS to scale both the cluster capacity and number of the services based on the size of the SQS queue. Correct Answer

Key Takeaways

The scenario tests the ability to decouple compute from request ingestion via SQS while selecting a container-based runtime (ECS) capable of handling large model loads that exceed Lambda memory limits.

This question evaluates the design of a scalable, loosely coupled architecture for asynchronous machine learning workloads using Amazon ECS and Amazon SQS to handle irregular traffic patterns.

Common mistake: Many candidates select AWS Lambda (Option C or A), incorrectly assuming it can handle the 1 GB model data load; however, Lambda's maximum memory limit is 10 GB but cold starts and execution time constraints make it less suitable than ECS for persistent, heavy model loading in this specific context.

Expert Analysis

Why the Answer Is Correct

The correct solution is D because it perfectly addresses the requirements for loose coupling and scalability. Using Amazon SQS decouples the API layer from the processing layer, allowing the system to absorb bursts of thousands of requests without overwhelming the compute resources. Amazon ECS is chosen over Lambda because the models require loading approximately 1 GB of data into memory at startup. While Lambda supports up to 10 GB of memory, the requirement to keep models in memory for serving suggests a longer-running process better suited to containers. ECS allows for persistent connections and efficient memory management for these heavy microservices.

Why the Other Options Are Wrong

Option A and C suggest using AWS Lambda. Lambda has a maximum execution timeout of 15 minutes and, more importantly, incurs cold start latency every time the function is invoked after an idle period. Given that "some models are not used for days or weeks," the cold start penalty combined with the need to load 1GB of data makes Lambda inefficient and potentially slow for user requests. Option B uses an Application Load Balancer (ALB), which does not provide the necessary decoupling; if the backend ECS instances are overwhelmed by sudden bursts, the ALB will simply queue or drop requests rather than buffering them as SQS would.

Community Comment Notes

As Erfan noted, SQS decouples the API from ML processing, ensuring reliable request processing. RamanAgarwal argued for Lambda due to cost, but missed the implication of "irregular" usage causing cold starts. FlyingHawk highlighted that ECS is safer because it avoids Lambda's execution limits and cold start issues. LeonSauveterre pointed out that loading 1GB into memory is challenging for Lambda's ephemeral nature compared to ECS containers.

Exam Strategy

When designing for asynchronous workloads with variable traffic, always prioritize decoupling components using managed queues like SQS or SNS. For compute, evaluate whether the workload requires long-running processes or large memory footprints that might incur significant cold-start penalties on serverless options.

Frequently Asked Questions

Why not use Lambda for ML models?

Lambda has cold start issues and execution timeouts. Loading 1GB of data repeatedly for irregular traffic makes it slower and less reliable than persistent ECS containers.

Is SQS necessary if we have ALB?

Yes. SQS provides buffering and decoupling. An ALB cannot buffer thousands of concurrent requests effectively if the backend is slow, leading to timeouts.

More SAA-C03 FAQ →

Community Votes

64%

18%

64% of anonymous learners picked answer D. Votes are pick records left by other test-takers — they are not the verified answer.

Discussion 5 contributions

Compiled from public community threads and reader submissions, lightly edited for clarity. Handles and vote tallies are shown as recorded, and comments reflect the views of individual contributors.

From public community threads 5

[Removed] 👍 5 Answered D

D is the answer: SQS Queue: Directing API requests to SQS decouples the API from ML processing, efficiently handles high traffic, and ensures reliable request processing without overloading the ML models. Amazon ECS Services: Running ML models on ECS provides effective management of containerized applications, ideal for handling ML workloads. Auto Scaling: ECS auto scales based on SQS queue size, adjusting container and cluster capacity to match demand, ensuring efficient handling of varying workloads.
RamanAgarwal 👍 2 Answered C

Since AWS Lambda supports 1GB of memory and can be scaled seamlessly, the ans should be C. Question says a lot of APIs are not used frequently so keeping them in ECS will result in higher costs. Also these are async operations means the response is not time bound hence we can wait for lambda to startup and scale up based on the size of the SQS queue.
kbgsgsgs 👍 2 Answered B

Why should I use SQS in option D? Wouldn't ALB be enough?
LeonSauveterre 👍 1 Answered D

A - Lambda has a cold start latency. There are ways to optimize that (like the environment variable config which I actually did when I used Lambda once), but loading 1GB into memory is still too much. B - Using ALB makes handling sudden bursts of traffic even more challenging. C - Just like A if I'm correct. D - Each ML model needs to load 1 GB of model data at startup ==> Containers in ECS are well-suited for this because they provide persistent execution environments that avoid the overhead of reloading the model data frequently.
FlyingHawk 👍 1 Answered D

Given that the problem statement does not specify the processing time, Option D (ECS) is the safer choice because it does not have the same limitations as Lambda(cold start and 15 ,minutes execution). However, if the processing time is short and cold starts are acceptable, Option C (Lambda) could also be a cost-effective solution.

Reader comments & corrections

No reader corrections yet — spotted an error or have a note? Share it below.

Practice All SAA-C03 Questions

Access 120 questions with complete answers and detailed explanations.

View Full SAA-C03 Practice Test →

← Back to SAA-C03 Study Guide