Cost-Effective Serverless Data Processing for Small Files
A car sales company maintains data about cars that are listed for sale in an area. The company receives data about new car listings from vendors who upload the data daily as compressed files into Amazon S3. The compressed files are up to 5 KB in size. The company wants to see the most up-to-date listings as soon as the data is uploaded to Amazon S3. A data engineer must automate and orchestrate the data processing workflow of the listings to feed a dashboard. The data engineer must also provide the ability to perform one-time queries and analytical reporting. The query solution must be scalable. Which solution will meet these requirements MOST cost-effectively?
Community Votes
100% of anonymous learners picked answer D. Votes are pick records left by other test-takers — they are not the verified answer.
Community Insight
It tests the ability to recognize that serverless orchestration (Lambda/Step Functions) and query engines (Athena) are more cost-effective for small files than provisioned clusters or EMR.
This question evaluates the selection of a serverless, cost-effective data processing architecture using AWS Glue and Athena for small, frequent S3 uploads.
Learners often choose EMR-based solutions (Options A or B) because they associate EMR with big data, failing to realize that starting an EMR cluster for 5KB files is prohibitively expensive and slow.
Community Discussion (3 comments)
Comments & Corrections
No comments yet — spotted an error or have a note? Share it below.
Expert Analysis
Why the Answer Is Correct
Option D is the correct answer because it utilizes a fully serverless stack that aligns perfectly with the requirements for handling small (5 KB), high-frequency files. AWS Glue is the appropriate ETL service for transforming this data without managing infrastructure. Orchestrating workflows via AWS Lambda triggered by S3 Event Notifications provides immediate processing upon upload, satisfying the 'most up-to-date' requirement. Amazon Athena allows for scalable, one-time queries directly on the S3 data without loading it into a warehouse, which is ideal for ad-hoc reporting.Why the Other Options Are Wrong
Options A and B propose using Amazon EMR. Provisioning an EMR cluster involves significant startup time and fixed costs, making it highly inefficient and costly for processing tiny 5 KB files. Option C suggests using Redshift Spectrum; while Spectrum can query S3 data, setting up and managing a Redshift environment adds unnecessary complexity and potential latency compared to the simplicity of Athena for this specific scale. Additionally, OpenSearch Service (used in A and C) is generally overkill for simple dashboarding of structured sales listings compared to QuickSight.Community Comment Notes
The community consensus strongly supports Option D. User axantroff noted a hesitation about Lambda orchestrating workflows but correctly identified Athena as superior to Redshift for this use case. User HagarTheHorrible highlighted that setting up a Redshift cluster takes much longer than the serverless approach in D. The comments reflect a clear understanding that 'serverless equals cost-effective' for small-file scenarios.Exam Strategy
When you see keywords like 'small files', 'cost-effective', and 'ad-hoc queries', immediately rule out provisioned compute services like EMR or Redshift clusters. Look for the combination of Glue (ETL), Lambda/SNS (Trigger), and Athena (Query).
Frequently Asked Questions
Why not use EMR for data processing?
EMR requires provisioning a cluster, which incurs high startup costs and latency. For 5 KB files, the overhead outweighs the benefits, making Glue + Lambda far more cost-effective.
Is Lambda suitable for orchestrating workflows?
Yes, for simple event-driven triggers. While Step Functions handles complex state machines, Lambda is sufficient here to trigger Glue jobs upon S3 upload, keeping the architecture lightweight.
Related Analysis
Practice All DEA-C01 Questions
Access 100 questions with complete answers and detailed explanations.
View Full DEA-C01 Practice Test →