AWS Glue Workflows vs Step Functions for ETL Orchestration
A company maintains multiple extract, transform, and load (ETL) workflows that ingest data from the company's operational databases into an Amazon S3 based data lake. The ETL workflows use AWS Glue and Amazon EMR to process data. The company wants to improve the existing architecture to provide automated orchestration and to require minimal manual effort. Which solution will meet these requirements with the LEAST operational overhead?
Community Votes
74% of anonymous learners picked answer B. Votes are pick records left by other test-takers — they are not the verified answer.
Community Insight
The core trap is assuming Step Functions is required to bridge different services; however, AWS Glue Workflows natively support triggering both Glue Jobs and EMR applications, making them the preferred choice for data lake pipelines.
This question addresses the best practice for orchestrating AWS Glue and EMR ETL workflows with minimal operational overhead. It clarifies that while Step Functions is a general-purpose orchestrator, AWS Glue Workflows are the native, low-code solution specifically designed for this data engineering stack.
Most learners choose Step Functions (B) because they believe it is the only way to orchestrate heterogeneous services (Glue + EMR), overlooking that Glue Workflows have expanded to include EMR triggers.
Community Discussion (21 comments)
Comments & Corrections
No comments yet — spotted an error or have a note? Share it below.
Expert Analysis
Why the Answer Is Correct
AWS Glue Workflows are specifically designed to automate and manage complex ETL pipelines involving multiple AWS Glue jobs, crawlers, and triggers. Crucially, AWS Glue Workflows also support Amazon EMR as a step in the workflow. By using Glue Workflows, you can define the sequence of Glue and EMR jobs without writing custom code or managing additional infrastructure, thus meeting the 'LEAST operational overhead' requirement.Why the Other Options Are Wrong
Step Functions (B) is a powerful serverless orchestrator, but it requires significant manual effort to set up state machines, error handling, and IAM roles to connect Glue and EMR. While possible, it does not offer the 'least operational overhead' compared to the managed, purpose-built Glue Workflows. Amazon MWAA (D) introduces the complexity of managing Apache Airflow clusters, which is higher overhead than Glue Workflows. Lambda (C) is not an orchestration tool suitable for long-running ETL dependencies.Community Comment Notes
Many commenters argue for Step Functions based on older documentation stating Glue couldn't trigger EMR. As one user noted, "For me it's B because I did not found a possibility how Glue can trigger/orchestrate EMR processes OOTB." However, AWS has since added EMR support to Glue Workflows, rendering this reasoning outdated. Another commenter correctly identified Glue Workflows as the best fit for 'least operational overhead' by leveraging seamless integration.Official Reference
Exam Strategy
When asked about 'least operational overhead' for AWS-native data services (Glue, EMR, Redshift), look for the service's native management features first. Do not default to general-purpose tools like Step Functions unless the architecture explicitly requires cross-service logic beyond what the native tool supports.
Frequently Asked Questions
Can AWS Glue Workflows trigger Amazon EMR jobs?
Yes, AWS Glue Workflows support Amazon EMR applications as steps, allowing you to chain Glue jobs and EMR processing tasks together.
Why is Step Functions not the best choice here?
Step Functions require more configuration and coding effort. Glue Workflows are purpose-built for ETL and offer lower operational overhead for this specific stack.
Related Analysis
Practice All DEA-C01 Questions
Access 100 questions with complete answers and detailed explanations.
View Full DEA-C01 Practice Test →