AWS Glue Workflows vs Step Functions for ETL Orchestration

DEA-C01 By islever Editorial Last updated

Orchestrate data pipelines.

Short answer

A Use AWS Glue Workflows to natively orchestrate AWS Glue and Amazon EMR jobs with minimal setup.

Question

A company maintains multiple extract, transform, and load (ETL) workflows that ingest data from the company's operational databases into an Amazon S3 based data lake. The ETL workflows use AWS Glue and Amazon EMR to process data. The company wants to improve the existing architecture to provide automated orchestration and to require minimal manual effort. Which solution will meet these requirements with the LEAST operational overhead?
  1. AWS Glue workflows Correct Answer
  2. AWS Step Functions tasks
  3. AWS Lambda functions
  4. Amazon Managed Workflows for Apache Airflow (Amazon MWAA) workflows

Key Takeaways

The core trap is assuming Step Functions is required to bridge different services; however, AWS Glue Workflows natively support triggering both Glue Jobs and EMR applications, making them the preferred choice for data lake pipelines.

This question addresses the best practice for orchestrating AWS Glue and EMR ETL workflows with minimal operational overhead. It clarifies that while Step Functions is a general-purpose orchestrator, AWS Glue Workflows are the native, low-code solution specifically designed for this data engineering stack.

Common mistake: Most learners choose Step Functions (B) because they believe it is the only way to orchestrate heterogeneous services (Glue + EMR), overlooking that Glue Workflows have expanded to include EMR triggers.

Expert Analysis

Why the Answer Is Correct

AWS Glue Workflows are specifically designed to automate and manage complex ETL pipelines involving multiple AWS Glue jobs, crawlers, and triggers. Crucially, AWS Glue Workflows also support Amazon EMR as a step in the workflow. By using Glue Workflows, you can define the sequence of Glue and EMR jobs without writing custom code or managing additional infrastructure, thus meeting the 'LEAST operational overhead' requirement.

Why the Other Options Are Wrong

Step Functions (B) is a powerful serverless orchestrator, but it requires significant manual effort to set up state machines, error handling, and IAM roles to connect Glue and EMR. While possible, it does not offer the 'least operational overhead' compared to the managed, purpose-built Glue Workflows. Amazon MWAA (D) introduces the complexity of managing Apache Airflow clusters, which is higher overhead than Glue Workflows. Lambda (C) is not an orchestration tool suitable for long-running ETL dependencies.

Community Comment Notes

Many commenters argue for Step Functions based on older documentation stating Glue couldn't trigger EMR. As one user noted, "For me it's B because I did not found a possibility how Glue can trigger/orchestrate EMR processes OOTB." However, AWS has since added EMR support to Glue Workflows, rendering this reasoning outdated. Another commenter correctly identified Glue Workflows as the best fit for 'least operational overhead' by leveraging seamless integration.

Official Reference

https://docs.aws.amazon.com/glue/latest/dg/automating-with-workflows.html

This link opens the official vendor documentation in a new tab.

Exam Strategy

When asked about 'least operational overhead' for AWS-native data services (Glue, EMR, Redshift), look for the service's native management features first. Do not default to general-purpose tools like Step Functions unless the architecture explicitly requires cross-service logic beyond what the native tool supports.

Frequently Asked Questions

Can AWS Glue Workflows trigger Amazon EMR jobs?

Yes, AWS Glue Workflows support Amazon EMR applications as steps, allowing you to chain Glue jobs and EMR processing tasks together.

Why is Step Functions not the best choice here?

Step Functions require more configuration and coding effort. Glue Workflows are purpose-built for ETL and offer lower operational overhead for this specific stack.

More DEA-C01 FAQ →

Community Votes

B
74%
A
26%

74% of anonymous learners picked answer B. Votes are pick records left by other test-takers — they are not the verified answer.

Discussion 21 contributions

Compiled from public community threads and reader submissions, lightly edited for clarity. Handles and vote tallies are shown as recorded, and comments reflect the views of individual contributors.

From public community threads 21

  1. valuedate 👍 15 Answered B
    Glue Workflow only orchestrate crawlers and glue jobs
  2. DevoteamAnalytix 👍 7 Answered B
    For me it's B because I did not found a possibility how Glue can trigger/orchestrate EMR processes OOTB. But with StepFunction there is a way: https://aws.amazon.com/blogs/big-data/orchestrate-amazon-emr-serverless-jobs-with-aws-step-functions/
  3. lucas_rfsb 👍 6 Answered A
    Since it seems to me that this pipeline is complex, with multiple workflows, I would go for Glue workflows.
  4. [Removed] 👍 4 Answered B
    Orchestrating different AWS services is a typical use case for Step Functions: https://docs.aws.amazon.com/step-functions/latest/dg/connect-emr.html https://docs.aws.amazon.com/step-functions/latest/dg/connect-glue.html
  5. VerRi 👍 4 Answered B
    There is no way for Glue Workflow to trigger EMR
Show 16 more comments
  1. rralucard_ 👍 3 Answered A
    Option A, AWS Glue Workflows, seems to be the best solution to meet the requirements with the least operational overhead. It offers a seamless integration with the company's existing AWS Glue and Amazon EMR setup, providing a managed and straightforward way to orchestrate their ETL workflows without extensive additional setup or manual intervention.
  2. jasango 👍 3
    Yo me voy por la D) Amazon MWAA porque Glue Workflows solo admite Jobs de Glue y Step Function puede fucionar pero no son workflows de datos. Amazon MWAA son workflows de datos y esta integrado tanto con Glue como EMR: https://aws.amazon.com/blogs/big-data/simplify-aws-glue-job-orchestration-and-monitoring-with-amazon-mwaa/
  3. FunkyFresco 👍 3 Answered B
    EMR in workflows , i dont think so
  4. certplan 👍 2
    Here's an example of how you can use AWS Glue to initiate an EMR (Elastic MapReduce) job: Let's assume you have an AWS Glue job that performs ETL tasks on data stored in Amazon S3. You want to leverage EMR for a specific task within this job, such as running a complex Spark job. 1. Define a Glue Job: Create an AWS Glue job using the AWS Glue console, SDK, or CLI. Define the input and output data sources, as well as the transformations you want to apply. 2. Incorporate EMR Step: Within the Glue job script, include a section where you define an EMR step. An EMR step is a unit of work that performs a specific task on an EMR cluster. Code follows in the next entry...
  5. acoshi 👍 2 Answered A
    https://aws.amazon.com/blogs/big-data/orchestrate-an-etl-pipeline-using-aws-glue-workflows-triggers-and-crawlers-with-custom-classifiers/
  6. ttpro1995 👍 2 Answered B
    We have both Glue job and EMR job, so we need Step Functions to connect those. Airflow can do it, but required more dev work.
  7. V0811 👍 1 Answered A
    AWS Glue Workflows are specifically designed for orchestrating ETL jobs in AWS Glue. They allow you to define and manage complex workflows that include multiple jobs and triggers, all within the AWS Glue environment.Integration: AWS Glue workflows seamlessly integrate with other AWS Glue components, making it easier to manage ETL processes without the need for external orchestration tools.Minimal Operational Overhead: Since AWS Glue is a fully managed service, using Glue workflows will reduce the operational overhead compared to managing separate orchestrators or building custom solutions.While D. Amazon Managed Workflows for Apache Airflow (Amazon MWAA) is also a good choice for more complex orchestration, it may involve more management overhead compared to the more straightforward AWS Glue workflows. Thus, AWS Glue workflows provide the least operational overhead given the context of this scenario.
  8. Rpathak4 👍 1 Answered A
    Why Not the Other Options? B. AWS Step Functions More flexible but requires manual setup of states and transitions for Glue & EMR. Higher operational overhead than Glue Workflows. C. AWS Lambda Lambda is not ideal for long-running ETL workflows. Best suited for lightweight data transformations or event-driven tasks. D. Amazon MWAA (Apache Airflow) More control but requires cluster management and custom DAGs. Higher maintenance than Glue Workflows.
  9. plutonash 👍 1 Answered B
    it is interesting to choose A for minimum effort but only step functions can trigger the work both on EMR and on GLUE jobs
  10. Adrifersilva 👍 1 Answered A
    glue workflows is part of the glue ecosystem so its provides seamless integration with minimal changes
  11. Shanmahi 👍 1 Answered A
    Glue workflows are managed services and best for considering least operational overhead.
  12. Palee 👍 1 Answered B
    The company wants to improve the existing architecture so A cannot be the right choice
  13. HunkyBunky 👍 1 Answered B
    B - because AWS Glue can't trigger EMR
  14. GiorgioGss 👍 5 Answered B
    orchestrating = step function
  15. TonyStark0122 👍 2
    Glue Work flows
  16. Shatheesh 👍 1
    Answer A, Glue workflows

Reader comments & corrections

No reader corrections yet — spotted an error or have a note? Share it below.

Log in to comment, report an error, or add a note about this question.

Amazon Redshift Compound Sort Key Query Optimization

S-grade

Secure Redshift Spectrum Access with VPC Endpoints

S-grade

Athena Federated Query for Multi-Source Iceberg Pipelines

S-grade

Reclaiming Storage in Amazon Redshift Materialized Views

S-grade

Practice All DEA-C01 Questions

Access 100 questions with complete answers and detailed explanations.

View Full DEA-C01 Practice Test →

← Back to DEA-C01 Study Guide