Which AWS service cost-effectively orchestrates Glue crawler and ETL to S3?

DEA-C01 By islever Editorial Last updated

Orchestrate data pipelines. Perform data ingestion. Understand data cataloging systems.

Short answer

B AWS Glue workflows natively orchestrate the SQL Server crawler and downstream Glue ETL job to Amazon S3 more cost-effectively than Step Functions or MWAA.

Question

A company created an extract, transform, and load (ETL) data pipeline in AWS Glue. A data engineer must crawl a table that is in Microsoft SQL Server. The data engineer needs to extract, transform, and load the output of the crawl to an Amazon S3 bucket. The data engineer also must orchestrate the data pipeline. Which AWS service or feature will meet these requirements MOST cost-effectively?
  1. AWS Step Functions
  2. AWS Glue workflows Correct Answer
  3. AWS Glue Studio
  4. Amazon Managed Workflows for Apache Airflow (Amazon MWAA)

Key Takeaways

This question tests native orchestration of Glue crawlers and jobs; the trap is confusing Glue Studio's visual authoring with Glue workflows' scheduling and dependency management.

AWS Glue workflows are the most cost-effective way to orchestrate a Glue crawler over Microsoft SQL Server and the downstream ETL job that writes to Amazon S3. This page explains why Glue workflows beat Step Functions, Glue Studio, and Amazon MWAA for this exact DEA-C01 scenario.

Common mistake: A frequent mistake is choosing AWS Step Functions because it is a familiar orchestrator, but it requires extra custom integration and state-machine work for a Glue crawler-to-job flow that Glue workflows handle natively and more cheaply.

Expert Analysis

Why the Answer Is Correct

AWS Glue workflows are purpose-built to chain Glue crawlers, triggers, and ETL jobs with dependencies, so the SQL Server crawl and the downstream load to Amazon S3 can be orchestrated as one managed workflow. The crawler supports Microsoft SQL Server through a JDBC connection, and the Glue job writes the transformed output to S3. Because Glue workflows are serverless and you pay mainly for the crawler and job runs, this avoids the fixed environment cost of Amazon MWAA and the custom state-machine development that Step Functions would require. The question asks for the most cost-effective service or feature, and Glue workflows are the native, lowest-overhead fit for this exact crawler-to-ETL pipeline.

Why the Other Options Are Wrong

AWS Step Functions is a general-purpose orchestrator that can invoke Glue jobs, but it requires you to design and maintain a state machine and add custom integration for a workflow that Glue workflows already model natively. AWS Glue Studio is primarily a visual authoring environment for creating Glue jobs; it is not the orchestration layer for managing crawler triggers and job dependencies across the pipeline. Amazon MWAA is a managed Apache Airflow service that is appropriate for complex, Airflow-specific DAGs, but its environment cost and operational overhead make it a poor fit when the requirement is a simple Glue crawler and ETL job sequence. Glue workflows therefore satisfy the crawling, ETL, orchestration, and cost-effectiveness requirements more directly than any other option.

Community Comment Notes

milofficial points to an AWS Big Data blog post about orchestrating an ETL pipeline using Glue workflows, triggers, and crawlers, which matches the scenario's crawler-to-job orchestration need. DevoteamAnalytix agrees with B and notes that "CRAWLING and ETL are the main functions of a Glue workflow" while also confirming Microsoft SQL Server support. dev_vicente's AI-assisted comparison concludes that Glue workflows are designed specifically for ETL and integrate directly with sources such as Microsoft SQL Server, and Shubham1989 calls Glue the easiest choice here.

Official Reference

These links open the official vendor documentation in a new tab.

Exam Strategy

When a question asks for orchestration of Glue crawlers and jobs with the least operational overhead and cost, look for Glue workflows before general-purpose orchestrators. Reserve Step Functions for cross-service workflows and Amazon MWAA for complex Airflow-specific needs.

Frequently Asked Questions

Why is AWS Glue Studio not the answer for orchestrating the SQL Server crawler and ETL job?

Glue Studio is primarily a visual authoring environment for Glue jobs; orchestration of crawlers, triggers, and job dependencies is handled by Glue workflows.

When should I choose Step Functions or Amazon MWAA instead of Glue workflows?

Step Functions suits cross-service workflows with custom states, and MWAA suits complex Airflow DAGs; for a Glue crawler-to-job pipeline both add cost and effort.

More DEA-C01 FAQ →

Community Votes

B
100%

100% of anonymous learners picked answer B. Votes are pick records left by other test-takers — they are not the verified answer.

Discussion 6 contributions

Compiled from public community threads and reader submissions, lightly edited for clarity. Handles and vote tallies are shown as recorded, and comments reflect the views of individual contributors.

From public community threads 6

  1. milofficial 👍 9 Answered B
    Glue workflows are the easiest solution here: https://aws.amazon.com/blogs/big-data/orchestrate-an-etl-pipeline-using-aws-glue-workflows-triggers-and-crawlers-with-custom-classifiers/ https://aws.amazon.com/blogs/big-data/extracting-multidimensional-data-from-microsoft-sql-server-analysis-services-using-aws-glue/
  2. dev_vicente 👍 7 Answered B
    I asked an AI. Analysis of the answers: A. AWS Step Functions: It is a good option for orchestrating workflows with steps from different AWS services, but requires additional development to connect to Microsoft SQL Server. B. AWS Glue Workflows: This is the best and most profitable option. AWS Glue is designed specifically for ETL on AWS and integrates directly with data sources such as Microsoft SQL Server through connectors. This allows for easier configuration and avoids the need for additional development. C. AWS Glue Studio: It is a visual interface for AWS Glue that makes it easy to create and manage ETL jobs. However, the underlying functionality comes from AWS Glue (B) workflows. D. Amazon Managed Workflows for Apache Airflow (Amazon MWAA): It's a viable option, but it's generally more expensive than native AWS services like AWS Glue Workflows. Additionally, it requires some Airflow experience for setup and maintenance.
  3. DevoteamAnalytix 👍 3 Answered B
    Agree with B. CRAWLING and ETL are the main functions of a Glue workflow and MS SQL is supported: https://docs.aws.amazon.com/glue/latest/dg/crawler-data-stores.html
  4. Adrifersilva 👍 1 Answered B
    https://community.aws/content/2iBQiAGS4RvEolgSQKu4iF8InTV/choose-the-right-data-orchestration-service-for-your-data-pipeline?lang=en
  5. Shubham1989 👍 1 Answered B
    Glue is easiest here to choose from.
Show 1 more comment
  1. Alcee 👍 1
    Is B !

Reader comments & corrections

No reader corrections yet — spotted an error or have a note? Share it below.

Log in to comment, report an error, or add a note about this question.

Amazon Redshift Compound Sort Key Query Optimization

S-grade

Secure Redshift Spectrum Access with VPC Endpoints

S-grade

Athena Federated Query for Multi-Source Iceberg Pipelines

S-grade

Reclaiming Storage in Amazon Redshift Materialized Views

S-grade

Practice All DEA-C01 Questions

Access 100 questions with complete answers and detailed explanations.

View Full DEA-C01 Practice Test →

← Back to DEA-C01 Study Guide