AWS Glue Workflow for Schema-Aware Redshift ETL
Short answer
BD Use AWS Glue Workflows with integrated Crawlers to dynamically update the Data Catalog and process schema-evolving data into Redshift.
Question
Key Takeaways
The question tests the ability to handle dynamic schema changes in ETL pipelines; the trap is choosing static scheduled jobs over workflows with crawler integration.
This page explains how to build a robust AWS Glue workflow that handles schema evolution by integrating crawlers with jobs, ensuring correct data loading into Amazon Redshift.
Common mistake: Many learners select Option A because it is simpler, failing to realize that a standalone Glue job cannot automatically adapt to schema changes without manual intervention.
Expert Analysis
Why the Answer Is Correct
Options B and D are correct because they utilize AWS Glue Workflows, which can orchestrate dependent tasks like crawlers and jobs. The requirement for the pipeline to "function correctly despite changes to the data schema" necessitates the use of an AWS Glue Crawler to update the Data Catalog before the ETL Job runs. Both options implement this pattern: B uses a scheduled EventBridge trigger, while D uses an S3 event trigger via Lambda to start the workflow. This ensures the schema is refreshed dynamically.
Why the Other Options Are Wrong
Option A is incorrect because a standalone Glue Job does not automatically detect schema changes; it relies on the existing catalog definition, which would break if the source schema evolved. Option C is overly complex and introduces unnecessary latency by splitting the trigger logic across two Lambda functions instead of using a native workflow or direct invocation. Option E attempts to use Kinesis Data Firehose, which is not suitable for batch ETL loads requiring schema discovery and transformation logic typical of Glue Jobs, and it complicates the architecture unnecessarily.
Community Comment Notes
The community strongly agreed on BD (94% votes). As rralucard_ noted, the key is the "streamlined process" of workflows handling schema changes. Another user, HagarTheHorrible, emphasized that "change od schema is the key," highlighting why static options fail. Some users mistakenly chose A, but comments clarified that A lacks the crawler step required for schema adaptability.
Exam Strategy
Always look for keywords like 'schema changes' or 'evolution' in ETL questions. If present, the solution must include a mechanism to update the metadata catalog (like a Glue Crawler) before processing data. Use Workflows to chain these steps together.
Frequently Asked Questions
Why can't I just use a Glue Job triggered by EventBridge?
What is the difference between Option B and Option D?
Community Votes
100% of anonymous learners picked answer BD. Votes are pick records left by other test-takers — they are not the verified answer.
Discussion 8 contributions
Compiled from public community threads and reader submissions, lightly edited for clarity. Handles and vote tallies are shown as recorded, and comments reflect the views of individual contributors.
From public community threads 8
-
rralucard_ 👍 10Option B: Amazon EventBridge Rule with AWS Glue Workflow Job Every 15 Minutes - for its streamlined process, automated scheduling, and ability to handle schema changes. Option D: AWS Lambda to Invoke AWS Glue Workflow When a File is Loaded - for its responsiveness to file arrival and adaptability to schema changes, though it is slightly more complex than option B.
-
Felix_G 👍 2The two data pipeline solutions that will meet the requirements are: A. Use an Amazon EventBridge rule to run an AWS Glue job every 15 minutes. Configure the AWS Glue job to process and load the data into the Amazon Redshift tables. B. Use an Amazon EventBridge rule to invoke an AWS Glue workflow job every 15 minutes. Configure the AWS Glue workflow to have an on-demand trigger that runs an AWS Glue crawler and then runs an AWS Glue job when the crawler finishes running successfully. Configure the AWS Glue job to process and load the data into the Amazon Redshift tables. These solutions leverage AWS Glue to process and load the data from different file formats in the S3 bucket into the Amazon Redshift tables, while also handling changes to the data schema.
-
evntdrvn76 👍 1The correct answers are A. Use an Amazon EventBridge rule to run an AWS Glue job every 15 minutes. Configure the AWS Glue job to process and load the data into the Amazon Redshift tables and B. Use an Amazon EventBridge rule to invoke an AWS Glue workflow job every 15 minutes. Configure the AWS Glue workflow to have an on-demand trigger that runs an AWS Glue crawler and then runs an AWS Glue job when the crawler finishes running successfully. Configure the AWS Glue job to process and load the data into the Amazon Redshift tables. These solutions automate the ETL pipeline with minimal operational overhead.
-
arvehisa 👍 1I don't think this pipeline should be triggered by an s3 file upload. However seems A cannot handle the data schema change. if s3 trigger is good, then C and E are unnessessarily complexed. so I would go with B & D (despite the s3 trigger)
-
valuedate 👍 1eventbridge rule or event trigger
Show 3 more comments
-
lucas_rfsb 👍 3I will go with BD
-
HagarTheHorrible 👍 1change od schema is the key
-
Ousseyni 👍 1ChatCGT sid A and E
Reader comments & corrections
No reader corrections yet — spotted an error or have a note? Share it below.
Related Analysis
Practice All DEA-C01 Questions
Access 100 questions with complete answers and detailed explanations.
View Full DEA-C01 Practice Test →