How to process only daily incremental S3 data in AWS Glue with least effort?

Transform and process data. Automate data processing by using AWS services.

Answer Correct answer: B — Enable AWS Glue job bookmarks on the ETL jobs so each run tracks previously processed S3 data and reads only the daily increment.

A company has developed several AWS Glue extract, transform, and load (ETL) jobs to validate and transform data from Amazon S3. The ETL jobs load the data into Amazon RDS for MySQL in batches once every day. The ETL jobs use a DynamicFrame to read the S3 data. The ETL jobs currently process all the data that is in the S3 bucket. However, the company wants the jobs to process only the daily incremental data. Which solution will meet this requirement with the LEAST coding effort?

Create an ETL job that reads the S3 file status and logs the status in Amazon DynamoDB.
Enable job bookmarks for the ETL jobs to update the state after a run to keep track of previously processed data. Correct Answer
Enable job metrics for the ETL jobs to help keep track of processed objects in Amazon CloudWatch.
Configure the ETL jobs to delete processed objects from Amazon S3 after each run.

Community Votes

100%

100% of anonymous learners picked answer B. Votes are pick records left by other test-takers — they are not the verified answer.

Community Insight

The question tests native incremental processing state in AWS Glue; the trap is confusing CloudWatch job metrics or deleting S3 objects with true bookmark-based tracking.

AWS Glue job bookmarks track previously processed S3 objects so an ETL job can read only daily incremental data with minimal code. This page confirms option B as the least-effort solution and explains why custom DynamoDB tracking, CloudWatch job metrics, or S3 object deletion do not meet the requirement.

A common wrong choice is C: enabling Glue job metrics provides CloudWatch visibility but does not track which S3 objects were already processed, so the job still reads the full bucket.

Community Discussion (4 comments)

tgv 👍 8 Selected: B

AWS Glue job bookmarks are designed to handle incremental data processing by automatically tracking the state.

andrologin 👍 1 Selected: B

AWS Glue Bookmarks can be used to pin where the data processing last stopped hence help with incremental processing.

HunkyBunky 👍 1 Selected: B

B - bookmarks is a key

bakarys 👍 3 Selected: B

The solution that will meet this requirement with the least coding effort is Option B: Enable job bookmarks for the ETL jobs to update the state after a run to keep track of previously processed data. AWS Glue job bookmarks help ETL jobs to keep track of data that has already been processed during previous runs. By enabling job bookmarks, the ETL jobs can skip the processed data and only process the new, incremental data. This feature is designed specifically for this use case and requires minimal coding effort. Options A, C, and D would require additional coding and operational effort. Option A would require creating a new ETL job and managing a DynamoDB table. Option C would involve setting up job metrics and CloudWatch, which doesn’t directly address processing incremental data. Option D would involve deleting data from S3 after processing, which might not be desirable if the original data needs to be retained. Therefore, Option B is the most suitable solution.

Comments & Corrections

No comments yet — spotted an error or have a note? Share it below.

Expert Analysis

Why the Answer Is Correct

The ETL jobs read S3 through a DynamicFrame, and AWS Glue job bookmarks are the native state-tracking mechanism for exactly that pattern: after a run, Glue stores checkpoint information and the next run skips objects or partitions already processed. Because the requirement is daily incremental processing with the least coding effort, enabling bookmarks (option B) satisfies both the functional need and the code-minimization constraint. No custom status table, no manual deletion, and no external orchestration is required; the job itself can retain its batch load into RDS for MySQL while reading only new S3 data. This is why option B aligns with Glue's designed behavior rather than with a workaround.

Why the Other Options Are Wrong

Option A would require building and maintaining a DynamoDB status ledger plus code to read and update it before and after each Glue run, which is more coding and operational overhead than bookmarks. Option C enables Glue job metrics, which are useful for CloudWatch monitoring of run duration, success, and throughput, but metrics are observability data and do not prevent reprocessing of old S3 objects. Option D deletes processed S3 objects after each run; even if that appears to leave only new data, it risks data loss, breaks replay and backfill, and is not a state-tracking solution for partial or failed loads. None of these alternatives meets the "least coding effort" qualifier as cleanly as job bookmarks.

Community Comment Notes

Community learners were aligned on B: tgv explained that Glue job bookmarks are designed for incremental processing by automatically tracking state. andrologin described the mechanism as "AWS Glue Bookmarks can be used to pin where the data processing last stopped", which captures the checkpoint behavior well. HunkyBunky likewise called bookmarks "a key", reinforcing that this is the standard Glue feature rather than a custom DynamoDB or CloudWatch approach. The comments add no dissenting technical rationale that would outweigh the AWS Glue documentation.

Official Reference

https://docs.aws.amazon.com/glue/latest/dg/monitor-continuations.html

Exam Strategy

When a DEA-C01 question asks for the least coding effort to process only new S3 data in Glue, look for native Glue job bookmarks before custom state tracking. Distinguish observability features such as job metrics from stateful incremental processing.

Frequently Asked Questions

Why doesn't enabling Glue job metrics make the job read only new S3 data?

Glue job metrics publish run-level CloudWatch metrics; they do not persist which S3 objects or partitions were already processed, so the next run still scans all data.

Do Glue job bookmarks work with DynamicFrame reads from Amazon S3?

Yes. Job bookmarks are Glue's native state-tracking mechanism for S3-backed sources, so a DynamicFrame read can skip previously processed files without a custom DynamoDB table.

More DEA-C01 FAQ →

Related Analysis

Practice All DEA-C01 Questions

Access 100 questions with complete answers and detailed explanations.

View Full DEA-C01 Practice Test →

← Back to DEA-C01 Study Guide