Least-Overhead Schema Discovery for Changing S3 Text Files

DEA-C01 By islever Editorial Last updated

Design data models and schema evolution. Understand data cataloging systems.

Short answer

C Use an AWS Glue crawler with the built-in classifier to catalog the S3 text files so Athena and Quick Suite query predefined columns with automatic schema updates.

Question

A company has text files that are stored in an Amazon S3 bucket. The structure of the files can change over time as the company adds new columns to the files. The company wants to give users the ability to query the data by using Amazon Athena and Amazon Quick Suite (previously known as Amazon QuickSight). The company wants to use predefined column names for the data in the text files.
Which solution will meet these requirements with the LEAST operational overhead?
  1. Create a table in Athena that points to the S3 location and uses the predefined column names. Use the table in Athena and Quick Suite. Update the table when the company adds new columns.
  2. Write a custom grok classifier to match the predefined column names and schema. Create an AWS Glue crawler that uses the grok classifier and creates a table. Use the table in Athena and Quick Suite. Run the crawler when the company adds new columns.
  3. Use the built-in classifier in AWS Glue. Create an AWS Glue crawler that creates a table. Use the table in Athena and Quick Suite. Run the crawler when the company adds new columns. Correct Answer
  4. Write a custom XML classifier to match the predefined column names and schema. Create an AWS Glue crawler that uses the XML classifier and creates a table. Use the table in Athena and Quick Suite. Run the crawler when the company adds new columns.

Key Takeaways

The question tests who owns schema evolution: the LEAST-overhead choice delegates column-name mapping and new-column detection to AWS Glue's built-in classifier and crawler instead of custom grok/XML patterns or manual DDL.

For text files in Amazon S3 whose columns change over time, an AWS Glue crawler using the built-in classifier creates and refreshes the Data Catalog table that Amazon Athena and Amazon Quick Suite query. This answer (C) minimizes operational overhead versus custom classifiers or manual Athena table edits.

Common mistake: Option A is the tempting wrong answer: a direct Athena table with predefined column names seems simplest, but manually updating the table whenever new columns appear is recurring operational overhead.

Expert Analysis

The deciding qualifier is "LEAST operational overhead"

Every option satisfies the functional requirement — all four end with a table that Amazon Athena and Amazon Quick Suite can query, so the discriminator is purely maintenance effort as columns keep being added to the text files. The qualifier eliminates candidates one by one.

Option A gives predefined column names immediately, but "Update the table when the company adds new columns" is a manual ALTER TABLE task each time the file structure changes. That is recurring human effort, so A fails the LEAST test.

Option B and Option D both require you to "Write a custom grok classifier" / "Write a custom XML classifier" that matches the predefined column names and schema. Authoring, testing and maintaining a custom pattern is exactly the overhead the question wants you to avoid — and an XML classifier is mismatched to plain text files anyway. Both are eliminated by the same overhead clause.

Option C is the only one that leans on the "built-in classifier in AWS Glue": the crawler infers the schema from the S3 objects and writes it to the Data Catalog, and re-running the crawler picks up newly added columns automatically. Human involvement is reduced to scheduling or triggering the crawler — no custom patterns to maintain and no hand-written DDL. That is why C is the correct answer: built-in classification plus crawler-driven schema discovery yields the least operational overhead while still producing predefined, catalogued column names usable from Athena and Quick Suite.

A common counterargument is that A looks simpler because it is fewer components; but "simplicity of setup" is not "least operational overhead over time" — the ongoing manual updates make A the heavier choice.

Official Reference

https://docs.aws.amazon.com/glue/latest/dg/add-crawler.html

This link opens the official vendor documentation in a new tab.

Exam Strategy

On LEAST/most-overhead questions, ignore setup complexity and count who performs each future change: if a human must re-run, rewrite or edit something every time the data shifts, it loses. Cross out any option that requires writing custom code (grok/XML classifiers) before comparing the remaining automated ones, and prefer the managed, built-in path that a scheduled crawler can repeat unattended.

Frequently Asked Questions

Why does a Glue crawler reduce operational overhead for changing S3 schemas?

The crawler inspects the S3 objects with a built-in classifier, infers column names and types, and updates the Data Catalog table on each run. Re-running it picks up newly added columns without hand-written DDL.

Why are the custom grok and XML classifier options worse here?

Both require writing, testing and maintaining a custom pattern that must be revised every time the file layout changes. An XML classifier also does not fit plain text/CSV-style files.

Can Amazon Athena query the Glue Data Catalog table directly?

Yes. Athena uses the Glue Data Catalog as its metastore, and Amazon Quick Suite (formerly QuickSight) can connect to that same Athena table, so both tools share the crawler-created schema.

More DEA-C01 FAQ →

Community Votes

C
68%
A
23%
B
9%

68% of anonymous learners picked answer C. Votes are pick records left by other test-takers — they are not the verified answer.

Discussion 1 contributions

Compiled from public community threads and reader submissions, lightly edited for clarity. Handles and vote tallies are shown as recorded, and comments reflect the views of individual contributors.

From public community threads 1

  1. Max1948 👍 1 Answered C
    Least Operational Overhead: Using AWS Glue's built-in classifiers automatically detects and infers schemas from text/CSV/JSON files in Amazon S3 without requiring you to manually write, test, or maintain custom grok or XML patterns (eliminating B and D).

Reader comments & corrections

No reader corrections yet — spotted an error or have a note? Share it below.

Log in to comment, report an error, or add a note about this question.

Practice All DEA-C01 Questions

Access 100 questions with complete answers and detailed explanations.

View Full DEA-C01 Practice Test →

← Back to DEA-C01 Study Guide