Athena Query Performance with AWS Glue Crawler Exclusions

DEA-C01 By islever Editorial Last updated

Analyze data by using AWS services.

Short answer

C Relocate the .json files to a different path within the S3 bucket.

Question

A data engineer is using an AWS Glue crawler to catalog data that is in an Amazon S3 bucket. The S3 bucket contains both .csv and json files. The data engineer configured the crawler to exclude the .json files from the catalog. When the data engineer runs queries in Amazon Athena, the queries also process the excluded .json files. The data engineer wants to resolve this issue. The data engineer needs a solution that will not affect access requirements for the .csv files in the source S3 bucket. Which solution will meet this requirement with the SHORTEST query times?
  1. Adjust the AWS Glue crawler settings to ensure that the AWS Glue crawler also excludes .json files.
  2. Use the Athena console to ensure the Athena queries also exclude the .json files.
  3. Relocate the .json files to a different path within the S3 bucket. Correct Answer
  4. Use S3 bucket policies to block access to the .json files.

Key Takeaways

The core concept is that Athena ignores Glue crawler exclusion patterns, making physical relocation of files the only effective optimization method.

This page explains how to resolve discrepancies between AWS Glue crawler exclusions and Amazon Athena query scans to achieve the shortest query times for mixed-format S3 data.

Common mistake: Most candidates incorrectly assume that adjusting AWS Glue crawler settings or modifying Athena query logic will prevent the scanning of excluded file types.

Expert Analysis

Why the Answer Is Correct

Amazon Athena queries the underlying Amazon S3 data directly and does not respect the exclusion patterns configured in an AWS Glue crawler; it processes all files matching the table's location. Relocating the.json files to a different S3 path effectively removes them from the table's scope, ensuring Athena only scans the desired.csv files. This approach minimizes I/O overhead and results in the shortest possible query execution time.

Why the Other Options Are Wrong

Adjusting the AWS Glue crawler settings (Option A) only affects metadata registration in the Data Catalog, not the actual data scanned by Athena. Modifying Athena queries (Option B) to filter out specific file types is inefficient because Athena must still read and parse the entire dataset before applying filters. Using S3 bucket policies (Option D) would block access entirely, which violates the requirement to maintain access to the.csv files.

Community Comment Notes

As noted by user teo2157, "Athena does not recognize exclude patterns that you specify an AWS Glue crawler," confirming that physical separation is required. Another commenter highlighted that while query adjustments are feasible, relocating files yields the best performance, aligning with the exam's focus on efficiency.

Official Reference

https://docs.aws.amazon.com/athena/latest/ug/troubleshooting-athena-crawler-exclude-patterns.html

This link opens the official vendor documentation in a new tab.

Exam Strategy

Always prioritize data locality when optimizing query performance in Athena. If metadata configurations fail to restrict data processing, physically separate the unwanted data from the source path.

Frequently Asked Questions

Does AWS Glue crawler exclusion affect Amazon Athena?

No. Athena scans all files in the table location regardless of Glue crawler exclusion patterns.

Can I use SQL to ignore .json files in Athena?

You can filter rows via SQL, but Athena still reads the files, causing higher latency than relocating them.

More DEA-C01 FAQ →

Community Votes

C
100%

100% of anonymous learners picked answer C. Votes are pick records left by other test-takers — they are not the verified answer.

Discussion 3 contributions

Compiled from public community threads and reader submissions, lightly edited for clarity. Handles and vote tallies are shown as recorded, and comments reflect the views of individual contributors.

From public community threads 3

  1. teo2157 👍 8 Answered C
    Athena does not recognize exclude patterns that you specify an AWS Glue crawler. For example, if you have an Amazon S3 bucket that contains both .csv and .json files and you exclude the .json files from the crawler, Athena queries both groups of files. To avoid this, place the files that you want to exclude in a different location. https://docs.aws.amazon.com/athena/latest/ug/troubleshooting-athena.html
  2. AdityaB 👍 1
    If the AWS Glue crawler is configured to exclude .json files, then the AWS Glue Data Catalog will not have any metadata related to those .json files. In this case, the Athena table that uses the Glue Data Catalog would not be aware of the .json files at all, and Athena queries would only process the files that are included in the Glue catalog (e.g., .csv files).
  3. BenLearningDE 👍 1
    Athena will scan both types of files. Although it may be feasible to adjust Athena query to exclude .json, the SHORTEST query times would be via relocating .json files to different path.

Reader comments & corrections

No reader corrections yet — spotted an error or have a note? Share it below.

Log in to comment, report an error, or add a note about this question.

Amazon Redshift Compound Sort Key Query Optimization

S-grade

Secure Redshift Spectrum Access with VPC Endpoints

S-grade

Athena Federated Query for Multi-Source Iceberg Pipelines

S-grade

Reclaiming Storage in Amazon Redshift Materialized Views

S-grade

Practice All DEA-C01 Questions

Access 100 questions with complete answers and detailed explanations.

View Full DEA-C01 Practice Test →

← Back to DEA-C01 Study Guide