Migrating Hive Metastore to Serverless AWS Glue Data Catalog

DEA-C01 By islever Editorial Last updated

Understand data cataloging systems. Automate data processing by using AWS services.

Short answer

B Configure a Hive metastore in Amazon EMR and migrate the existing on-premises Hive metastore into Amazon EMR, then use AWS Glue Data Catalog to store the company's data catalog as an external data catalog.

Question

A company is planning to migrate on-premises Apache Hadoop clusters to Amazon EMR. The company also needs to migrate a data catalog into a persistent storage solution. The company currently stores the data catalog in an on-premises Apache Hive metastore on the Hadoop clusters. The company requires a serverless solution to migrate the data catalog. Which solution will meet these requirements MOST cost-effectively?
  1. Use AWS Database Migration Service (AWS DMS) to migrate the Hive metastore into Amazon S3. Configure AWS Glue Data Catalog to scan Amazon S3 to produce the data catalog.
  2. Configure a Hive metastore in Amazon EMR. Migrate the existing on-premises Hive metastore into Amazon EMR. Use AWS Glue Data Catalog to store the company's data catalog as an external data catalog. Correct Answer
  3. Configure an external Hive metastore in Amazon EMR. Migrate the existing on-premises Hive metastore into Amazon EMR. Use Amazon Aurora MySQL to store the company's data catalog.
  4. Configure a new Hive metastore in Amazon EMR. Migrate the existing on-premises Hive metastore into Amazon EMR. Use the new metastore as the company's data catalog.

Key Takeaways

The exam tests knowledge of AWS managed metadata services, specifically identifying AWS Glue as the serverless alternative to self-managed Hive metastores while retaining EMR compatibility.

This question addresses migrating an on-premises Apache Hive metastore to a serverless, cost-effective AWS solution for data cataloging. It establishes that using AWS Glue Data Catalog as an external catalog with EMR is the optimal architecture.

Common mistake: Candidates often choose Option A because AWS DMS is a well-known migration tool, but it is not designed for migrating Hive metastore metadata structures to S3 for cataloging purposes in this context.

Expert Analysis

Why the Answer Is Correct

Option B is correct because it leverages Amazon EMR to run Hadoop workloads while offloading the metadata management to AWS Glue Data Catalog. AWS Glue is a fully serverless service, which meets the 'serverless' requirement perfectly and reduces operational overhead compared to managing a database like Aurora (Option C) or maintaining a local metastore in EMR (Option D). By configuring Glue as an external data catalog, EMR can read/write metadata without being tightly coupled to a specific metastore implementation, providing flexibility and persistence.

Why the Other Options Are Wrong

Option A suggests migrating to S3 via DMS, which is incorrect because DMS migrates structured data, not complex metadata schemas, and S3 is not a metadata store. Option C uses Amazon Aurora MySQL, which is a managed relational database, not a serverless solution by default in this context (though Aurora Serverless exists, Glue is the native serverless catalog). Option D keeps the metastore within EMR, which requires managing EC2 instances and software updates, violating the 'serverless' and 'cost-effective' requirements for the catalog itself.

Community Comment Notes

Community consensus strongly favors Option B. Users noted that AWS Glue is synonymous with 'serverless catalog' in AWS exams. One commenter highlighted that Glue allows seamless migration from on-premises Hive metastores by acting as an external catalog, ensuring compatibility with existing Hadoop setups while eliminating infrastructure management.

Official Reference

https://aws.amazon.com/blogs/big-data/migrate-and-deploy-your-apache-hive-metastore-on-amazon-emr/

This link opens the official vendor documentation in a new tab.

Exam Strategy

When a question asks for a 'serverless' metadata or catalog solution in the AWS Big Data space, immediately look for AWS Glue Data Catalog. If the workload is Hadoop/EMR-based, remember that EMR can use Glue as an external catalog, decoupling compute from metadata storage.

Frequently Asked Questions

Why is AWS DMS not used for Hive metastore migration?

DMS migrates structured data tables, not the complex schema and metadata definitions stored in a Hive metastore database.

Can Amazon Aurora be considered serverless here?

While Aurora Serverless exists, AWS Glue is the dedicated, fully managed serverless metadata catalog service, making it more appropriate and cost-effective for this specific use case.

More DEA-C01 FAQ →

Community Votes

B
82%
A
18%

82% of anonymous learners picked answer B. Votes are pick records left by other test-takers — they are not the verified answer.

Discussion 9 contributions

Compiled from public community threads and reader submissions, lightly edited for clarity. Handles and vote tallies are shown as recorded, and comments reflect the views of individual contributors.

From public community threads 9

  1. rralucard_ 👍 3 Answered B
    https://aws.amazon.com/blogs/big-data/migrate-and-deploy-your-apache-hive-metastore-on-amazon-emr/ Option B is likely the most suitable. Migrating the Hive metastore into Amazon EMR and using AWS Glue Data Catalog as an external catalog provides a balance between leveraging the scalable and managed services of AWS (like EMR and Glue Data Catalog) and ensuring a smooth transition from the on-premises setup. This approach leverages the serverless nature of AWS Glue Data Catalog, minimizing operational overhead and potentially reducing costs compared to managing database servers.
  2. Christina666 👍 2 Answered B
    Serverless and Cost-Efficient: AWS Glue Data Catalog offers a serverless metadata repository, reducing operational overhead and making it cost-effective. Using it as an external data catalog means you don't have to manage additional database infrastructure. Seamless Migration: Migrating your existing Hive metastore to Amazon EMR ensures compatibility with your current Hadoop setup. EMR is designed to run Hadoop workloads, facilitating this process. Flexibility: An external data catalog in AWS Glue offers flexibility and separation of concerns. Your metastore remains managed by EMR for your Hadoop workloads, while Glue provides a centralized catalog for broader AWS data sources.
  3. lucas_rfsb 👍 2 Answered A
    I will go with A. Besides DMS is typical for migration, it's the only choice which explicitly concerns about how the migration itself will be made. Other choices would demand a script or GLUE ETL job if you will. But this logic of migration was never put
  4. arvehisa 👍 2 Answered B
    B. https://aws.amazon.com/jp/blogs/big-data/migrate-and-deploy-your-apache-hive-metastore-on-amazon-emr/
  5. damaldon 👍 1
    B. Set up an AWS Glue ETL job which extracts metadata from your Hive metastore (MySQL) and loads it into your AWS Glue Data Catalog. This method requires an AWS Glue connection to the Hive metastore as a JDBC source. An ETL script is provided to extract metadata from the Hive metastore and write it to AWS Glue Data Catalog. https://github.com/aws-samples/aws-glue-samples/blob/master/utilities/Hive_metastore_migration/README.md
Show 4 more comments
  1. Asmunk 👍 1 Answered B
    A and D can be discarded because of added steps. This link provides documentation for this exact use case : https://aws.amazon.com/blogs/big-data/migrate-and-deploy-your-apache-hive-metastore-on-amazon-emr/ C is also discarded because of the serverless key word, although Aurora can be serverless it is not specified in the choice.
  2. nyaopoko 👍 1
    B is answer! By leveraging AWS Glue Data Catalog as an external data catalog and migrating the existing Hive metastore into Amazon EMR, the company can achieve a serverless, persistent, and cost-effective solution for storing and managing their data catalog.
  3. jpmadan 👍 1 Answered B
    serverless catalog in AWS == glue
  4. LeoSantos121212121212121 👍 2
    I will go with A

Reader comments & corrections

No reader corrections yet — spotted an error or have a note? Share it below.

Log in to comment, report an error, or add a note about this question.

Amazon Redshift Compound Sort Key Query Optimization

S-grade

Secure Redshift Spectrum Access with VPC Endpoints

S-grade

Athena Federated Query for Multi-Source Iceberg Pipelines

S-grade

Reclaiming Storage in Amazon Redshift Materialized Views

S-grade

Practice All DEA-C01 Questions

Access 100 questions with complete answers and detailed explanations.

View Full DEA-C01 Practice Test →

← Back to DEA-C01 Study Guide