Modernizing Hadoop and Airflow workloads to Google Cloud
You need to modernize your existing on-premises data strategy. Your organization currently uses: • Apache Hadoop clusters for processing multiple large data sets, including on-premises Hadoop Distributed File System (HDFS) for data replication. • Apache Airflow to orchestrate hundreds of ETL pipelines with thousands of job steps. You need to set up a new architecture in Google Cloud that can handle your Hadoop workloads and requires minimal changes to your existing orchestration processes. What should you do?
Community Votes
100% of anonymous learners picked answer B. Votes are pick records left by other test-takers — they are not the verified answer.
Community Insight
This question tests the identification of direct managed service equivalents for Hadoop and Airflow, with the trap being services like Dataflow or Data Fusion that require significant code refactoring.
To modernize on-premises Hadoop and Airflow on Google Cloud with minimal changes, use Dataproc and Cloud Composer. The community consensus confirms this approach preserves existing orchestration logic while leveraging managed services.
Choosing Option C is a common mistake because while Dataflow is powerful, it requires rewriting ETL pipelines in Apache Beam, violating the requirement for minimal changes.
Community Discussion (6 comments)
Comments & Corrections
No comments yet — spotted an error or have a note? Share it below.
Expert Analysis
Why the Answer Is Correct
Option B is correct because Dataproc is Google Cloud's managed Apache Hadoop and Spark service, designed specifically to migrate existing clusters with minimal code changes. Cloud Storage is used to replace HDFS, offering a highly durable and scalable object store that integrates natively with Dataproc. Cloud Composer is a fully managed workflow orchestration service built on Apache Airflow, allowing the organization to reuse its existing Airflow DAGs with almost no modification.Why the Other Options Are Wrong
Option A is incorrect because Bigtable is a managed NoSQL database and is not a direct replacement for Hadoop processing clusters. Option C is incorrect because converting ETL pipelines to Dataflow requires rewriting the logic in Apache Beam, which contradicts the requirement for minimal changes to orchestration. Option D is incorrect because Cloud Data Fusion is a visual ETL tool that would require a complete redesign of existing Airflow pipelines rather than reusing them.Community Comment Notes
Community comments strongly support Option B, highlighting that Cloud Composer is the managed version of Airflow, ensuring minimal changes to orchestration processes. Users emphasized that Dataproc is the direct equivalent for Hadoop clusters, making it the obvious choice for migration. One comment succinctly summarized the logic as 'Airflow -> Composer' and 'Minimum changes -> Dataproc'.Official Reference
Exam Strategy
When you see keywords like 'minimal changes' or 'minimal refactoring,' look for managed services that are direct equivalents to the open-source tools currently in use. This strategy helps eliminate options that require learning new frameworks or rewriting code.