Running HiveQL on Cloud Storage Data with Minimal Overhead
Short answer
D Deploy a Dataproc cluster with Hive services enabled to provide a managed environment for executing existing HiveQL queries directly against Parquet files in Cloud Storage.
Question
Key Takeaways
The main trap is choosing BigQuery for its low overhead, but it fails because it does not natively support the specific HiveQL syntax required by the team's existing queries.
This question explores how to run interactive HiveQL queries on Parquet data in Cloud Storage with minimal operational overhead. The correct solution involves deploying a managed Dataproc cluster with Hive services enabled.
Common mistake: Many candidates choose C (BigQuery External Tables) because it offers the lowest operational overhead and fast interactive SQL. However, this is incorrect because BigQuery does not execute HiveQL; it executes Standard SQL, making it incompatible with the team's existing HiveQL scripts.
Expert Analysis
The core requirement is to run interactive SQL queries using HiveQL syntax against data stored in Cloud Storage (Parquet) while minimizing operational overhead.
Option C, configuring BigQuery with an external table definition, allows interactive SQL queries on Parquet files in Cloud Storage with zero operational overhead. However, BigQuery uses standard SQL or legacy SQL, not HiveQL. While there are some syntactic overlaps, a team "familiar with Apache Hive" and wanting to leverage "existing HiveQL queries" cannot simply point BigQuery at the data and expect their existing scripts to work without modification. BigQuery does not support the full HiveQL dialect out-of-the-box for direct execution.
Option D, deploying a Dataproc cluster with Hive services enabled, directly addresses the need for HiveQL compatibility. Dataproc is a managed Hadoop service that reduces operational overhead compared to manually installing Hadoop/Hive on Compute Engine instances (Option A). By enabling the Hive service on Dataproc, you can use the Hive Metastore and HiveServer2 to submit HiveQL queries. To query Cloud Storage directly, you would typically configure the Hive metastore to reference the Cloud Storage bucket or use the gs:// scheme in Hive queries if configured correctly (e.g., via the Hadoop GCS connector). This setup provides the exact environment requested: HiveQL execution capability over Cloud Storage data.
While Option B mentions loading data into BigQuery and using a connector, it introduces significant operational overhead (data movement/loading) and still faces the dialect mismatch issue unless specifically bridged, which is less direct than running Hive on Dataproc. The key trade-off here is between the absolute lowest overhead of BigQuery (which lacks HiveQL support) and the managed Hive environment of Dataproc (which supports HiveQL but requires cluster management). Given the explicit constraint of leveraging existing HiveQL queries, Dataproc is the correct architectural choice.
Exam Strategy
When a question specifies a particular query language or engine (like HiveQL, Pig, or Spark SQL), prioritize the managed service that natively supports that technology over general-purpose analytics platforms. Do not let the 'minimal operational overhead' constraint lure you into choosing a platform (like BigQuery) that requires rewriting queries. Look for managed versions of the specific open-source tools mentioned.
Frequently Asked Questions
Can I run HiveQL queries directly in BigQuery?
Is Dataproc considered low operational overhead?
Community Votes
100% of anonymous learners picked answer D. Votes are pick records left by other test-takers — they are not the verified answer.
Discussion 2 contributions
Compiled from public community threads and reader submissions, lightly edited for clarity. Handles and vote tallies are shown as recorded, and comments reflect the views of individual contributors.
From public community threads 2
-
hola_prep 👍 2 Answered DHiveQL compatibility Direct access to Parquet in Cloud Storage: Interactive queries: Low operational overhead: Dataproc is a managed service. You don’t need to manually install, configure, or maintain Hadoop/Hive clusters Scalability: Dataproc clusters can scale up or down as needed
-
AwesomeGCP 👍 1 Answered DThe correct answer is D. Deploy a Dataproc cluster with Hive services enabled.
Reader comments & corrections
No reader corrections yet — spotted an error or have a note? Share it below.