Migrating HDFS Data Lake to Cloud Storage with Column-Level Security
You migrated your on-premises Apache Hadoop Distributed File System (HDFS) data lake to Cloud Storage. The data scientist team needs to process the data by using Apache Spark and SQL. Security policies need to be enforced at the column level. You need a cost-effective solution that can scale into a data mesh. What should you do?
Community Votes
100% of anonymous learners picked answer B. Votes are pick records left by other test-takers — they are not the verified answer.
Community Insight
The question tests whether you know that BigLake tables can enforce column-level access control via Data Catalog policy tags while keeping data in Cloud Storage — no long-running Dataproc cluster or BigQuery managed storage is needed.
For migrating an on-premises HDFS data lake to Cloud Storage while processing with Spark and SQL and enforcing column-level security, BigLake is the recommended cost-effective solution. Community consensus strongly favors option B, citing its scalability into a data mesh and use of Data Catalog policy tags.
The most common mistake is choosing C (load data to BigQuery), because policy tags can be applied there. However, this copies data out of the data lake, increases storage costs, and weakens the data mesh architecture; BigLake is designed for exactly this scenario.
Community Discussion (5 comments)
- BigLake Integration: BigLake allows you to define tables on top of data in Cloud Storage, providing a bridge between data lake storage and BigQuery's powerful analytics capabilities. This approach is cost-effective and scalable. - Data Catalog for Governance: Creating a taxonomy of policy tags in Google Cloud's Data Catalog and applying these tags to specific columns in your BigLake tables enables fine-grained, column-level access control. - Processing with Spark and SQL: The Spark-BigQuery connector allows data scientists to process data using Apache Spark directly against BigQuery (and BigLake tables). This supports both Spark and SQL processing needs. - Scalability into a Data Mesh: BigLake and Data Catalog are designed to scale and support the data mesh architecture, which involves decentralized data ownership and governance.
Comments & Corrections
No comments yet — spotted an error or have a note? Share it below.
Expert Analysis
Why the Answer Is Correct
Option B is correct because BigLake allows you to define tables on top of data in Cloud Storage without moving the data. This directly bridges the data lake and BigQuery analytics. As comment [1] notes, BigLake integration is cost-effective and scalable, and you can create a taxonomy of policy tags in Data Catalog and apply these tags to specific columns in your BigLake tables, enabling column-level security.Why the Other Options Are Wrong
A is wrong because a long-living Dataproc cluster with Hive and Ranger is expensive and not serverless, and it doesn't integrate as cleanly with BigQuery SQL or Spark. C is wrong because loading data into BigQuery tables moves it out of the data lake, reducing data mesh flexibility and incurring extra storage costs. D is wrong because IAM policies at the file level in Cloud Storage cannot enforce column-level security, which is a row/column granularity requirement.Community Comment Notes
All commenters agree on B. Comment [1] explains the BigLake + Data Catalog approach clearly, and comment [2] emphasizes that BigLake leverages existing Cloud Storage infrastructure, eliminating the need for a dedicated Dataproc cluster and reducing costs. Comment [5] incorrectly suggests loading to BigQuery, but the community consensus and cloud-native best practice favor BigLake.Official Reference
Exam Strategy
Remember that BigLake tables let you query data in Cloud Storage with fine-grained access through Data Catalog policy tags, without copying data or running a permanent cluster. When you see 'cost-effective,' 'column-level security,' and 'data mesh' together, think BigLake first.