Cost-Effective Dataproc Job Over Regional GCS Telemetry

PCA Last updated

Designing network, storage, and compute resources. Configuring individual storage systems. Configuring compute systems.

Short answer

D Preprocess and compress the raw telemetry in each region, move it into a regional GCS bucket, and finish the analysis with a single regional Cloud Dataproc cluster.

Question

TerramEarth's 20 million vehicles are scattered around the world. Based on the vehicle's location, its telemetry data is stored in a Google Cloud Storage (GCS) regional bucket (US, Europe, or Asia). The CTO has asked you to run a report on the raw telemetry data to determine why vehicles are breaking down after 100 K miles. You want to run this job on all the data. What is the most cost-effective way to run this job?

  1. Move all the data into 1 zone, then launch a Cloud Dataproc cluster to run the job
  2. Move all the data into 1 region, then launch a Google Cloud Dataproc cluster to run the job
  3. Launch a cluster in each region to preprocess and compress the raw data, then move the data into a multi-region bucket and use a Dataproc cluster to finish the job
  4. Launch a cluster in each region to preprocess and compress the raw data, then move the data into a region bucket and use a Cloud Dataproc cluster to finish the job Correct Answer

Key Takeaways

The question tests how data gravity and egress pricing drive processing cost in Google Cloud; the trap is choosing to 'move all the data' (A/B) or to consolidate into a multi-region bucket (C), ignoring that intercontinental transfer and multi-region storage are the dominant costs.

TerramEarth stores vehicle telemetry in US, Europe, and Asia regional GCS buckets, and the cheapest way to analyze it all is to preprocess and compress the raw data locally in each region before centralizing it. Option D is correct because it minimizes costly cross-continent egress and keeps a single regional Cloud Dataproc cluster doing the heavy final processing.

Common mistake: The most common wrong choice is B (or A), which copies the entire uncompressed raw dataset across continents. Learners assume one cluster and one bucket is automatically cheaper, but bulk raw egress at roughly $0.12 per GB dwarfs the savings, and a Cloud Dataproc cluster cannot even be launched in a single GCS 'zone' as option A implies.

Expert Analysis

Why the Answer Is Correct

Option D preprocesses and compresses the telemetry inside each region, where VM-to-storage and in-region reads are free, so only the far smaller compressed output crosses continents at intercontinental egress rates. It then consolidates into a regional bucket and runs one Cloud Dataproc cluster to finish the job, and Dataproc clusters are regional resources, so a regional bucket is a natural fit. As jespinosar calculated, regional storage (~USD 200 for 10 TB/month) is materially cheaper than multi-regional (~USD 260), and MaxNRG correctly emphasized that egress rates—$0.12/GB intercontinental versus $0.01/GB cross-region and free in-region—are the dominant cost driver.

Why the Other Options Are Wrong

Options A and B require moving all raw data before any filtering, which maximizes cross-continent egress on the largest possible volume; as cetanx noted, vehicles under 100K miles are irrelevant to the breakdown analysis, so transferring everything is pure waste. Option A is additionally invalid because Cloud Storage has no 'zone' bucket class in this context, and a zonal Dataproc cluster cannot read regional/multi-region buckets efficiently. Option C still moves the full uncompressed footprint out to a multi-region bucket, paying both higher multi-region storage pricing and the same egress, and Dataproc cannot run inside a multi-region anyway—the cluster still has to live in a specific region.

Community Comment Notes

Most commenters converged on D, with tartar simply stating "D is ok" and MamthaSJ declaring "Answer is D". JoeShmoe captured the technical anchor that "D is the most cost effective and DataProc is regional", while Zarmi warned that "Moving data between region it's traffic, and it's not cheap." The dissenting view came from vindahake, who argued that extra regional compute would exceed the data-transfer savings, and Jos, who believed centralizing first and deploying only one cluster is cheaper—both underestimate the size of raw telemetry egress relative to VM-hours.

Official Reference

Exam Strategy

On PCA cost questions, identify the dominant cost driver before choosing: here it is cross-continent egress on raw telemetry, so the winning pattern is compress-in-place locally, then move the reduced dataset to one region. Also remember that Cloud Dataproc clusters are regional resources and cannot be created in a multi-region or zone.

Frequently Asked Questions

Why can't I just move all the data into one region and run one Dataproc cluster (option B)?

Because moving the full uncompressed raw dataset intercontinentally costs about $0.12 per GB, which dominates the total bill. Compressing locally first shrinks the transfer volume dramatically before any cross-region copy.

Why is a multi-region bucket (option C) more expensive than a regional bucket here?

Multi-region storage costs roughly 30% more per GB-month than regional storage (about USD 260 vs USD 200 for 10 TB/month), and Dataproc cannot run inside a multi-region anyway—the cluster must live in a specific region.

Is Cloud Dataproc a zonal or regional service?

Cloud Dataproc clusters are regional resources; you choose a region (and optional zone placement for nodes) when you create the cluster, so a regional GCS bucket is the natural co-located storage target.

More PCA FAQ →

Discussion 66 contributions

Compiled from public community threads and reader submissions, lightly edited for clarity. Handles and vote tallies are shown as recorded.

From public community threads 30 of 66

  1. cetanx 👍 68
    I will look at it from a different perspective; A, B says "move all data" but analysis will try to reveal breaking down after 100K miles so there is no point of transferring data of the vehicles with less than 100K milage. Therefore, transferring all data is just waste of time and money. There is one thing for sure here. If we move/copy data between continents it will cost us money therefore compressing the data before copying to another region/continent makes sense. Preprocessing also makes sense because we probably want to process smaller chunks of data first (remember 100K milage). So now type of target bucket; multi-region or standard? multi-region is good for high-availability and low latency with a little more cost however question doesn't require any of these features. Therefore I think standard storage option is good to go given lower costs are always better. So my answer would be D
  2. JoeShmoe 👍 32
    D is the most cost effective and DataProc is regional
  3. tartar 👍 11
    D is ok
  4. MaxNRG 👍 6
    D – Launch a cluster in each region to pre-process and compress the raw data, then move the data into a regional bucket and use Cloud Dataproc cluster. Egress rates are most important. It is free inside of region - so make sense to move all data into one region for processing/performance (from all continents). Cross-region cost is 0.01$ per GB, and inter-continent 0.12$ per GB. If to consider just option B (moving all raw data into one region) then just monthly volume would cost: 900 TB (all 20M units daily) 30 days 0.12 $ = 3.24 M $ (just for data transfer). So, it definitely makes sense to preprocess/compress data per region, and then move all that data into one region for final analysis. That would save up to 10-100 times on egress costs. Also, important aspect is processing time - running it in parallel on all regions accelerates overall analysis effort. Faster result - faster in-field improvements. Look this interesting video about price optimization in GCP (first 11.5 mins are about Storage/Network) https://cloud.google.com/storage/docs/locations#considerations
  5. Jos 👍 6
    I think that most cost-effective way to do it is moving all to ONE storage location so that only ONE cluster is deployed (cluster cost money, moving data between buckets dont)
Show 25 more comments
  1. Rafaa 👍 5
    I agree with you on removing C,D. I think it should be B because there is no 'Zone' as such with respect to GCS right?
  2. jespinosar 👍 4
    D) Is the right option because it is the most cost effective. If you use the pricing calculator, 10 TB/Month of Regional is ~ USD 200 and Multi-Regional ~USD 260 (US vs us-central1) Because we used the regional map-reduce aproach, we can use the region with most data to move the other regions' data to that region and then launch DataProc regional. The cost of moving to a regional or multi-regional would be the same: if continents are crossed, you pay egress data. To sum up: D) is cheaper, so D) is the right answer Please notice that should the question state "the most reliable" or "the less latency", then multi-regional should be used.
  3. vindahake 👍 4
    I think running additional compute regionally will be more expensive than data transfer charges and centrally processing them
  4. Zarmi 👍 4
    Moving data between region it's traffic, and it's not cheap. I think D.
  5. bnlcnd 👍 3
    Although many people think D is right. I still think C make some sense too. 1) DataProc can be global. 2) global DP should be able to handle mutil regional buket better. In theory, it shoul avoid transferring data cross regions. It actually distribute the computation across regions. This will reduce the cost a lot. Remember, the data load and analytic need to be done every day.
  6. PRC 👍 3
    Question says - "run a report on the raw telemetry data". Option C & D are pre-processing the data which violates the principles laid out in the question - How about A? Move data into one zone and then do the needful.
  7. MamthaSJ 👍 3
    Answer is D
  8. KNG 👍 2
    I think it is D When used in a region, Standard Storage is appropriate for storing data in the same location as Google Kubernetes Engine clusters or Compute Engine instances that use the data. Co-locating your resources maximizes the performance for data-intensive computations and can reduce network charges. When used in a dual-region, you still get optimized performance when accessing Google Cloud products that are located in one of the associated regions, but you also get the improved availability that comes from storing data in geographically separate locations. When used in a multi-region, Standard Storage is appropriate for storing data that is accessed around the world, such as serving website content, streaming videos, executing interactive workloads, or serving data supporting mobile and gaming applications.
  9. omermahgoub 👍 2
    Answer is C To run the report on all of the raw telemetry data for TerramEarth's vehicles in the most cost-effective way, it would be best to launch a cluster in each region to preprocess and compress the raw data. This will allow you to process the data in place, which will minimize the amount of data that needs to be transferred between regions. After the data has been preprocessed and compressed, you can then move it into a multi-region bucket and use a Dataproc cluster to finish the job.
  10. VedaSW 👍 2
    The CTO is not solution architect, so "run a raw data" is a solution in addressing a need/requirement. As a solution architect, I would focus on the objective: "finding out why vehicles are breaking down?" So long the solution can achieve the result and better than the "suggested" solution, I will likely go for the better solution.
  11. msahdra 👍 2 Answered C
    While regional preprocessing can be efficient, moving the data back to regional buckets after compression defeats the purpose of a multi-region bucket. It adds unnecessary data transfer costs and reduces the availability of the preprocessed data for global analysis.
  12. HKim 👍 2
    I think C is more cost effective than D since moving from regional buckets to a multi-region bucket in the same continent cost nothing while moving from regional to regional in the same continent incurs cost. Cross continent data moves will incur charges regardless.
  13. DiegoQ 👍 2
    I totally agree with you, and I think that what confuse people here is the "run a raw data", but preprocess doesn´t mean to mandatory transform raw data, it could be to only select the data that you need (as you said: vehicles with less than 100K milage)
  14. AdityaGupta 👍 2
    There is nothing called zonal in GCS, ( I agree with Rafaa) and moving data from a multi-regional bucket to a regional bucket incurs egress charges at a rate of $0.01/GB. I will go with answer B.
  15. saurabh1805 👍 2
    in question CTO has asked report to be run on raw data hence C and D can never be correct answer. For me B seems to be correct answer as that is what i am using in my current project.
  16. vincy2202 👍 2 Answered D
    D is the correct answer. Regional bucket is required, since multi regional bucket will incur additional cost to transfer the data to a centralized location.
  17. Here4cloud 👍 2
    Actually read somewhere that ‘Input into Dataproc or compute processing’ is a use case for regional storage class..so looks like D is correct.
  18. AzureDP900 👍 2
    D is fine, There is no need of multi-region as mentioned in C. D is right in my opinion.
  19. RomiAwasthy 👍 2
    C as all the data across regions need to be analyzed, so would need multi region cluster
  20. passnow 👍 2
    Honestly, if we read the question well and factor in cost, D would be a better option
  21. Mimgq 👍 2
    D would be more cost-effective. You can't afford to move that large of a dataset.
  22. thewalker 👍 2
    D Considering https://cloud.google.com/storage/docs/locations#considerations
  23. Rafaa 👍 2
    Hold on guys, you do not need to 'preprocess' the data. This rules out C,D.
  24. AMohanty 👍 2
    What is the use of Multi-Regional DataProc if ur Storage Data is Regional
  25. Mimgq 👍 2
    This is B, there is nothing that says they pre-process the 120 fields

Reader comments & corrections

No reader corrections yet — spotted an error or have a note? Share it below.

Log in to comment, report an error, or add a note about this question.

Submitted for moderation before publishing. Keep it helpful and respectful.

Optimizing Data Transfer for ETL with Cellular Connections

S-Grade

Streamlining Cloud IAM Audit Analysis with BigQuery

S-Grade

Auditing BigQuery User Query Counts

S-Grade

Practice All PCA Questions

Access 81 questions with complete answers and detailed explanations.

View Full PCA Practice Test →

← Back to PCA Study Guide