Configuring Dataflow for Zonal Resilience

PDE By islever Editorial Last updated

Short answer

A Launch the Dataflow job with the --region=us-central1 parameter to distribute workers across multiple zones for zonal resilience.

Question

You are designing a stateful data processing pipeline that reads data from a Cloud Storage bucket and writes transformed data to a BigQuery table. The pipeline must be highly available and resilient to zonal failures within the us-central1 region. You need to configure a Dataflow pipeline ensuring minimal disruption during a zonal outage. What should you do?
  1. Launch the Dataflow job with the --region=us-central1 parameter. Correct Answer
  2. Deploy the Dataflow job to a single zone within us-central1 and configure it to use a regional persistent disk to store its state.
  3. Deploy the Dataflow job to a single zone within us-central1 and use a multi-regional Cloud Storage bucket to store its state.
  4. Launch the Dataflow job with the --zone=us-central1a parameter.

Key Takeaways

The question tests the understanding that 'region' implies multi-zone distribution for HA, while 'zone' implies a single location vulnerable to outages.

To ensure a Dataflow pipeline is resilient to zonal failures, launch it with the --region flag instead of specifying a single zone. This distributes workers across multiple zones within us-central1.

Common mistake: Choosing B or C because they mention robust storage solutions (regional disks/multi-regional buckets). These address data durability but not compute availability; if workers are in one zone, the pipeline stops during a zonal outage.

Expert Analysis

Regional vs. Zonal Deployment

The core distinction here is between regional and zonal resource configurations in Google Cloud. A region (e.g., us-central1) is a collection of multiple, isolated zones (e.g., us-central1-a, b, c). To achieve high availability and resilience against zonal failures, workloads must be deployed across the entire region rather than confined to a single zone.

Dataflow Regionality

When launching a Dataflow pipeline, specifying --region=us-central1 instructs the service to deploy worker instances across all available zones within that region. This ensures that if one zone experiences an outage, the pipeline can continue processing using workers in other zones within the same region, minimizing disruption. In contrast, options B, C, and D explicitly limit the deployment to a single zone (us-central1a or similar). While state storage mechanisms like regional persistent disks or multi-regional buckets are robust, they do not mitigate the risk of a single-zone compute failure. If the workers themselves are confined to one zone, a zonal outage halts the pipeline regardless of where the state is stored.

Conclusion

Option A is the only configuration that distributes compute resources across multiple zones, satisfying the requirement for zonal failure resilience. Options B, C, and D create a single point of failure at the zone level.

Exam Strategy

Always distinguish between 'region' and 'zone' parameters in cloud architecture questions. If the requirement is high availability or fault tolerance against infrastructure failures, look for options that distribute resources across multiple zones (typically via a region setting). Single-zone deployments are inherently less resilient unless specifically required for latency or cost reasons.

Frequently Asked Questions

Why doesn't storing state in a multi-regional bucket help with zonal outages?

State storage handles data persistence, but compute workers handle the processing. If workers are confined to a single zone, a zonal outage prevents them from running, even if the state is safely stored elsewhere.

What is the difference between --region and --zone in Dataflow?

--region deploys workers across all available zones in that region for high availability. --zone restricts workers to a specific single zone, creating a potential single point of failure.

More PDE FAQ →

Community Votes

A
100%

100% of anonymous learners picked answer A. Votes are pick records left by other test-takers — they are not the verified answer.

Discussion 3 contributions

Compiled from public community threads and reader submissions, lightly edited for clarity. Handles and vote tallies are shown as recorded, and comments reflect the views of individual contributors.

From public community threads 3

  1. Mohamed_Tareq 👍 1 Answered A
    A is the right option since it is the only one introduce the 'region' option and "Region = Multiple zones" B, C, &D all are wrong since they introduce a single zone solution which is might fail during a zonal outage as in the question
  2. hola_prep 👍 1 Answered A
    To ensure high availability and resilience against zonal failures with minimal disruption, you should deploy the Dataflow job regionally using --region=us-central1 (Option A).
  3. AwesomeGCP 👍 1 Answered A
    The correct answer is A. Launch the Dataflow job with the --region=us-central1 parameter.

Reader comments & corrections

No reader corrections yet — spotted an error or have a note? Share it below.

Log in to comment, report an error, or add a note about this question.

← Back to PDE Study Guide