How to Run Dataflow with Internal-IP-Only Workers Accessing Cloud Storage and BigQuery?

You are deploying a batch pipeline in Dataflow. This pipeline reads data from Cloud Storage, transforms the data, and then writes the data into BigQuery. The security team has enabled an organizational constraint in Google Cloud, requiring all Compute Engine instances to use only internal IP addresses and no external IP addresses. What should you do?

Ensure that your workers have network tags to access Cloud Storage and BigQuery. Use Dataflow with only internal IP addresses.
Ensure that the firewall rules allow access to Cloud Storage and BigQuery. Use Dataflow with only internal IPs.
Create a VPC Service Controls perimeter that contains the VPC network and add Dataflow, Cloud Storage, and BigQuery as allowed services in the perimeter. Use Dataflow with only internal IP addresses.
Ensure that Private Google Access is enabled in the subnetwork. Use Dataflow with only internal IP addresses. Source Reference Answer

Community Votes

73%

27%

73% of anonymous learners picked answer D. Votes are pick records left by other test-takers — they are not the verified answer.

Community Insight

This question tests your understanding of how Dataflow workers with internal IPs access Google services: Private Google Access is required, and the trap is mistaking VPC Service Controls (a data-security perimeter) for a connectivity solution.

To run a Dataflow pipeline with Compute Engine workers that have no external IP addresses, you must enable Private Google Access on the subnetwork so workers can reach Google APIs like Cloud Storage and BigQuery. The community consensus is that option D is correct, while VPC Service Controls is a security boundary, not a connectivity enabler.

Choosing C (VPC Service Controls) is the most common mistake. Test takers see 'organizational constraint' and assume VPC SC is needed, but VPC SC does not provide network connectivity for internal-only VMs; it only restricts and monitors service access from defined perimeters.

Community Discussion (8 comments)

raaad 👍 5 Selected: D

Private Google Access for services allows VM instances with only internal IP addresses in a VPC network or on-premises networks (via Cloud VPN or Cloud Interconnect) to reach Google APIs and services. - When you launch a Dataflow job, you can specify that it should use worker instances without external IP addresses if Private Google Access is enabled on the subnetwork where these instances are launched. - This way, your Dataflow workers will be able to access Cloud Storage and BigQuery without violating the organizational constraint of no external IPs.

Lestrang 👍 1 Selected: D

No way it is C. Like the use case for Google VPC Service Controls perimeter is not to establish secure connectivity on its own but rather to control connectivity, like allowing vms within x premise to access, and blocking vms outside premise even if in same VPC from access. D on the other hand is completely sensical.

Moss2011 👍 1 Selected: C

According to this documentation: https://cloud.google.com/vpc-service-controls/docs/overview I think the correct answer is C. Take into account the phrase "organizational constraint" and the VPC Service Control allow you to do that.

Tryolabs 👍 1 Selected: D

https://cloud.google.com/vpc/docs/private-google-access "VM instances that only have internal IP addresses (no external IP addresses) can use Private Google Access. They can reach the external IP addresses of Google APIs and services."

pandeyspecial 👍 1 Selected: C

It should be C

Matt_108 👍 1 Selected: C

Option D, as GCP001 said

GCP001 👍 4 Selected: D

https://cloud.google.com/dataflow/docs/guides/routes-firewall

scaenruy 👍 1 Selected: C

C. Create a VPC Service Controls perimeter that contains the VPC network and add Dataflow, Cloud Storage, and BigQuery as allowed services in the perimeter. Use Dataflow with only internal IP addresses.

Comments & Corrections

No comments yet — spotted an error or have a note? Share it below.

Expert Analysis

Why the Answer Is Correct

Option D is correct because Dataflow workers are Compute Engine instances, and when they have only internal IP addresses they cannot reach Google APIs through the public internet. Private Google Access enables VMs in a VPC subnetwork to reach Google APIs and services (like Cloud Storage and BigQuery) using their internal IP addresses, without needing an external IP. This is a documented requirement for running Dataflow jobs with internal IPs only.

Why the Other Options Are Wrong

Option A is wrong because network tags control firewall rules and routing, but they do not by themselves enable access to Google APIs for internal-only VMs. Option B is also wrong because firewall rules alone are insufficient; without Private Google Access, traffic to Google API external IPs from internal-only VMs will still be blocked. Option C is incorrect because VPC Service Controls create a security perimeter around Google Cloud services, but they do not provide the underlying network path needed by internal-only Dataflow workers; Private Google Access is still required.

Community Comment Notes

Several comments correctly cited Private Google Access documentation, noting that VMs with only internal IP addresses can reach Google APIs and services when PGA is enabled. One comment from "GCP001" explicitly rejected option C, explaining that VPC SC is meant to control connectivity between locations, not to establish secure connectivity on its own. Another comment pointed to the Dataflow routes/firewall guide, which reinforces the need for proper networking configuration, including PGA, when using internal IPs.

Official Reference

Exam Strategy

When a question says Dataflow workers must have no external IP addresses, immediately look for an option that enables Private Google Access on the subnetwork. Remember that VPC Service Controls is a red herring in networking-connectivity questions—it is about security boundaries, not about giving VMs a path to Google APIs.

Related Analysis

← Back to PDE Study Guide