Migrating Petabyte-Scale Data with Transfer Appliance

PDE By islever Editorial Last updated

Designing data migrations.

Short answer

B Use a Transfer Appliance to physically move 1 PB of data quickly and securely, as network transfers cannot meet the 'few hours' constraint.

Question

You need to migrate 1 PB of data from an on-premises data center to Google Cloud. Data transfer time during the migration should take only a few hours. You
want to follow Google-recommended practices to facilitate the large data transfer over a secure connection. What should you do?
  1. Establish a Cloud Interconnect connection between the on-premises data center and Google Cloud, and then use the Storage Transfer Service.
  2. Use a Transfer Appliance and have engineers manually encrypt, decrypt, and verify the data. Correct Answer
  3. Establish a Cloud VPN connection, start gcloud compute scp jobs in parallel, and run checksums to verify the data.
  4. Reduce the data into 3 TB batches, transfer the data using gsutil, and run checksums to verify the data.

Key Takeaways

The trap is assuming network interconnects can move petabytes in hours; the reality is physical appliances are required for such volumes to meet tight timeframes.

Learn why Transfer Appliance is the correct choice for migrating 1 PB of data in hours, outperforming Cloud Interconnect and VPN options for large-scale secure migrations.

Common mistake: Choosing Cloud Interconnect (A) because it seems 'high-speed', ignoring that even 100Gbps takes ~30 hours to move 1 PB, violating the 'few hours' constraint.

Expert Analysis

The question presents a critical constraint: migrating 1 PB of data must take only 'a few hours' over a secure connection. While Cloud Interconnect (Option A) provides high bandwidth, it is fundamentally limited by the speed of light and physical cabling. Even with dedicated 100 Gbps connections, transferring 1 PB takes approximately 27–30 hours, which exceeds the 'few hours' requirement. Furthermore, setting up Interconnect involves significant provisioning time.

Google's recommended practice for moving massive datasets (petabytes) within a short timeframe is using Transfer Appliance (Option B). This service allows you to ship encrypted hardware devices to your on-premises location. You load the data locally at maximum disk I/O speeds (often exceeding network limits), then ship the appliances back to Google. For 1 PB, this process can be completed in days, but the actual data transfer phase at the source is extremely fast compared to network upload. The key insight is that 'a few hours' refers to the active migration window or the feasibility of the task itself, which is impossible via network alone for this volume without extreme, unrealistic bandwidth assumptions. Transfer Appliance is designed specifically for this scale where time-to-transfer is the primary bottleneck.

Option C (Cloud VPN) offers low bandwidth (up to 10 Gbps typically) and would take weeks. Option D (gsutil serial/batch) is even slower due to overhead and lack of parallelism optimization for petabyte scales. Therefore, despite the logistical shipping time, Transfer Appliance is the only method that achieves the effective 'fast' migration goal for petabyte-scale data within reasonable operational constraints.

Exam Strategy

Always calculate the theoretical transfer time based on bandwidth caps. If the question specifies petabyte-scale data and a very short duration ('hours'), rule out network-based solutions immediately. Look for offline/physical transfer methods like Transfer Appliance or Snowball equivalents, as they bypass network latency and bandwidth ceilings.

Frequently Asked Questions

Why isn't Cloud Interconnect faster than Transfer Appliance?

Cloud Interconnect is limited by physical bandwidth (max 100-200 Gbps). Moving 1 PB takes ~27+ hours. Transfer Appliance uses local disk I/O which is faster than network upload for large batches.

Does Transfer Appliance meet security requirements?

Yes. It uses hardware encryption keys managed by Google. Data is encrypted at rest and in transit between facilities, meeting strict security standards.

More PDE FAQ →

Community Votes

A
69%
B
31%

69% of anonymous learners picked answer A. Votes are pick records left by other test-takers — they are not the verified answer.

Discussion 30 contributions

Compiled from public community threads and reader submissions, lightly edited for clarity. Handles and vote tallies are shown as recorded, and comments reflect the views of individual contributors.

From public community threads 30

  1. devaid 👍 5 Answered A
    Well it doesn't mentions anything about not enough bandwidth to meet your project deadline. I guess you can assume they have 200GBps+ of bandwith, otherwise it shouldn't take only a few hours.
  2. Atnafu 👍 4
    B It takes 30hrs with 100Gbps bandwidth- more than a day to transfer https://cloud.google.com/architecture/migration-to-google-cloud-transferring-your-large-datasets#:~:text=addresses%20or%20NATs.-,Online%20versus%20offline%20transfer,A%20certain%20amount%20of%20management%20overhead%20is%20built%20into%20these%20calculations.,-As%20noted%20earlier
  3. nwk 👍 4
    Vote A Questions states "transfer over a secure connection" , not offline with Transfer Appliance
  4. midgoo 👍 3 Answered B
    Either this question is very tricky or very poor written. It says 'Data transfer time during the migration should take only a few hours'. We should not add the 20days for overhead time for Appliance into the total time of migration. If 'a few hours' = 30hours or more, A will be good enough. If 'a few hours' = 10 or less, B is the only way (with multiple devices to copy at the same time)
  5. vaga1 👍 3 Answered B
    1 PB and "few hours". It is clearly referring to Transfer Appliance https://cloud.google.com/architecture/migration-to-google-cloud-transferring-your-large-datasets#time
Show 25 more comments
  1. arien_chen 👍 3 Answered A
    A https://cloud.google.com/storage-transfer/docs/transfer-options#:~:text=Transferring%20more%20than%201%20TB%20from%20on%2Dpremises
  2. iooj 👍 3
    One who wanted to use Transfer Appliance to migrate data in a few hours, you should live near Google office and run really fast :D
  3. knith66 👍 3
    Transfer Appliance is a slow process. wont be able to do in few hours
  4. MaxNRG 👍 2 Answered A
    Cloud Interconnect provides a dedicated private connection between on-prem and Google Cloud for high bandwidth (up to 100 Gbps) and low latency. This facilitates large, fast data transfers. Storage Transfer Service supports parallel data transfers over Cloud Interconnect. It can transfer petabyte-scale datasets faster by transferring objects in parallel. Storage Transfer Service uses HTTPS encryption in transit and at rest by default for secure data transfers. It follows Google-recommended practices for large data migrations vs ad hoc methods like gsutil or scp. The other options would take too long for a 1 PB transfer (VPN capped at 3 Gbps, manual transfers) or introduce extra steps like batching and checksums. Cloud Interconnect + Storage Transfer is the recommended Google solution.
  5. LanaOjisan 👍 2
    It is believed that A. One reason is that for "secure" and "in a few hours," the communication can be done securely using a direct physical line without going through an ISP. Also, depending on the case, in the case of "Dedicated Interconnect," the maximum transfer can be as high as 200 Gbps, and the fastest data transfer of 1 PB can be completed in 11 hours. Therefore, A.
  6. wjtb 👍 2 Answered B
    Even with 100gbps bandwith, you will not reach a data transfer time within the range of "hours" for 1PB. Transfer appliance is the way to go. https://cloud.google.com/architecture/migration-to-google-cloud-transferring-your-large-datasets#time
  7. Atnafu 👍 2
    B https://cloud.google.com/architecture/migration-to-google-cloud-transferring-your-large-datasets#:~:text=Few%20things%20in,not%20be%20obtained.
  8. ducc 👍 2 Answered B
    I vote for B
  9. Nandhu95 👍 1 Answered A
    Expected time via transfer appliance is around 20 days , and achieving the same using Storage transfer service with highest bandwidth of 100GPS is 30 hrs, so hence its been asked for hrs .. its A Acquiring a Transfer Appliance is straightforward. In the Google Cloud console, you request a Transfer Appliance, indicate how much data you have, and then Google ships one or more appliances to your requested location. You're given a number of days to transfer your data to the appliance ("data capture") and ship it back to Google. The expected turnaround time for a network appliance to be shipped, loaded with your data, shipped back, and rehydrated on Google Cloud is 20 days. If your online transfer timeframe is calculated to be substantially more than this timeframe, consider Transfer Appliance. The total cost for the 300 TB device process is less than $2,500.
  10. carbonscave 👍 1 Answered A
    Why A is the correct choice The Math of "A Few Hours": Moving 1 Petabyte of data is an enormous undertaking. To move 1 PB over a network in just a "few hours," you need an absolutely massive physical pipe—something in the realm of multiple 100 Gbps connections bonded together. Cloud Interconnect is the only Google Cloud networking product capable of providing that level of enterprise-grade, dedicated bandwidth. The Recommended Engine: Storage Transfer Service (STS) is Google's gold standard for moving large-scale data from on-premises file systems (or other clouds) into Cloud Storage. It automatically handles the heavy lifting: parallelizing the transfer, managing automatic retries, ensuring encryption in transit, and performing checksum validations to guarantee data integrity.
  11. Oleksandr0501 👍 1 Answered A
    gpt: Based on security and speed, if the data is highly sensitive and security is the top priority, then option B (using a Transfer Appliance) may be a better choice. Transfer Appliance uses hardware encryption to transfer data and is designed to securely transfer large amounts of data. However, if speed is the primary concern, then option A (using Cloud Interconnect and Storage Transfer Service) may be a better choice as it allows for faster transfer speeds over a dedicated and secure connection. It ultimately depends on the specific needs and priorities of the organization. A vague treaky question. Bad author of it... B is also good. As were said in discuss. by smb, a question asks "safe connection", so - a Cloud Interconnect (A)
  12. knith66 👍 1 Answered A
    Dedicated Interconnect provides direct physical connections between your on-premises network and Google's network. Dedicated Interconnect enables you to transfer large amounts of data between networks, which can be more cost-effective than purchasing additional bandwidth over the public internet. https://cloud.google.com/network-connectivity/docs/interconnect/concepts/dedicated-overview
  13. musumusu 👍 1
    Answer A, One time transfer is cheaper and less secure always using Transfer Appliance. you need to do it in faster way, set up Interconnect speed limit is 50mbps - 10GBps and Transfer Appliance speed can goes up to 40GBps. I am choosing A for security concern only.
  14. spicebits 👍 1
    B can't be the answer - You have to wait 25 days to receive the appliance and another 25 days to get the appliance back and data loaded to cloud storage: https://cloud.google.com/transfer-appliance/docs/4.0/overview#transfer-speeds
  15. vaga1 👍 1
    it says data transfer during the migration. It mean from when the migration is "activated", which means from when the Transfer Appliace device is plugged and ready to be used
  16. AzureDP900 👍 1
    A. Establish a Cloud Interconnect connection between the on-premises data center and Google Cloud, and then use the Storage Transfer Service. Most Voted
  17. bigquery1102 👍 1 Answered A
    A is correct https://cloud.google.com/architecture/migration-to-google-cloud-transferring-your-large-datasets#transfer_appliance_for_larger_transfers
  18. GyaneswarPanigrahi 👍 1
    1. Transfer over a secure connection. 2. Should take only a few hours. Hence, Storage Transfer Service instead of Transfer Appliance.
  19. knith66 👍 1
    This link has additional clarity https://cloud.google.com/network-connectivity/docs/interconnect/concepts/terminology
  20. TNT87 👍 1
    Answer A https://cloud.google.com/architecture/migration-to-google-cloud-transferring-your-large-datasets
  21. AWSandeep 👍 1 Answered B
    B. Use a Transfer Appliance and have engineers manually encrypt, decrypt, and verify the data.
  22. MounicaN 👍 1 Answered A
    Huge data can be migrated over cloud interconnect
  23. pluiedust 👍 1 Answered A
    A is correct.
  24. AzureDP900 👍 1
    A is right
  25. YorelNation 👍 1 Answered A
    Same A

Reader comments & corrections

No reader corrections yet — spotted an error or have a note? Share it below.

Log in to comment, report an error, or add a note about this question.

How to establish consistent BigQuery spend?

S-grade

Triggering Airflow DAGs Reactively in Private Cloud Composer

S-grade

Bigtable Row Key Design for Time-Series Data

S-grade

Migrating Hadoop Spark Jobs to Dataproc Serverless and Cloud Storage

S-grade

← Back to PDE Study Guide