How to configure distributed training with minimal code changes?

You developed a Transformer model in TensorFlow to translate text. Your training data includes millions of documents in a Cloud Storage bucket. You plan to use distributed training to reduce training time. You need to configure the training job while minimizing the effort required to modify code and to manage the cluster’s configuration. What should you do?

  1. Create a Vertex AI custom training job with GPU accelerators for the second worker pool. Use tf.distribute.MultiWorkerMirroredStrategy for distribution. Source Reference Answer
  2. Create a Vertex AI custom distributed training job with Reduction Server. Use N1 high-memory machine type instances for the first and second pools, and use N1 high-CPU machine type instances for the third worker pool.
  3. Create a training job that uses Cloud TPU VMs. Use tf.distribute.TPUStrategy for distribution.
  4. Create a Vertex AI custom training job with a single worker pool of A2 GPU machine type instances. Use tf.distribute.MirroredStrategv for distribution.

Community Votes

A
100%

100% of anonymous learners picked answer A. Votes are pick records left by other test-takers — they are not the verified answer.

Community Insight

The question tests the selection of a managed distributed training solution that balances performance with low operational overhead, specifically identifying MultiWorkerMirroredStrategy as the low-effort standard for multi-GPU setups.

To minimize code modification and cluster management effort for distributed TensorFlow training, use a Vertex AI custom training job with GPU accelerators and the tf.distribute.MultiWorkerMirroredStrategy.

Selecting Cloud TPU VMs (Option C) is a frequent error because, while TPUs are fast for Transformers, they typically require significant code changes compared to the minimal refactoring needed for MultiWorkerMirroredStrategy on GPUs.

Community Discussion (4 comments)

fitri001 👍 3 Selected: A
Vertex AI custom training job: This leverages a managed service within GCP, reducing cluster configuration and management overhead. GPU accelerators for the second worker pool: This allows for distributed training across multiple GPUs, significantly speeding up training compared to a single worker pool. tf.distribute.MultiWorkerMirroredStrategy: This is a TensorFlow strategy specifically designed for distributed training on multiple machines. It minimizes code changes as it handles data parallelization and model replication across devices.
Carlose2108 👍 2
Why not C?
guilhermebutzke 👍 2 Selected: A
My Answer: A - Distributed training: Utilizes GPUs in 2nd worker pool for speedup. - Minimal code changes: Vertex AI custom job for ease of use. - Managed cluster: No manual configuration needed. Other options: - B: Complex setup with different machine types and Reduction Server. - C: TPUs may not be optimal for Transformers and require code changes. - D: Lacks distributed training, limiting speed improvement.
pikachu007 👍 3 Selected: A
Minimizes code modification: MultiWorkerMirroredStrategy often requires minimal code changes to distribute training across multiple workers, aligning with the goal of minimizing effort. Simplifies cluster management: Vertex AI handles cluster configuration and scaling for custom training jobs, reducing the need for manual management. Effective distributed training: MultiWorkerMirroredStrategy is well-suited for large models and datasets, efficiently distributing training across GPUs.

Comments & Corrections

No comments yet — spotted an error or have a note? Share it below.

Log in to comment, report an error, or add a note about this question.

Submitted for moderation before publishing. Keep it helpful and respectful.

Expert Analysis

Why the Answer Is Correct

Option A is the correct choice because Vertex AI custom training jobs provide a managed environment that handles cluster provisioning and scaling, significantly reducing management effort. Utilizing tf.distribute.MultiWorkerMirroredStrategy allows for synchronous data-parallel training across multiple GPU workers, which is ideal for Transformer models, and generally requires minimal code changes—mostly just wrapping the model creation and compilation in a strategy scope.

Why the Other Options Are Wrong

Option B is incorrect because introducing a Reduction Server adds configuration complexity and is typically reserved for specific asynchronous training scenarios or extreme parameter server loads, rather than standard synchronous training. Option C is incorrect because migrating to Cloud TPUs often necessitates substantial code alterations, such as adjusting input pipelines and enabling XLA compilation, which violates the requirement to minimize code changes. Option D is incorrect because tf.distribute.MirroredStrategy is designed for single-machine, multi-GPU training and does not support distributed training across a cluster of machines, which is necessary to efficiently process millions of documents.

Community Comment Notes

Community comments strongly validate Option A, emphasizing its ability to leverage managed infrastructure for reduced overhead and its compatibility with existing TensorFlow code. Commenters specifically ruled out Option C due to the code modification requirements for TPUs and Option B due to its complex setup involving different machine types and Reduction Servers.

Official Reference

Exam Strategy

When exam questions emphasize minimizing code changes and management effort for distributed training, always prefer Vertex AI Custom Jobs with MultiWorkerMirroredStrategy over TPUs or complex parameter server architectures, unless the workload explicitly demands TPU-specific optimizations.

Related Analysis

You work at a bank. You need to develop a credit risk model

A-Grade

You work as an analyst at a large banking firm. You are deve

A-Grade

You are building a custom image classification model and pla

A-Grade

Practice All PMLE Questions

Access 65 questions with complete answers and detailed explanations.

View Full PMLE Practice Test →

← Back to PMLE Study Guide