How to Minimize Development Time for Weekly TensorFlow Retraining in Vertex AI?

PMLE By islever Editorial Last updated

Short answer

A Use the Kubeflow Pipelines SDK to implement the pipeline. Use the BigQueryJobOp component to run the preprocessing script and the CustomTrainingJobOp component to launch a Vertex AI training job.

Question

You recently developed a wide and deep model in TensorFlow. You generated training datasets using a SQL script that preprocessed raw data in BigQuery by performing instance-level transformations of the data. You need to create a training pipeline to retrain the model on a weekly basis. The trained model will be used to generate daily recommendations. You want to minimize model development and training time. How should you develop the training pipeline?
  1. Use the Kubeflow Pipelines SDK to implement the pipeline. Use the BigQueryJobOp component to run the preprocessing script and the CustomTrainingJobOp component to launch a Vertex AI training job. Correct Answer
  2. Use the Kubeflow Pipelines SDK to implement the pipeline. Use the DataflowPythonJobOp component to preprocess the data and the CustomTrainingJobOp component to launch a Vertex AI training job.
  3. Use the TensorFlow Extended SDK to implement the pipeline Use the ExampleGen component with the BigQuery executor to ingest the data the Transform component to preprocess the data, and the Trainer component to launch a Vertex AI training job.
  4. Use the TensorFlow Extended SDK to implement the pipeline Implement the preprocessing steps as part of the input_fn of the model. Use the ExampleGen component with the BigQuery executor to ingest the data and the Trainer component to launch a Vertex AI training job.

Key Takeaways

It evaluates selecting the right orchestration framework to reuse existing BigQuery SQL preprocessing, with the common trap being over-engineering with TFX when simpler KFP components suffice.

This question tests optimal pipeline orchestration for retraining TensorFlow models on Vertex AI while minimizing development overhead. The community consensus aligns with using Kubeflow Pipelines and BigQuery Job components to leverage existing SQL preprocessing rather than rebuilding logic in TFX.

Common mistake: Many candidates choose Option C, assuming TFX's specialized ML components automatically reduce development time. However, TFX would require reimplementing the existing BigQuery SQL instance-level transformations, increasing setup complexity and contradicting the minimize development time requirement.

Expert Analysis

Why the Answer Is Correct

Option A correctly leverages Kubeflow Pipelines SDK alongside Vertex AI-specific operators. Using BigQueryJobOp directly executes the existing SQL preprocessing script without code migration, while CustomTrainingJobOp efficiently launches the TensorFlow training job on managed infrastructure. This approach minimizes boilerplate configuration and aligns with Google's recommended pattern for tabular data workflows where preprocessing is already optimized in BigQuery.

Why the Other Options Are Wrong

Option B incorrectly routes data through Dataflow, adding unnecessary service switching when the transformation logic already resides in BigQuery SQL. Option C proposes TensorFlow Extended, which is powerful but forces developers to rewrite SQL-based instance transformations into TFX Transform components, increasing development time. Option D suggests embedding preprocessing in input_fn, which violates TensorFlow best practices and degrades training performance by blocking data ingestion during computation.

Community Comment Notes

Candidates are evenly split between A and C, often debating whether TFX's specialized architecture inherently saves time. [1] correctly cites Vertex AI's tabular workflow documentation, emphasizing that reusing existing BigQuery jobs reduces overhead. [9] highlights that KFP's flexibility avoids the steep learning curve of configuring full TFX pipelines, while [3] reinforces why input_fn preprocessing is strongly discouraged by official TensorFlow guidelines.

Official Reference

These links open the official vendor documentation in a new tab.

Exam Strategy

When a question emphasizes minimizing development time and explicitly mentions existing preprocessing scripts, prioritize solutions that reuse those scripts via managed operators rather than adopting a new ML framework. Always match the tool's native capabilities to the stated constraint before defaulting to popular frameworks like TFX.

Community Votes

A
50%
C
50%

50% of anonymous learners picked answer A. Votes are pick records left by other test-takers — they are not the verified answer.

Discussion 13 contributions

Compiled from public community threads and reader submissions, lightly edited for clarity. Handles and vote tallies are shown as recorded, and comments reflect the views of individual contributors.

From public community threads 13

  1. guilhermebutzke 👍 3
    My Answer: A According with this documentation: https://cloud.google.com/vertex-ai/docs/tabular-data/tabular-workflows/overview A: CORRECT: BigQueryJobOp for running the existing preprocessing script that already resides there, CustomTrainingJobOp for launching custom training jobs on Vertex AI, which aligns with the requirement of using the pre-trained TensorFlow model. B: Not Correct: While DataflowPythonJobOp can be used for preprocessingthis increasing development time compared to the simpler BigQueryJobOp approach. C and D: Not Correct: While possible, using the TensorFlow Extended SDK with its components introduces unnecessary complexity for this specific scenario. For example, why use ExampleGen? Implementing preprocessing within the model's input_fn is generally not recommended due to potential efficiency drawbacks and training-serving skew issues.
  2. lunalongo 👍 2 Answered C
    C is the best option because: - TFX is designed for ML pipelines, reducing custom code needs and development time and training time as the statement requires - ExampleGen with the BQ executor eliminate data export needs; - Trainer component seamlessly integrates with Vertex AI, leveraging its managed infrastructure for training, further reducing development and operational overhead. A & B uses Kubeflow Pipelines, which would mean more development time and code customization (your model is in TensorFlow); D puts preprocessing inside input_fn, which is generally less efficient for large datasets and complex transformations.
  3. BlehMaks 👍 2 Answered A
    D is wrong. Google doesn't recommend to use input_fn for preprocessing https://www.tensorflow.org/tfx/guide/tft_bestpractices#preprocessing_options_summary
  4. Shark0 👍 1 Answered C
    Given the requirement to minimize model development and training time while creating a training pipeline for a wide and deep model trained on datasets preprocessed using a SQL script in BigQuery, the most suitable option is: C. Use the TensorFlow Extended SDK to implement the pipeline. Use the ExampleGen component with the BigQuery executor to ingest the data, the Transform component to preprocess the data, and the Trainer component to launch a Vertex AI training job. This option leverages TensorFlow Extended (TFX), which is designed for scalable and production-ready machine learning pipelines. The ExampleGen component with the BigQuery executor efficiently ingests data from BigQuery. The Transform component applies preprocessing steps to the data, and the Trainer component launches a Vertex AI training job, minimizing the time and effort required for model development and training.
  5. batevv 👍 1 Answered A
    The correct answer is: A. Use the Kubeflow Pipelines SDK to implement the pipeline. Use the BigQueryJobOp component to run the preprocessing script and the CustomTrainingJobOp component to launch a Vertex AI training job. Kubeflow Pipelines (KFP) is a good choice for orchestrating training pipelines, especially since the requirement is to minimize model development and training time. The BigQueryJobOp component is appropriate because the data preprocessing is already performed in BigQuery using SQL scripts. Using BigQueryJobOp avoids unnecessary additional processing layers. The CustomTrainingJobOp component allows launching a Vertex AI training job, which aligns with the need for scalable and managed model training.
Show 8 more comments
  1. tdum76000 👍 1 Answered D
    "If you use TensorFlow in an ML workflow that processes terabytes of structured data or text data, we recommend that you build your pipeline using TFX." https://cloud.google.com/vertex-ai/docs/pipelines/build-pipeline Google recommends TFX for large amount of structured data. Use input_fn for the Tensorflow model as it will output a tf.data.Dataset object. Note: As it is not mentionned that we are working with terabytes of data, Kubeflow is a viable option and i would choose answer A but i'll stick to google's recommendations
  2. pikachu007 👍 1 Answered D
    Addressing Limitations of Other Options: Kubeflow Pipelines (A and B): While Kubeflow offers flexibility, it might require more setup and configuration, potentially increasing development time compared to TFX's integrated approach. Separate Preprocessing (C): Using a separate Transform component for preprocessing can add complexity and potential overheads, especially for instance-level transformations that can often be directly integrated within the model's input pipeline.
  3. forport 👍 1 Answered C
    Option C is the most suitable because TFX provides a comprehensive MLOps framework, seamlessly integrating data ingestion, preprocessing, and model training, while also offering strong support for Vertex AI, making it the most efficient solution for the given use case.
  4. SausageMuffins 👍 1 Answered C
    Example Gen directly ingest data from BigQuery and the transform component makes it more efficient than using an input fn. I chose C over A and B because kubeflow pipelines is more sophisticated and requires more setup and effort because of it's customizability.
  5. TanTran04 👍 1 Answered A
    I go with A Kubeflow Pipelines SDK: supports machine learning and includes components specifically for tasks like data preprocessing, model training, and validation. BigQueryJobOp: enabling you to preprocess data using SQL scripts efficiently within BigQuery.
  6. AK2020 👍 1 Answered C
    C. Use the TensorFlow Extended SDK to implement the pipeline. Use the ExampleGen component with the BigQuery executor to ingest the data, the Transform component to preprocess the data, and the Trainer component to launch a Vertex AI training job.
  7. gscharly 👍 1 Answered A
    agree with guilhermebutzke
  8. Carlose2108 👍 1
    Why not C?

Reader comments & corrections

No reader corrections yet — spotted an error or have a note? Share it below.

Log in to comment, report an error, or add a note about this question.

How to Handle Categorical Features in BigQuery ML for Fast Deployment?

S-grade

How to preprocess BigQuery data for TensorFlow?

S-grade

How to Track Model and Prediction Lineage in Vertex AI?

S-grade

How to Automate Vertex AI Model Retraining Efficiently?

S-grade

Practice All PMLE Questions

Access 65 questions with complete answers and detailed explanations.

View Full PMLE Practice Test →

← Back to PMLE Study Guide