Minimizing Development Time for Weekly Retraining Pipelines in Vertex AI

PMLE By islever Editorial Last updated

Developing end-to-end ML pipelines. Training models.

Short answer

A Use Kubeflow Pipelines SDK with BigQueryJobOp to run the existing SQL script and CustomTrainingJobOp to launch Vertex AI training, minimizing development time through asset reuse.

Question

You recently developed a wide and deep model in TensorFlow. You generated training datasets using a SQL script that preprocessed raw data in BigQuery by performing instance-level transformations of the data. You need to create a training pipeline to retrain the model on a weekly basis. The trained model will be used to generate daily recommendations. You want to minimize model development and training time. How should you develop the training pipeline?
  1. Use the Kubeflow Pipelines SDK to implement the pipeline. Use the BigQueryJobOp component to run the preprocessing script and the CustomTrainingJobOp component to launch a Vertex AI training job. Correct Answer
  2. Use the Kubeflow Pipelines SDK to implement the pipeline. Use the DataflowPythonJobOp component to preprocess the data and the CustomTrainingJobOp component to launch a Vertex AI training job.
  3. Use the TensorFlow Extended SDK to implement the pipeline Use the ExampleGen component with the BigQuery executor to ingest the data the Transform component to preprocess the data, and the Trainer component to launch a Vertex AI training job.
  4. Use the TensorFlow Extended SDK to implement the pipeline Implement the preprocessing steps as part of the input_fn of the model. Use the ExampleGen component with the BigQuery executor to ingest the data and the Trainer component to launch a Vertex AI training job.

Key Takeaways

The main trap is choosing TFX for its ML-native features; however, KFP is superior here because it allows direct reuse of existing BigQuery SQL scripts via BigQueryJobOp, avoiding the need to rewrite preprocessing logic.

Learn how to minimize model development time for weekly retraining by using Kubeflow Pipelines SDK with BigQueryJobOp to execute existing SQL scripts alongside Vertex AI Custom Training Jobs.

Common mistake: Choosing Option C (TFX) is common because TFX is designed for ML pipelines, but it fails the 'minimize development time' constraint by requiring the reimplemention of existing SQL-based preprocessing into TFX Transform components.

Expert Analysis

The key to this question lies in the specific constraint: minimize model development and training time. The scenario provides a pre-existing SQL script for instance-level transformations. Option A leverages Kubeflow Pipelines (KFP) SDK with the BigQueryJobOp component, which allows you to execute your existing SQL script directly without rewriting logic or migrating data. This reusability significantly reduces development time compared to building custom TFX components or rewriting preprocessing logic.

While Option C suggests using TensorFlow Extended (TFX), it proposes using the Transform component for preprocessing. However, the prompt states transformations are already done via SQL in BigQuery. Using TFX's Transform would require reimplementing that logic in TensorFlow Transform, increasing development effort. Furthermore, KFP is often preferred for orchestrating existing tools (like BigQuery jobs) and custom training jobs (CustomTrainingJobOp) when the goal is efficient pipeline orchestration rather than end-to-end ML-specific feature engineering within the TF ecosystem. The BigQueryJobOp acts as a native connector to run the SQL script efficiently.

Option B uses DataflowPythonJobOp, which is overkill for simple SQL transformations and adds complexity. Option D integrates preprocessing into input_fn, which is not suitable for scalable, offline training pipelines where preprocessing should be decoupled and optimized separately. Therefore, Option A offers the most direct path by reusing existing assets and leveraging managed pipeline operators.

Official Reference

https://cloud.google.com/vertex-ai/docs/training/custom-training

This link opens the official vendor documentation in a new tab.

Exam Strategy

When an exam question emphasizes 'minimizing development time' or 'reusing existing code', look for options that integrate with current tools rather than replacing them. If a SQL script exists for preprocessing, choose a pipeline option that can execute that SQL directly (e.g., BigQueryJobOp) instead of one that requires rewriting logic in a new framework (e.g., TFX Transform).

Frequently Asked Questions

Why is Kubeflow Pipelines better than TFX for this scenario?

Kubeflow Pipelines with BigQueryJobOp allows you to run the existing SQL script directly. TFX would require rewriting the preprocessing logic into Transform components, increasing development time.

What does BigQueryJobOp do in this pipeline?

BigQueryJobOp is a pre-built component in Kubeflow Pipelines that executes SQL queries or scripts in BigQuery, allowing seamless integration of existing data processing steps.

More PMLE FAQ →

Community Votes

A
47%
C
40%
D
13%

47% of anonymous learners picked answer A. Votes are pick records left by other test-takers — they are not the verified answer.

Discussion 16 contributions

Compiled from public community threads and reader submissions, lightly edited for clarity. Handles and vote tallies are shown as recorded, and comments reflect the views of individual contributors.

From public community threads 16

  1. bigdapper 👍 7 Answered A
    Ans: A The goal is to minimize model development and training time. The processing script is already developed in BQ. Option C requires rewriting and reimplementing the transformation logic.
  2. guilhermebutzke 👍 5
    My Answer: A According with this documentation: https://cloud.google.com/vertex-ai/docs/tabular-data/tabular-workflows/overview A: CORRECT: BigQueryJobOp for running the existing preprocessing script that already resides there, CustomTrainingJobOp for launching custom training jobs on Vertex AI, which aligns with the requirement of using the pre-trained TensorFlow model. B: Not Correct: While DataflowPythonJobOp can be used for preprocessingthis increasing development time compared to the simpler BigQueryJobOp approach. C and D: Not Correct: While possible, using the TensorFlow Extended SDK with its components introduces unnecessary complexity for this specific scenario. For example, why use ExampleGen? Implementing preprocessing within the model's input_fn is generally not recommended due to potential efficiency drawbacks and training-serving skew issues.
  3. lunalongo 👍 5 Answered C
    C is the best option because: - TFX is designed for ML pipelines, reducing custom code needs and development time and training time as the statement requires - ExampleGen with the BQ executor eliminate data export needs; - Trainer component seamlessly integrates with Vertex AI, leveraging its managed infrastructure for training, further reducing development and operational overhead. A & B uses Kubeflow Pipelines, which would mean more development time and code customization (your model is in TensorFlow); D puts preprocessing inside input_fn, which is generally less efficient for large datasets and complex transformations.
  4. batevv 👍 3 Answered A
    The correct answer is: A. Use the Kubeflow Pipelines SDK to implement the pipeline. Use the BigQueryJobOp component to run the preprocessing script and the CustomTrainingJobOp component to launch a Vertex AI training job. Kubeflow Pipelines (KFP) is a good choice for orchestrating training pipelines, especially since the requirement is to minimize model development and training time. The BigQueryJobOp component is appropriate because the data preprocessing is already performed in BigQuery using SQL scripts. Using BigQueryJobOp avoids unnecessary additional processing layers. The CustomTrainingJobOp component allows launching a Vertex AI training job, which aligns with the need for scalable and managed model training.
  5. SausageMuffins 👍 3 Answered C
    Example Gen directly ingest data from BigQuery and the transform component makes it more efficient than using an input fn. I chose C over A and B because kubeflow pipelines is more sophisticated and requires more setup and effort because of it's customizability.
Show 11 more comments
  1. TanTran04 👍 3 Answered A
    I go with A Kubeflow Pipelines SDK: supports machine learning and includes components specifically for tasks like data preprocessing, model training, and validation. BigQueryJobOp: enabling you to preprocess data using SQL scripts efficiently within BigQuery.
  2. BlehMaks 👍 3 Answered A
    D is wrong. Google doesn't recommend to use input_fn for preprocessing https://www.tensorflow.org/tfx/guide/tft_bestpractices#preprocessing_options_summary
  3. Shark0 👍 2 Answered C
    Given the requirement to minimize model development and training time while creating a training pipeline for a wide and deep model trained on datasets preprocessed using a SQL script in BigQuery, the most suitable option is: C. Use the TensorFlow Extended SDK to implement the pipeline. Use the ExampleGen component with the BigQuery executor to ingest the data, the Transform component to preprocess the data, and the Trainer component to launch a Vertex AI training job. This option leverages TensorFlow Extended (TFX), which is designed for scalable and production-ready machine learning pipelines. The ExampleGen component with the BigQuery executor efficiently ingests data from BigQuery. The Transform component applies preprocessing steps to the data, and the Trainer component launches a Vertex AI training job, minimizing the time and effort required for model development and training.
  4. tdum76000 👍 2 Answered D
    "If you use TensorFlow in an ML workflow that processes terabytes of structured data or text data, we recommend that you build your pipeline using TFX." https://cloud.google.com/vertex-ai/docs/pipelines/build-pipeline Google recommends TFX for large amount of structured data. Use input_fn for the Tensorflow model as it will output a tf.data.Dataset object. Note: As it is not mentionned that we are working with terabytes of data, Kubeflow is a viable option and i would choose answer A but i'll stick to google's recommendations
  5. forport 👍 2 Answered C
    Option C is the most suitable because TFX provides a comprehensive MLOps framework, seamlessly integrating data ingestion, preprocessing, and model training, while also offering strong support for Vertex AI, making it the most efficient solution for the given use case.
  6. AK2020 👍 2 Answered C
    C. Use the TensorFlow Extended SDK to implement the pipeline. Use the ExampleGen component with the BigQuery executor to ingest the data, the Transform component to preprocess the data, and the Trainer component to launch a Vertex AI training job.
  7. OpenKnowledge 👍 1 Answered C
    TFX Components: ExampleGen: ingesting and optionally splitting the input dataset into training and evaluation sets. StatisticsGen: Generates descriptive statistics for the dataset, which are crucial for understanding data characteristics and identifying potential issues. Transform: Performs feature engineering and data preprocessing using TensorFlow Transform. It applies transformations to the raw data to prepare it for model training. Trainer: Trains the machine learning model using the preprocessed data and specified model architecture. Tuner: Optimizes the hyperparameters of the model to improve performance.
  8. taylz876 👍 1 Answered A
    You already have a working SQL script in BigQuery that does the preprocessing, and the transformations are instance-level (each row is transformed on its own). That means you don't need to redo the preprocessing in a new tool. In A, the prebuilt BigQueryJobOp component runs your existing SQL script as it is, and the prebuilt CustomTrainingJobOp component launches the training job. There's no custom code to write, which minimizes development time, and the pipeline can be scheduled to run weekly. The managed runner is Agent Platform Pipelines (formerly Vertex AI Pipelines).
  9. pikachu007 👍 1 Answered D
    Addressing Limitations of Other Options: Kubeflow Pipelines (A and B): While Kubeflow offers flexibility, it might require more setup and configuration, potentially increasing development time compared to TFX's integrated approach. Separate Preprocessing (C): Using a separate Transform component for preprocessing can add complexity and potential overheads, especially for instance-level transformations that can often be directly integrated within the model's input pipeline.
  10. gscharly 👍 1 Answered A
    agree with guilhermebutzke
  11. Carlose2108 👍 1
    Why not C?

Reader comments & corrections

No reader corrections yet — spotted an error or have a note? Share it below.

Log in to comment, report an error, or add a note about this question.

Monitoring ML Model Performance with Precision and Recall Sampling

S-grade

Optimizing ML Model Recall for Compliance Risk

S-grade

How to Minimize Development Time for Weekly TensorFlow Retraining in Vertex AI?

S-grade

Building CLV Models with BigQuery ML in Vertex AI Workbench

S-grade

Practice All PMLE Questions

Access 65 questions with complete answers and detailed explanations.

View Full PMLE Practice Test →

← Back to PMLE Study Guide