TFX Pipelines for TensorFlow Workflows

PMLE By islever Editorial Last updated

Short answer

D Use the TensorFlow Extended (TFX) SDK to create components that match the described workflow and deploy on Vertex AI Pipelines.

Question

You are building a TensorFlow text-to-image generative model by using a dataset that contains billions of images with their respective captions. You want to create a low maintenance, automated workflow that reads the data from a Cloud Storage bucket collects statistics, splits the dataset into training/validation/test datasets performs data transformations trains the model using the training/validation datasets, and validates the model by using the test dataset. What should you do?
  1. Use the Apache Airflow SDK to create multiple operators that use Dataflow and Vertex AI services. Deploy the workflow on Cloud Composer.
  2. Use the MLFlow SDK and deploy it on a Google Kubernetes Engine cluster. Create multiple components that use Dataflow and Vertex AI services.
  3. Use the Kubeflow Pipelines (KFP) SDK to create multiple components that use Dataflow and Vertex AI services. Deploy the workflow on Vertex AI Pipelines.
  4. Use the TensorFlow Extended (TFX) SDK to create multiple components that use Dataflow and Vertex AI services. Deploy the workflow on Vertex AI Pipelines. Correct Answer

Key Takeaways

The question tests the ability to recognize that a described workflow (read, statistics, split, transform, train, validate) maps exactly to the canonical components of a TFX pipeline.

This page explains why TensorFlow Extended (TFX) is the correct SDK for building an automated, low-maintenance ML pipeline on Vertex AI Pipelines when the workflow steps align with TFX's built-in components.

Common mistake: Choosing Kubeflow Pipelines (KFP) because of generic documentation recommendations, ignoring that the specific workflow steps are natively provided by TFX, which reduces custom code and maintenance.

Expert Analysis

Canonical TFX Workflow

The sequence of operations described—reading data, collecting statistics, splitting, transforming, training, and evaluating—is the exact canonical architecture of a TensorFlow Extended (TFX) pipeline. TFX provides built-in components for each of these steps (e.g., ExampleGen, StatisticsGen, Transform, Trainer, Evaluator), making it the natural choice for a low-maintenance, automated TensorFlow workflow.

Why Not KFP (Option C)

While Kubeflow Pipelines (KFP) is a robust framework for orchestrating ML workflows on Vertex AI Pipelines, it is a generic orchestrator. To achieve the specific statistical and transformation steps natively in a TensorFlow pipeline, KFP would require custom component development, increasing maintenance. As one comment notes, "TFX is going to be easier than kubeflow with custom code, as it basically does exactly what is listed there, by default."

Addressing the Documentation Nuance

Some comments cite Google Cloud documentation stating, "If you use TensorFlow in an ML workflow that processes terabytes of structured data or text data, we recommend that you build your pipeline using TFX. For other use cases, we recommend that you build your pipeline using the Kubeflow Pipelines SDK." While the dataset includes images, the explicit mention of "TensorFlow" combined with the precise enumeration of standard TFX pipeline stages overrides the generic KFP recommendation. The workflow described is fundamentally a TFX pipeline.

Operational Consequences

Choosing KFP (Option C) for a workflow that perfectly aligns with TFX's built-in components would force the engineering team to reinvent the wheel, writing and maintaining custom code for statistics generation and data transformation that TFX handles natively. This contradicts the "low maintenance" requirement. Options A and B fail on the maintenance requirement due to the operational overhead of managing Cloud Composer or a GKE cluster for a purely ML-centric workflow.

Official Reference

Exam Strategy

When a question lists a sequence of ML workflow steps (reading data, statistics, splitting, transforming, training, evaluating) and mentions TensorFlow, map these directly to TFX's built-in components. Do not default to generic orchestrators like KFP or Airflow unless the steps require custom logic outside of TFX's standard library.

Community Votes

D
67%
C
33%

67% of anonymous learners picked answer D. Votes are pick records left by other test-takers — they are not the verified answer.

Discussion 13 contributions

Compiled from public community threads and reader submissions, lightly edited for clarity. Handles and vote tallies are shown as recorded, and comments reflect the views of individual contributors.

From public community threads 13

  1. pinimichele01 👍 6 Answered C
    If you use TensorFlow in an ML workflow that processes terabytes of structured data or text data, we recommend that you build your pipeline using TFX. For other use cases, we recommend that you build your pipeline using the Kubeflow Pipelines SDK https://cloud.google.com/vertex-ai/docs/pipelines/build-pipeline#sdk
  2. fitri001 👍 3
    why not others? A. Airflow with Dataflow and Vertex AI: While Airflow is a powerful workflow management tool, deploying it on Cloud Composer adds additional complexity compared to the managed environment of Vertex AI Pipelines. B. MLflow with Dataflow and Vertex AI: MLflow focuses primarily on model lifecycle management. While it can be used for building pipelines, KFP offers a more specialized and user-friendly approach for this specific use case. D. TFX with Dataflow and Vertex AI: TFX is a comprehensive end-to-end ML platform. While it offers several functionalities, it might be an overkill for this scenario focusing on data processing, training, and validation. KFP provides a simpler solution for this specific workflow.
  3. pikachu007 👍 3 Answered D
    Airflow (A): While versatile, Airflow often requires more manual configuration and integration with ML services, potentially increasing maintenance effort. MLFlow (B): MLFlow focuses on experiment tracking and model management, lacking built-in pipeline components for data processing and model training. Kubeflow Pipelines (C): KFP is flexible but requires more setup and infrastructure management compared to TFX's managed services.
  4. baimus 👍 3 Answered D
    TFX is going to be easier than kubeflow with custom code, as it basically does exactly what is listed there, by default.
  5. PhilipKoku 👍 3 Answered D
    D) TFX is the way forward as it has services to support every step of the use case presented.
Show 8 more comments
  1. fitri001 👍 2 Answered C
    KFP Pipelines: Kubeflow Pipelines (KFP) is a popular open-source framework for building and deploying machine learning workflows. It provides a user-friendly SDK for defining pipelines as components and simplifies workflow orchestration. Vertex AI Pipelines Integration: Vertex AI Pipelines is a managed service from Google Cloud that integrates seamlessly with KFP. You can deploy your KFP-defined workflow on Vertex AI Pipelines, leveraging its features like scheduling, monitoring, and versioning. Dataflow and Vertex AI Services: Both Dataflow and Vertex AI are Google Cloud services well-suited for this workflow
  2. alja12 👍 2 Answered D
    I would vote for D. If you read the question carefully you will see that you're processing billions of images with captions (TEXT+ images). So we have to preprocess text. Plus, you want to build a TensorFlow text-to-image generative model. For me it points to the option D. Although at the very beginning C was my primary choice.
  3. winston9 👍 2 Answered D
    C and D are valid options. if the model is created in TF, use TFX, in any other case, use KFP; therefore, here is D
  4. wences 👍 2 Answered D
    in this one will go with D, TFX is more specialized than kfp
  5. dija123 👍 2 Answered D
    Agree with TFX
  6. taylz876 👍 1 Answered D
    Answer: D (use the TensorFlow Extended (TFX) SDK to create multiple components that use Dataflow and Vertex AI services, and deploy the workflow on Vertex AI Pipelines) Why D is right The workflow the question lists is exactly the standard TFX pipeline, step by step: • Read the data from Cloud Storage: the ExampleGen component. • Collect statistics: StatisticsGen. • Split into training, validation and test sets: ExampleGen does the split. • Data transformations: the Transform component. • Train the model: Trainer. • Validate on the test dataset: Evaluator. TFX is built for TensorFlow models, and its components run on Dataflow, so they scale to billions of images. Deploying on Vertex AI Pipelines (now Agent Platform Pipelines) makes it automated and low maintenance, because it is serverless and you don't run any infrastructure.
  7. pinimichele01 👍 1
    https://cloud.google.com/vertex-ai/docs/pipelines/build-pipeline#sdk
  8. kornick 👍 1 Answered C
    TFX -> processes terabytes of structured data or text data

Reader comments & corrections

No reader corrections yet — spotted an error or have a note? Share it below.

Log in to comment, report an error, or add a note about this question.

Practice All PMLE Questions

Access 65 questions with complete answers and detailed explanations.

View Full PMLE Practice Test →

← Back to PMLE Study Guide