How to Build a Managed MLOps Pipeline with PySpark Preprocessing on Google Cloud?

PMLE By islever Editorial Last updated Aug 31, 2026

Short answer

B Set up a Vertex AI Pipelines to orchestrate the MLOps pipeline. Use the predefined Dataproc component for the PySpark-based workloads.

Question

You are tasked with building an MLOps pipeline to retrain tree-based models in production. The pipeline will include components related to data ingestion, data processing, model training, model evaluation, and model deployment. Your organization primarily uses PySpark-based workloads for data preprocessing. You want to minimize infrastructure management effort. How should you set up the pipeline?

Set up a TensorFlow Extended (TFX) pipeline on Vertex AI Pipelines to orchestrate the MLOps pipeline. Write a custom component for the PySpark-based workloads on Dataproc.
Set up a Vertex AI Pipelines to orchestrate the MLOps pipeline. Use the predefined Dataproc component for the PySpark-based workloads. Correct Answer
Set up Kubeflow Pipelines on Google Kubernetes Engine to orchestrate the MLOps pipeline. Write a custom component for the PySparkbased workloads on Dataproc.
Set up Cloud Composer to orchestrate the MLOps pipeline. Use Dataproc workflow templates for the PySpark-based workloads in Cloud Composer.

Key Takeaways

Tests choosing a fully managed ML pipeline orchestrator over custom or general-purpose workflow tools, with the common trap being overcomplicating the setup with custom components or non-ML-native services like Cloud Composer or Kubeflow.

This question tests selecting the optimal fully managed MLOps orchestration service that natively integrates with PySpark workloads while minimizing infrastructure overhead. The community unanimously agrees that Vertex AI Pipelines with its built-in Dataproc component is the most efficient and lowest-management solution.

Common mistake: Option D (Cloud Composer) is often mistakenly chosen because it is a managed workflow tool, but it lacks native ML pipeline features and requires more manual configuration than Vertex AI Pipelines for MLOps-specific tasks.

Expert Analysis

Why the Answer Is Correct

Vertex AI Pipelines is Google’s fully managed, purpose-built orchestration service for machine learning workflows, directly addressing the requirement to minimize infrastructure management. It offers native, predefined components for Dataproc, allowing seamless execution of PySpark preprocessing without writing custom containerized steps. This integration reduces development time, eliminates boilerplate code, and ensures tight coupling between data engineering and model training stages within the Vertex AI ecosystem.

Why the Other Options Are Wrong

Option A introduces unnecessary complexity by requiring a custom component for PySpark when a native predefined component already exists. Option C relies on Kubeflow Pipelines on GKE, which demands significant cluster management and contradicts the “minimize infrastructure management” constraint. Option D uses Cloud Composer, a general-purpose DAG orchestrator that lacks ML-specific features and does not provide the same level of seamless, managed integration for end-to-end MLOps pipelines as Vertex AI Pipelines.

Community Comment Notes

Contributors consistently highlight that Vertex AI Pipelines’ native Dataproc support directly satisfies the PySpark requirement while keeping operational overhead low [Comment 2]. Several users note that the phrase “minimize infrastructure management effort” immediately rules out self-managed options like Kubeflow on GKE [Comment 4]. The consensus emphasizes that leveraging predefined components over custom ones aligns perfectly with Google Cloud’s recommended best practices for managed MLOps architectures.

Official Reference

These links open the official vendor documentation in a new tab.

Exam Strategy

When questions emphasize “minimize infrastructure management,” prioritize fully managed Google Cloud services over self-hosted or Kubernetes-based alternatives. Always look for native, predefined components that match your existing tech stack to avoid unnecessary custom development and reduce exam time.

Community Votes

100%

100% of anonymous learners picked answer B. Votes are pick records left by other test-takers — they are not the verified answer.

Discussion 4 contributions

Compiled from public community threads and reader submissions, lightly edited for clarity. Handles and vote tallies are shown as recorded, and comments reflect the views of individual contributors.

From public community threads 4

carolctech 👍 2 Answered B

B) Best option due to higher ease of use, integration with existing PySpark infrastructure (via Dataproc) and minimal infrastructure management overhead, because: Vertex AI Pipelines is fully managed, minimizing infra management effort and natively integrated with Dataproc for PySpark (while Composer is not); Dataproc’s predefined component for PySpark workload reduces effort and error probability; It is suitable for tree-based models (other options are too, but with more effort)
AB_C 👍 2 Answered B

This is the most suitable approach
Omi_04040 👍 1 Answered B

A- Rejected due to component for the PySpark-based C- Kubeflow Pipelines not a managed service and the question mentions 'minimize infrastructure management effort' D-
Pau1234 👍 1 Answered B

minimize infrastructure management effort -- hence B

Reader comments & corrections

No reader corrections yet — spotted an error or have a note? Share it below.

Practice All PMLE Questions

Access 65 questions with complete answers and detailed explanations.

View Full PMLE Practice Test →

← Back to PMLE Study Guide