How Should You Compare ML Models Trained Across Vertex AI Pipelines and Notebooks?

Your team is training a large number of ML models that use different algorithms, parameters, and datasets. Some models are trained in Vertex AI Pipelines, and some are trained on Vertex AI Workbench notebook instances. Your team wants to compare the performance of the models across both services. You want to minimize the effort required to store the parameters and metrics. What should you do?

  1. Implement an additional step for all the models running in pipelines and notebooks to export parameters and metrics to BigQuery.
  2. Create a Vertex AI experiment. Submit all the pipelines as experiment runs. For models trained on notebooks log parameters and metrics by using the Vertex AI SDK. Source Reference Answer
  3. Implement all models in Vertex AI Pipelines Create a Vertex AI experiment, and associate all pipeline runs with that experiment.
  4. Store all model parameters and metrics as model metadata by using the Vertex AI Metadata API.

Community Votes

B
100%

100% of anonymous learners picked answer B. Votes are pick records left by other test-takers — they are not the verified answer.

Community Insight

The exam tests recognition that Vertex AI Experiments is designed to unify experiment tracking across multiple training services, and the common trap is choosing a generic solution like BigQuery export or re-implementing all models in pipelines.

Vertex AI Experiments is the recommended central service for storing and comparing parameters and metrics from models trained across both Vertex AI Pipelines and Workbench notebook instances, minimizing effort and leveraging built-in SDK support.

The most common wrong answer is A (export to BigQuery) because it seems like a centralized store, but it requires additional pipeline steps and manual setup, ignoring Vertex AI Experiments' native support for logging and comparison.

Community Discussion (6 comments)

fitri001 👍 3 Selected: B
Why B? Centralized Tracking: Vertex AI Experiments provides a central location to track and compare models trained in both pipelines and notebooks. Reduced Overhead: Submitting pipelines as experiment runs leverages the existing pipeline infrastructure for logging and avoids creating additional pipeline steps for all models. Notebook Integration: Vertex AI SDK allows notebooks to log parameters and metrics directly to the experiment, simplifying data collection from notebooks. why not C? C. All Models in Pipelines: Moving all models to pipelines might not be feasible or desirable. Pipelines are best suited for automated, repeatable training, while notebooks offer flexibility for exploration.
omermahgoub 👍 2 Selected: B
B. Create a Vertex AI experiment. Submit all the pipelines as experiment runs. For models trained on notebooks log parameters and metrics by using the Vertex AI SDK.
guilhermebutzke 👍 3 Selected: B
My Answer: B A: Not Correct: Not the best approach compared with Vertex AI experiment that does the same B: CORRECT: By submitting all pipelines as experiment runs, you can centralize the storage of parameters and metrics for models trained in Vertex AI Pipelines. This approach minimizes effort by providing a unified platform for storing and comparing model performance across different services. C: Not Correct: not feasible or ideal for models trained on Vertex AI Workbench notebook instances. D: Not Correct: If only basic parameter and metric storage is needed, and your team prioritizes simplicity over in-depth comparison, option D could be an alternative. For more complex scenarios requiring comprehensive analysis and comparison across diverse models, option B with Vertex AI Experiments
b1a8fae 👍 3 Selected: B
Divided between B and C. But logging parameters of models sounds easier than re-implementing a large amount of models as Vertex AI pipelines.
shadz10 👍 1 Selected: B
B is The correct answer here I believe - Vertex AI experiments - provides a unified way to store and compare model runs. pipeline runs - It provides a unified way to store and compare model runs. notebook instances - models trained on Vertex AI Workbench notebook instances, logging parameters and metrics using the Vertex AI SDK provides a consistent way to record the necessary information.
pikachu007 👍 1 Selected: C
Options A and B: Logging metrics to BigQuery involves additional setup and integration efforts. Option D: Loading Vertex ML Metadata into a pandas DataFrame for visualization requires manual work and doesn't leverage built-in visualization tools.

Comments & Corrections

No comments yet — spotted an error or have a note? Share it below.

Log in to comment, report an error, or add a note about this question.

Submitted for moderation before publishing. Keep it helpful and respectful.

Expert Analysis

Why the Answer Is Correct

B is correct because Vertex AI Experiments provides a centralized platform to track and compare models trained in both Vertex AI Pipelines and Workbench notebooks. When you submit pipeline runs as experiment runs, existing infrastructure handles logging without adding extra steps. For notebook models, the Vertex AI SDK logs parameters and metrics directly, ensuring consistent storage with minimal effort.

The official documentation and community comments (e.g., comment 1) highlight that this approach reduces overhead and integrates seamlessly with both services. Comment 5 also confirms that B offers a unified way to store and compare runs. This aligns perfectly with the requirement to minimize effort.

Why the Other Options Are Wrong

A (BigQuery export) is incorrect because it requires implementing an extra step for every model, adding maintenance burden and not leveraging Vertex AI's built-in experiment tracking. Comment 6 points out this involves additional setup and integration effort.

C is wrong because it forces re-implementing all notebook models into Vertex AI Pipelines, which is costly and unnecessary. Comment 3 notes that logging parameters via SDK is easier than re-implementing models. D (Metadata API) is wrong because it stores metadata but lacks the comparative visualization and experiment management features that Vertex AI Experiments provides, as mentioned in comment 6.

Community Comment Notes

The community overwhelmingly supports B (92 votes). Comment 1 is particularly useful, explaining centralized tracking, reduced overhead, and notebook integration. Comment 3 highlights the trade-off between B and C, noting that logging is simpler than re-implementing. Comment 6 correctly dismisses A and D for extra setup and limited visualization capabilities. These insights reinforce that B is the minimal-effort solution for cross-service comparison.

Official Reference

Exam Strategy

When a question asks about minimizing effort for cross-service model comparison, always look for a native unified service like Vertex AI Experiments. Focus on whether the option avoids re-architecting workflows or adding manual export steps, and prefer SDK-based logging over generic data stores.

Related Analysis

You work for a retail company. You have created a Vertex AI

A-Grade

You are developing a model to predict whether a failure will

S-Grade

You work for a retail company. You have been tasked with bui

S-Grade

Practice All PMLE Questions

Access 65 questions with complete answers and detailed explanations.

View Full PMLE Practice Test →

← Back to PMLE Study Guide