Agent Platform Pipelines: Gate Deployment on Held-Out Test Metrics
Short answer
D Add batch prediction on the held-out test dataset, evaluate its metrics in a pipeline component, and use a conditional task to deploy to the production Agent Platform endpoint only when the threshold is met.
Question
Key Takeaways
The question tests held-out evaluation plus a deployment gate in Pipelines, and the main trap is accepting training-data or training-metric checks as proof of production readiness.
This PMLE question tests how to gate a custom model's production deployment on held-out test metrics in Agent Platform Pipelines; the correct approach is D, using batch prediction, model evaluation, and a conditional task.
Common mistake: A plausible wrong choice is C, which deploys a temporary endpoint for online prediction; although it evaluates held-out data, it adds endpoint cost and administrative overhead that the question explicitly asks to minimize.
Expert Analysis
What the pipeline must prove
The workflow cannot accept training-time numbers as evidence of production quality. The question explicitly requires a held-out test dataset and a specific evaluation threshold. That means the evaluation step must consume predictions on data the model did not train on. Option D is the choice that combines a held-out test prediction step with a threshold gate without a temporary endpoint.
How the winning path runs
The training component produces and registers the custom model. Next, a batch prediction component runs that registered model against the held-out test dataset. That is the key cost-control step: batch prediction is a job, so there is no always-on endpoint to deploy, monitor, or tear down. The prediction output flows to a model evaluation component, which calculates the required metric(s) on the test data. Finally, a conditional task compares those metrics with the specified threshold. Only when the threshold is met does the pipeline deploy the model to the production Agent Platform endpoint. The answer takes effect at the evaluation component and conditional task, not at training time.
Why the alternatives fail
A evaluates on the training dataset, so it cannot demonstrate generalization. B uses training metrics and compares against a production model; it neither evaluates the held-out set nor enforces the stated threshold. C does evaluate on test data, but it deploys a temporary endpoint for online prediction, adding endpoint cost and administrative overhead. Community reasoning for D is consistent: batch prediction scores the held-out test set, the evaluation component computes the metrics, and the conditional task gates deployment without a temporary endpoint.
Minimizing cost and overhead
D fits both constraints: batch prediction is ephemeral, the evaluation is explicit, and deployment is conditional. The trap is treating any metric calculation as sufficient; here the data split and the deployment gate are as important as the component names.
Official Reference
https://cloud.google.com/vertex-ai/docs/predictions/batch-predictions
This link opens the official vendor documentation in a new tab.
Exam Strategy
When a pipeline question asks you to validate a custom model on held-out data, separate three concerns: where predictions are produced, where metrics are computed, and where deployment is gated. Prefer batch prediction or an evaluation component inside the pipeline over a temporary endpoint unless the scenario requires online-only behavior. Watch for choices that reuse training data or training metrics—they cannot prove generalization. The conditional task should act only after a test-set evaluation, which usually makes the lowest-overhead option with explicit held-out evaluation correct.
Frequently Asked Questions
Why is a temporary endpoint (option C) not the best way to evaluate the model?
Why can't the pipeline use metrics from the training dataset?
Where does the threshold check occur in the pipeline?
Community Votes
100% of anonymous learners picked answer D. Votes are pick records left by other test-takers — they are not the verified answer.
Discussion 2 contributions
Compiled from public community threads and reader submissions, lightly edited for clarity. Handles and vote tallies are shown as recorded, and comments reflect the views of individual contributors.
From public community threads 2
-
taylz876 👍 1 Answered DWhy D is right • Evaluate on the held-out test data: batch prediction scores the test dataset, and the evaluation component calculates the metrics from the predictions. • Deploy only if it passes: a conditional task compares the metric with the threshold and deploys only if it's met. • Minimum cost and overhead: batch prediction needs no endpoint, so you only pay while it runs, and these are prebuilt pipeline components. Why the others are wrong • A: Metrics from the training dataset don't show how the model performs on new data, which is what the test set is for. • B: Training metrics compared with the production model's isn't an evaluation on held-out data. • C: A temporary endpoint plus a custom component costs more and takes more work than batch prediction.
-
n2183712847 👍 1 Answered DCorrect answer: D. A batch prediction step can efficiently generate predictions on the held-out test set, and a model evaluation component can calculate the required metrics before a conditional deployment, avoiding the cost of deploying a temporary endpoint; A evaluates on the training data instead of the held-out set, B compares training metrics rather than test performance, and C incurs unnecessary online endpoint deployment and inference costs.
Reader comments & corrections
No reader corrections yet — spotted an error or have a note? Share it below.
Related Analysis
Practice All PMLE Questions
Access 65 questions with complete answers and detailed explanations.
View Full PMLE Practice Test →