How to test a retrained model in production on Vertex AI with minimal complexity?

You work for an organization that operates a streaming music service. You have a custom production model that is serving a “next song” recommendation based on a user's recent listening history. Your model is deployed on a Vertex AI endpoint. You recently retrained the same model by using fresh data. The model received positive test results offline. You now want to test the new model in production while minimizing complexity. What should you do?

Create a new Vertex AI endpoint for the new model and deploy the new model to that new endpoint. Build a service to randomly send 5% of production traffic to the new endpoint. Monitor end-user metrics such as listening time. If end-user metrics improve between models over time, gradually increase the percentage of production traffic sent to the new endpoint.
Capture incoming prediction requests in BigQuery. Create an experiment in Vertex AI Experiments. Run batch predictions for both models using the captured data. Use the user’s selected song to compare the models performance side by side. If the new model’s performance metrics are better than the previous model, deploy the new model to production.
Deploy the new model to the existing Vertex AI endpoint. Use traffic splitting to send 5% of production traffic to the new model. Monitor end-user metrics, such as listening time. If end-user metrics improve between models over time, gradually increase the percentage of production traffic sent to the new model. Source Reference Answer
Configure a model monitoring job for the existing Vertex AI endpoint. Configure the monitoring job to detect prediction drift and set a threshold for alerts. Update the model on the endpoint from the previous model to the new model. If you receive an alert of prediction drift, revert to the previous model.

Community Votes

100%

100% of anonymous learners picked answer C. Votes are pick records left by other test-takers — they are not the verified answer.

Community Insight

The exam tests canary deployment using Vertex AI's built-in traffic splitting; the trap is choosing options that add custom routing services, extra endpoints, or offline-only evaluation when the platform feature already does the job.

Vertex AI endpoints natively support traffic splitting, letting you deploy a retrained model to the existing endpoint and canary-test it with a small slice of live traffic. The community unanimously agrees that reusing the current endpoint with built-in traffic splitting is the simplest way to validate a new model in production.

The most common wrong answer is A: creating a new endpoint and building a custom service to route 5% of traffic. It achieves a similar test but violates the 'minimizing complexity' requirement because Vertex AI already provides native traffic splitting on a single endpoint.

Community Discussion (4 comments)

fitri001 👍 2 Selected: C

For Simplicity: If speed and simplicity are your top priorities, deploying to the existing endpoint with caution (close monitoring during deployment) can work.--> choose C For Safety and Control: If minimizing risk and having better control over the testing process are more important, creating a new endpoint is the better option. This is generally the recommended approach for most production deployments. --> choose A

daidai75 👍 3 Selected: C

Here's why the option C is preferable: Minimized complexity: Leverages existing endpoint: No need to create and manage a new endpoint, reducing setup and maintenance overhead. Traffic splitting readily available: Vertex AI provides built-in traffic splitting functionality, simplifying traffic distribution. Efficient testing and monitoring: Direct comparison: Sending a percentage of traffic to the new model allows for direct comparison with the current model's performance on real user data. Gradual rollout: Starting with a small percentage mitigates potential risks and allows for gradual transition based on observed improvements. End-user metric monitoring: Focusing on metrics like listening time directly reflects user engagement and preference for the new recommendations.

b1a8fae 👍 2 Selected: C

Traffic splitting is a feature of Vertex AI that allows you to distribute the prediction requests among multiple models or model versions within the same endpoint. You can specify the percentage of traffic that each model or model version receives, and change it at any time. Traffic splitting can help you test the new model in production without creating a new endpoint or a separate service. You can deploy the new model to the existing Vertex AI endpoint, and use traffic splitting to send 5% of production traffic to the new model. You can monitor the end-user metrics, such as listening time, to compare the performance of the new model and the previous model. If the end-user metrics improve between models over time, you can gradually increase the percentage of production traffic sent to the new model. This solution can help you test the new model in production while minimizing complexity and cost.

pikachu007 👍 2 Selected: C

Option A: Building a separate service adds unnecessary complexity and requires managing two endpoints. Option B: Batch predictions in Vertex AI Experiments might not reflect real-time user behavior and don't directly affect the production environment. Option D: Model monitoring alerts for prediction drift might be triggered by natural variations in user behavior instead of genuine performance issues and could lead to unnecessary model rollbacks.

Comments & Corrections

No comments yet — spotted an error or have a note? Share it below.

Expert Analysis

Why the Answer Is Correct

Option C leverages Vertex AI's built-in traffic splitting, which allows multiple model versions to share one endpoint while you route a small percentage (e.g., 5%) of production traffic to the new version. This is a classic canary deployment: the new model is validated against real user behavior, measured through end-user metrics such as listening time. Because no new endpoint, service, or pipeline must be built, it directly satisfies the requirement to minimize complexity. Traffic percentages can be adjusted at any time, enabling a gradual rollout as metrics improve.

Why the Other Options Are Wrong

Option A performs the same canary test but requires a second endpoint plus a custom traffic-routing service, adding setup and maintenance overhead that the question explicitly asks you to avoid. Option B relies on batch predictions over captured requests, which cannot capture real-time engagement signals like listening time and never truly tests the model in the production serving path. Option D swaps the model outright and depends on prediction-drift alerts, which can fire due to natural variation in user behavior rather than genuine model regressions, and it offers no gradual traffic control. None of these balance in-production validation with minimal complexity as well as C.

Community Comment Notes

Community consensus is unanimous (100% for C), and commenters consistently point to the native traffic-splitting feature as the deciding factor. Comment [1] emphasizes that reusing the existing endpoint reduces setup and maintenance overhead while enabling direct side-by-side comparison of models. Comment [2] frames the trade-off clearly: C is best for speed and simplicity, while a separate endpoint (A) offers more control but is generally unnecessary for this scenario. Comment [4] effectively debunks the distractors, noting that batch predictions miss real-time user behavior and that drift alerts can trigger on natural variations instead of true performance issues.

Official Reference

Exam Strategy

When a question says 'minimize complexity,' favor managed, built-in platform features over custom services or extra infrastructure. Vertex AI endpoints natively support traffic splitting between model versions, so canary testing rarely requires a second endpoint. Treat phrases like 'build a service' or 'create a new endpoint' as red flags whenever a simpler native capability exists.

Related Analysis

Practice All PMLE Questions

Access 65 questions with complete answers and detailed explanations.

View Full PMLE Practice Test →

← Back to PMLE Study Guide