How to scale Vertex AI endpoints for increased traffic?

You work for an online grocery store. You recently developed a custom ML model that recommends a recipe when a user arrives at the website. You chose the machine type on the Vertex AI endpoint to optimize costs by using the queries per second (QPS) that the model can serve, and you deployed it on a single machine with 8 vCPUs and no accelerators. A holiday season is approaching and you anticipate four times more traffic during this time than the typical daily traffic. You need to ensure that the model can scale efficiently to the increased demand. What should you do?

1. Maintain the same machine type on the endpoint.
1. Change the machine type on the endpoint to have 32 vCPUs.
1. Maintain the same machine type on the endpoint Configure the endpoint to enable autoscaling based on vCPU usage. Source Reference Answer
1. Change the machine type on the endpoint to have a GPU. Configure the endpoint to enable autoscaling based on the GPU usage.

Community Votes

100%

100% of anonymous learners picked answer C. Votes are pick records left by other test-takers — they are not the verified answer.

Community Insight

This question tests the knowledge of Vertex AI endpoint scaling strategies, specifically identifying that autoscaling is superior to vertical scaling or manual upgrades, with the trap being assuming a larger machine type or GPU is necessary for handling traffic spikes.

To handle increased traffic on a Vertex AI endpoint, enabling autoscaling is the preferred method over manual intervention or vertical scaling. The community agrees that configuring autoscaling based on vCPU usage optimizes costs while ensuring availability during demand spikes.

Selecting Option B is a common mistake because upgrading to 32 vCPUs seems to match the predicted traffic increase, but it ignores the efficiency and cost-effectiveness of autoscaling.

Community Discussion (10 comments)

fitri001 👍 6 Selected: C

Option A: Manually adding compute nodes after an alert might lead to delays and potential outages during peak traffic. Option B: Upgrading to 32 vCPUs upfront might be an overkill if the current machine type with 8 vCPUs can handle the typical daily traffic. Vertical scaling (more vCPUs) might be suitable only if the model can benefit from additional CPU power. Option D: Using a GPU is unlikely to benefit a recipe recommendation model, which likely doesn't involve intensive graphical processing. Additionally, monitoring GPU usage wouldn't be relevant.

lunalongo 👍 1 Selected: C

C) Option C is the best because: 1) It leverages the built-in autoscaling capabilities of Vertex AI. 2) It's the most efficient/cost-effective solution for fluctuating traffic. 2) Manually scaling (options A and B) is reactive and inefficient 3) A GPU is unnecessary, there is no intensive graphical processing

AzureDP900 👍 1

C is right because 1)Since you've already optimized your model's deployment on a single machine with 8 vCPUs, it makes sense to maintain the same machine type to avoid any potential performance issues. 2)Enabling autoscaling based on vCPU usage will allow your endpoint to automatically add more machines as needed to handle the increased traffic during the holiday season. This approach is more efficient and cost-effective than scaling up individual machines or adding new machines manually. 3)Monitoring CPU usage with a job and alerting when thresholds are exceeded allows you to detect potential issues before they impact performance.

omermahgoub 👍 1 Selected: C

C: Use Autoscaling Based on vCPU Usage

emsherff 👍 1 Selected: C

Autoscaling based on vCPU usage aligns well with the workload.

emsherff 👍 2

Option A is manual intervention Option B is overprovisioning preemptively, which is an overkill ( autoscaling should be preferred) Option D - Unless the recipe recommendation model uses GPU-accelerated computations (e.g., some deep learning models), adding a GPU won't be beneficial and will increase costs. I would go with C - Autoscaling based on vCPU usage which aligns well with the workload.

daidai75 👍 1 Selected: C

Option B can only support exact 4x times traffic, but the requirement is four times "more", so B is not the best at least for me.

b1a8fae 👍 1 Selected: C

I would go for C as it enables autoscaling when exceeding a determined CPU usage threshold.

pikachu007 👍 1 Selected: C

Cost Optimization: It starts with the current machine type, avoiding unnecessary upfront costs, and scales only when needed. Autoscaling: It automatically adjusts compute resources based on vCPU usage, ensuring the endpoint can handle traffic spikes without manual intervention. Monitoring and Alerting: It provides visibility into resource usage and triggers alerts for potential issues, enabling proactive actions. Investigation: It encourages investigation of alerts to identify any underlying problems beyond expected traffic growth, ensuring overall system health.

kalle_balle 👍 1 Selected: B

Voting for B as it's the only option to autoscale even though the cost will go up. All other options include manual intervention.

Comments & Corrections

No comments yet — spotted an error or have a note? Share it below.

Expert Analysis

Why the Answer Is Correct

Option C is correct because it leverages Vertex AI's built-in autoscaling capabilities to dynamically adjust the number of nodes based on vCPU usage. This ensures the endpoint can handle the 4x traffic surge during the holiday season without manual intervention. It also maintains cost efficiency by scaling back down when traffic decreases, avoiding the waste associated with permanent overprovisioning.

Why the Other Options Are Wrong

Option A is incorrect because it lacks a mechanism to handle the increased traffic, potentially leading to outages. Option B is incorrect because vertical scaling to 32 vCPUs is a rigid, overprovisioning strategy that wastes resources during normal traffic and lacks flexibility for unpredictable spikes. Option D is incorrect because adding a GPU increases costs unnecessarily for a model that does not require GPU acceleration, and scaling should be based on the actual resource bottleneck (vCPU).

Community Comment Notes

Commenters emphasized that Option A requires manual intervention, which is too slow for sudden traffic spikes. Several users noted that Option B is "overkill" and preemptive, whereas Option C scales resources only when needed. There was also a strong consensus that Option D is invalid because the recommendation model does not require GPU resources.

Official Reference

Exam Strategy

When facing questions about traffic spikes, prioritize autoscaling options over manual resizing or static vertical scaling. Remember that adding GPUs is only necessary for deep learning models, not for general CPU-bound prediction tasks.

Related Analysis

Practice All PMLE Questions

Access 65 questions with complete answers and detailed explanations.

View Full PMLE Practice Test →

← Back to PMLE Study Guide