How to fix Vertex AI model scaling stuck at one replica with low CPU?
You have deployed a scikit-team model to a Vertex AI endpoint using a custom model server. You enabled autoscaling: however, the deployed model fails to scale beyond one replica, which led to dropped requests. You notice that CPU utilization remains low even during periods of high load. What should you do?
Community Votes
56% of anonymous learners picked answer B. Votes are pick records left by other test-takers — they are not the verified answer.
Community Insight
The exam tests your understanding that autoscaling in Vertex AI relies on CPU utilization; if the model server uses too few workers, CPU stays low even under load, so adding workers is the solution, not adding GPU or changing replica counts.
The community consensus is that when a Vertex AI custom model server fails to scale beyond one replica despite high load and low CPU utilization, the correct fix is to increase the number of workers in the model server. This addresses the concurrency bottleneck that prevents autoscaling metrics from triggering.
Option A (attach a GPU) is the most common wrong answer because candidates assume a hardware accelerator is needed when performance is poor, but the real problem is underutilized CPU due to insufficient worker concurrency, not compute capacity.
Community Discussion (7 comments)
Comments & Corrections
No comments yet — spotted an error or have a note? Share it below.
Expert Analysis
Why the Answer Is Correct
Option B is correct because Vertex AI autoscaling is based on CPU utilization of the deployed model. A custom model server with only one worker cannot saturate the CPU even under high request load, so the CPU utilization metric remains low and no scaling is triggered. Increasing the number of workers allows the model server to handle more concurrent requests, which raises CPU utilization and lets autoscaling add more replicas. As comment [3] notes, “One worker isn't enough to saturate the CPU and so no scaling is triggered.”
Why the Other Options Are Wrong
Option A (attach a GPU) does not address the underlying issue of low CPU utilization; it would change the compute profile but still leave the worker concurrency bottleneck unresolved. Option C (scheduling scaling) manually manages capacity but does not fix the reason why autoscaling fails to react, and it is less dynamic than correcting the server configuration. Option D (increasing minReplicaCount) forces more replicas but does not solve the low CPU utilization problem and can cause unnecessary cost; it also does not address the root cause of underutilized resources.
Community Comment Notes
Many comments support B, with [2] explaining that low CPU utilization indicates a bottleneck within the model server itself and that increasing workers lets the server use available CPU resources effectively. Comment [1] provides an official recommendation: start with one worker per core and increase if CPU is low under high load. Comment [4] incorrectly suggests C and highlights a misunderstanding: they think B only speeds up prediction within a single replica, but the community consensus, including [6] and [7], confirms B is the right approach.
Official Reference
Exam Strategy
When you see autoscaling failing with low CPU utilization under load, immediately think of worker concurrency or resource limits inside the container, not hardware or replica counts. Focus on how Vertex AI uses CPU utilization as the metric for HPA and what affects that metric.
Related Analysis
Practice All PMLE Questions
Access 65 questions with complete answers and detailed explanations.
View Full PMLE Practice Test →