How to Fix Vertex AI Endpoint Autoscaling for Memory-Intensive Models?

You have built a custom model that performs several memory-intensive preprocessing tasks before it makes a prediction. You deployed the model to a Vertex AI endpoint, and validated that results were received in a reasonable amount of time. After routing user traffic to the endpoint, you discover that the endpoint does not autoscale as expected when receiving multiple requests. What should you do?

Use a machine type with more memory
Decrease the number of workers per machine
Increase the CPU utilization target in the autoscaling configurations.
Decrease the CPU utilization target in the autoscaling configurations Source Reference Answer

Community Votes

74%

26%

74% of anonymous learners picked answer D. Votes are pick records left by other test-takers — they are not the verified answer.

Community Insight

The exam tests whether you understand that lowering the CPU utilization target in autoscaling configurations makes the endpoint scale up sooner, accommodating memory-intensive preprocessing spikes even though CPU is not the true bottleneck.

This question tests Vertex AI endpoint autoscaling behavior with memory-intensive preprocessing models. The community strongly supports decreasing the CPU utilization target to trigger earlier scale-up, while changing the machine type is considered ineffective for scaling behavior.

Choosing A, 'Use a machine type with more memory,' is the most common wrong answer. It addresses memory capacity but does not fix autoscaling responsiveness, and it may waste resources when traffic is low.

Community Discussion (5 comments)

b1a8fae 👍 11 Selected: D

D. The idea behind this question is getting autoscaling to handle well the fluctuating input of requests. Changing the machine (A) is not related to autoscaling, and you might not be using the full potential of the machine during the whole time, bur rather only during instances of peak traffic. You need to lower the autoscaling threshold (the target utilization metric mentioned in the options is CPU, so we will go with this) so you make use of more resources whenever too many memory-intensive requests are happening. https://cloud.google.com/compute/docs/autoscaler/scaling-cpu#scaling_based_on_cpu_utilization https://cloud.google.com/compute/docs/autoscaler#autoscaling_policy

pikachu007 👍 6 Selected: A

B. Decreasing Workers: This might reduce memory usage per machine but could also decrease overall throughput, potentially impacting performance. C. Increasing CPU Utilization Target: This wouldn't directly address the memory bottleneck and could trigger unnecessary scaling based on CPU usage, not memory requirements. D. Decreasing CPU Utilization Target: This could lead to premature scaling, potentially increasing costs without addressing the root cause.

VinaoSilva 👍 1 Selected: D

"use autoscale" = deacrease cpu utilization target

fitri001 👍 2 Selected: D

D. Decrease the CPU utilization target: This is the most suitable approach. By lowering the CPU utilization target, the endpoint will scale up at a lower CPU usage level. This increases the likelihood of scaling up when the memory-intensive preprocessing tasks cause a rise in CPU utilization, even though memory is the root cause.

guilhermebutzke 👍 3 Selected: D

Option D, "Decrease the CPU utilization target in the autoscaling configurations," could be a valid approach to address the issue of autoscaling and anticipate spikes in traffic. By lowering the threshold, the autoscaling system would initiate scaling actions at a lower CPU utilization level, allowing for a more proactive response to increasing demands.

Comments & Corrections

No comments yet — spotted an error or have a note? Share it below.

Expert Analysis

Why the Answer Is Correct

Decreasing the CPU utilization target (D) lowers the threshold at which Vertex AI scales up the number of instances. This makes the endpoint more sensitive to CPU increases caused by memory-intensive preprocessing, leading to earlier and more proactive scaling when request traffic spikes. Comment [3] explains that lowering the threshold allows scaling actions at lower CPU utilization, which better anticipates traffic surges. Comment [4] reinforces that scaling at a lower CPU usage level accommodates the CPU rise caused by memory-preprocessing tasks, even though memory is the root cause.

The issue is not the lack of memory per se but the autoscaling algorithm not reacting quickly enough. By lowering the target, the endpoint scales out before CPU utilization reaches a high level, preventing latency from memory pressure and preprocessing delays. This aligns with the goal of adapting to fluctuating traffic, as stated in comment [1].

Why the Other Options Are Wrong

A is incorrect because changing the machine type to one with more memory does not change the autoscaling policy; the endpoint may still not scale out when traffic increases. B is incorrect because decreasing workers per machine would reduce throughput and potentially exacerbate performance, not help autoscaling react faster. C is incorrect because increasing the CPU utilization target makes the endpoint wait longer before scaling, worsening the problem by only scaling under heavier load. Comment [2] notes that increasing the target could trigger unnecessary scaling based on CPU usage and not address the memory bottleneck.

Community Comment Notes

Comment [1] explicitly states that changing the machine type is unrelated to autoscaling and that the machine may not be fully utilized except during peak traffic, so lowering the threshold is the right approach. Comment [3] supports D for proactive scaling. Comment [4] emphasizes that lowering the CPU utilization target increases the likelihood of scaling up when memory-intensive preprocessing causes CPU utilization to rise. Comment [2] is a dissenting view but actually discusses why other options fail, ultimately reinforcing D as the best choice given the memory-intensive context.

Official Reference

Exam Strategy

For Vertex AI autoscaling questions, remember that lowering the target utilization metric (e.g., CPU utilization) triggers scaling sooner. Focus on the scaling behavior itself rather than instance size or memory, and note that autoscaling decisions are based on configured utilization targets.

Related Analysis

Practice All PMLE Questions

Access 65 questions with complete answers and detailed explanations.

View Full PMLE Practice Test →

← Back to PMLE Study Guide