Why A100 GPU Utilization Drops to 0% in PyTorch ResNet-50 Training
Short answer
B The high CPU and intermittent 0% GPU utilization show the A100 is starved by data loading, so increase DataLoader num_workers and enable pin_memory to keep the ResNet-50 GPU fed.
Question
Key Takeaways
The exam tests whether you can distinguish GPU starvation from raw GPU compute limits; the trap is treating volatile GPU utilization as a reason to add GPUs or larger batches.
This PMLE question tests diagnosing a PyTorch data-loading bottleneck on Gemini Enterprise Agent Platform. The correct fix is B: increase DataLoader num_workers and enable pin_memory to keep the A100 GPU busy.
Common mistake: A is the most tempting wrong answer: increasing batch size seems to put more work on the A100, but it does not fix the CPU-side data loading that is leaving the GPU idle and may add memory pressure.
Expert Analysis
Reading the utilization signature
The job runs ResNet-50 in a custom PyTorch container on Gemini Enterprise Agent Platform with one A100. Volatile GPU utilization that repeatedly hits 0% while CPU stays above 90% is a classic starvation pattern: the training loop is not compute-bound; it is waiting for the input pipeline. The GPU finishes a batch quickly, then idles while Python workers decode, resize, and collate images. This is why B is correct: increasing DataLoader(num_workers=...) parallelizes host-side preprocessing, and pin_memory=True uses page-locked memory so host-to-device copies are faster and asynchronous. Both changes feed the A100 more consistently without changing the model.
Why the other levers miss
Option A may look attractive because a larger batch can saturate GPU memory, but it does not remove the CPU preprocessing bottleneck; it can worsen startup latency and memory pressure. Option C adds more A100s and DDP, but distributed training multiplies the number of data-hungry workers and does not fix per-process input starvation. Option D, AMP, speeds matrix math and reduces memory, but if the GPU is idle waiting for data, faster compute has little effect. Comments on this scenario make the same diagnostic point: high CPU and 0% GPU dips mean the GPU is waiting for the next batch, so the DataLoader is the target.
Adjacent decisions and scenario flips
This knowledge also covers choosing the right response when the symptom changes. If GPU utilization were consistently high and CPU low while step time is too long, AMP, a larger batch, or more GPUs could be reasonable—compute-bound, not input-bound. If CPU were moderate but disk or network I/O saturated, optimize storage/format, use more efficient image decoding, or cache/prefetch. If multiple GPUs were already present and underused, DDP tuning and per-worker sharding would matter. The key is to identify the constrained resource first.
Operational consequence
Getting this wrong is expensive: adding GPUs or larger batches without fixing data loading can raise cost and still leave the A100 idle, while AMP alone may leave the same 0% gaps. On Agent Platform, monitor GPU utilization, CPU utilization, and input throughput together; validate by watching GPU utilization rise and step time fall after the DataLoader change.
Official Reference
- https://pytorch.org/docs/stable/data.html
- https://pytorch.org/tutorials/recipes/recipes/tuning_guide.html
These links open the official vendor documentation in a new tab.
Exam Strategy
When a training job shows high CPU and GPU utilization that dips toward zero, read that as an input-pipeline stall before considering model or hardware changes. Separate symptoms by resource: sustained high GPU utilization with low CPU points to compute limits, while pinned CPU plus idle GPU points to DataLoader starvation. On PMLE questions, choose the smallest configuration change that fixes the bottleneck—here num_workers and pin_memory—and reject options that add hardware or precision changes without addressing the starving resource.
Frequently Asked Questions
Why does GPU utilization fall to 0% while CPU stays above 90%?
Does increasing batch size solve PyTorch GPU starvation?
When should I use AMP or more GPUs instead of changing DataLoader settings?
Community Votes
100% of anonymous learners picked answer B. Votes are pick records left by other test-takers — they are not the verified answer.
Discussion 2 contributions
Compiled from public community threads and reader submissions, lightly edited for clarity. Handles and vote tallies are shown as recorded, and comments reflect the views of individual contributors.
From public community threads 2
-
taylz876 👍 1 Answered BSimilar to Q28, Q65 and Q314 Why B is right • The symptom is a data loading bottleneck. The GPU keeps dropping to 0% while the CPU is pinned above 90%. The GPU is waiting for the CPU to prepare the next batch of images, so it sits idle between batches. • More workers prepare batches in parallel. In PyTorch, num_workers sets how many worker processes load and preprocess data at the same time, so batches are ready sooner. pin_memory places the data in a form that copies to the GPU faster. Together they keep the GPU fed. Why the others are wrong • A: A larger batch doesn't help if the GPU is waiting for data. It's idle because the input is too slow, not because the batch is small. • C: More GPUs would sit idle too, since the bottleneck is the CPU side. • D: Mixed precision speeds up the GPU's math, but the GPU isn't busy enough to benefit.
-
n2183712847 👍 1 Answered BCorrect answer: B. The high CPU usage and intermittent 0% GPU utilization indicate the GPU is waiting for data, so increasing DataLoader workers and enabling pin_memory improves data-loading throughput and keeps the GPU fed; A may increase memory pressure without fixing the CPU bottleneck, C adds GPUs without addressing the data pipeline bottleneck, and D can speed up GPU computation but does not resolve the CPU-side data-loading starvation.
Reader comments & corrections
No reader corrections yet — spotted an error or have a note? Share it below.
Related Analysis
Practice All PMLE Questions
Access 65 questions with complete answers and detailed explanations.
View Full PMLE Practice Test →