Monitoring Cloud Bigtable for Cluster Scaling Decisions

PDE By islever Editorial Last updated

Optimizing resources. Deploying and operationalizing the pipelines.

Short answer

CD Monitor write latency for sustained increases and track storage utilization against the 70% recommendation to decide when to scale the Cloud Bigtable cluster.

Question

You have a data pipeline that writes data to Cloud Bigtable using well-designed row keys. You want to monitor your pipeline to determine when to increase the
size of your Cloud Bigtable cluster. Which two actions can you take to accomplish this? (Choose two.)
  1. Review Key Visualizer metrics. Increase the size of the Cloud Bigtable cluster when the Read pressure index is above 100.
  2. Review Key Visualizer metrics. Increase the size of the Cloud Bigtable cluster when the Write pressure index is above 100.
  3. Monitor the latency of write operations. Increase the size of the Cloud Bigtable cluster when there is a sustained increase in write latency. Correct Answer
  4. Monitor storage utilization. Increase the size of the Cloud Bigtable cluster when utilization increases above 70% of max capacity. Correct Answer
  5. Monitor latency of read operations. Increase the size of the Cloud Bigtable cluster of read operations take longer than 100 ms.

Key Takeaways

The main trap is focusing on Key Visualizer pressure indices; however, with well-designed row keys, global write latency and overall storage utilization are the reliable signals for scaling compute and capacity.

Determine when to scale a Cloud Bigtable cluster by monitoring write latency and storage utilization. This guide explains why sustained write latency spikes and >70% storage usage are the key indicators for adding nodes.

Common mistake: Candidates often choose Key Visualizer metrics (A or B) thinking they are the primary scaling tools. However, since row keys are well-designed, hotspots are unlikely, and latency/storage are better indicators of general capacity needs.

Expert Analysis

The question asks for two actions to monitor a Cloud Bigtable pipeline to determine when to increase cluster size, given that row keys are well-designed. Well-designed row keys prevent hotspots, meaning performance issues will be global rather than localized to specific tablet servers.

Option C is correct because write latency is a direct indicator of compute and I/O saturation. When the cluster cannot keep up with the ingestion rate, write operations queue up, causing sustained increases in latency. Adding nodes distributes the load, reducing this latency. The comments highlight that "a sustained increase in write latency indicates that the cluster's compute resources are struggling," which is the primary signal for scaling out write-heavy workloads.

Option D is correct because storage utilization is a key metric for capacity planning. Google explicitly recommends not exceeding 70% of the hard limit on total storage to maintain headroom for data growth and ensure optimal performance. If storage utilization approaches or exceeds this threshold, adding nodes increases the total storage capacity and can also help with performance by distributing data across more tablets. The comments cite official documentation stating, "In general, do not use more than 70% of the hard limit on total storage... Add nodes to the cluster to reduce the disk load percentage."

Options A and B refer to Key Visualizer metrics (Read/Write pressure). While these indicate hotspotting, the premise states row keys are well-designed, minimizing hotspots. Furthermore, Key Visualizer is more about identifying skew than general capacity scaling triggers compared to latency and storage limits. Option E focuses on read latency; while important, the scenario emphasizes a pipeline writing data, making write latency the more critical concern for scaling based on ingestion. Additionally, the 100ms threshold is arbitrary and not a standard scaling trigger like the 70% storage rule.

Official Reference

These links open the official vendor documentation in a new tab.

Exam Strategy

When evaluating scaling triggers for managed databases, distinguish between symptoms of bottlenecks (latency) and capacity limits (storage %). For write-heavy pipelines with good schema design, ignore hotspot-specific metrics and focus on aggregate latency and storage thresholds defined in vendor best practices.

Frequently Asked Questions

Why isn't Key Visualizer used for scaling decisions here?

Key Visualizer helps identify hotspots caused by poor row key design. Since the question specifies well-designed row keys, hotspots are minimized, making aggregate latency and storage better scaling signals.

What is the recommended storage utilization threshold for Bigtable?

Google recommends not exceeding 70% of the hard limit on total storage to ensure room for data growth and maintain performance. Exceeding this suggests it is time to add nodes.

Does increasing cluster size improve read or write performance?

Adding nodes improves both, but in a write-heavy pipeline context, the primary concern is write latency. Scaled-out clusters distribute write load more effectively.

More PDE FAQ →

Community Votes

C
44%
D
39%
B
11%
A
6%

44% of anonymous learners picked answer C. Votes are pick records left by other test-takers — they are not the verified answer.

Discussion 34 contributions

Compiled from public community threads and reader submissions, lightly edited for clarity. Handles and vote tallies are shown as recorded, and comments reflect the views of individual contributors.

From public community threads 30 of 34

  1. jvg637 👍 53
    Answer is C & D. C –> Adding more nodes to a cluster (not replication) can improve the write performance https://cloud.google.com/bigtable/docs/performance D –> since Google recommends adding nodes when storage utilization is > 70% https://cloud.google.com/bigtable/docs/modifying-instance#nodes
  2. ch3n6 👍 14
    No. it is C, D. "You have a data pipeline that writes data to Cloud Bigtable using well-designed row keys." why are you monitoring read anyway? you are just writing.
  3. Barniyah 👍 11
    Key visualizer is bigtable metric , So A and B incorrect storage utilization also bigtable metric , So D incorrect The question want you to monitor pipeline metrics (which is dataflow metrics) , in our case we can only monitor latency . The answer will be : C & E
  4. Rajokkiyam 👍 7
    Well Defined RowKey So A & B out of scope. With Increase in cluster Size you can only reduce the latency of Write operations. Read latency doesn't have any impact with increasing the cluster size. Ready latency attributes to Cluster replication so Option E is also ruled out. Answer C & D
  5. Rajokkiyam 👍 5
    B is out of Scope because KeyVisualizer is used to identify the skew in data and help us to re-deisgn the rowkey. Since its already confirmed "Well Defined Rowkey" KeyViz is out of scope.
Show 25 more comments
  1. dabrat 👍 4
    Storage utilization (% max) The percentage of the cluster's storage capacity that is being used. The capacity is based on the number of nodes in your cluster. In general, do not use more than 70% of the hard limit on total storage, so you have room to add more data. If you do not plan to add significant amounts of data to your instance, you can use up to 100% of the hard limit. Important: If any cluster in an instance exceeds the hard limit on the amount of storage per node, writes to all clusters in that instance will fail until you add nodes to each cluster that is over the limit. Also, if you try to remove nodes from a cluster, and the change would cause the cluster to exceed the hard limit on storage, Cloud Bigtable will deny the request. If you are using more than the recommended percentage of the storage limit, add nodes to the cluster. You can also delete existing data, but deleted data takes up more space, not less, until a compaction occurs.
  2. ajay1709 👍 4
    C : To Improve write. D : As more storage is required
  3. MaxNRG 👍 3
    D: In general, do not use more than 70% of the hard limit on total storage, so you have room to add more data. If you do not plan to add significant amounts of data to your instance, you can use up to 100% of the hard limit C: If this value is frequently at 100%, you might experience increased latency. Add nodes to the cluster to reduce the disk load percentage. The key visualizer metrics options, suggest other things other than increase the cluster size. https://cloud.google.com/bigtable/docs/monitoring-instance
  4. musumusu 👍 3
    i am feeling to go with B and D. In option C, when latency is low, latency can be low for write operation for other reason. but in option B, its showing clearly when write pressure more than 100. But why no one is talking about B
  5. sumanshu 👍 3
    Vote for C & D, Option B eliminated, as Row are well defined (as per question) - so no need of key-visualizer
  6. RoshanAshraf 👍 3
    Key visualizer is Metrics for Performance issues. Ruled out Storage and Write Operations ; C and D
  7. Ward010 👍 3
    "Well designed" indeed excludes A&B as these refer to heavy loads on specific key ranges
  8. daghayeghi 👍 3
    C, D: https://cloud.google.com/bigtable/docs/monitoring-instance
  9. Parandhaman_Margan 👍 3
    Correct answers are B (Write pressure) and C (latency)
  10. dabrat 👍 3
    https://cloud.google.com/bigtable/docs/monitoring-instance
  11. Fezo 👍 3
    Answer: CD https://cloud.google.com/bigtable/docs/scaling
  12. medeis_jar 👍 3
    as explained by MaxNRG
  13. StefanoG 👍 2
    from https://cloud.google.com/bigtable/docs/monitoring-instance Disk load - If this value is frequently at 100%, you might experience increased latency. Add nodes to the cluster to reduce the disk load percentage. Storage utilization (% max) - In general, do not use more than 70% of the hard limit on total storage, so you have room to add more data. If you do not plan to add significant amounts of data to your instance, you can use up to 100% of the hard limit.
  14. sergio6 👍 2
    D--> 70% is the the recommended percentage of the cluster's storage capacity that is being used, If you are using more than 70% storage limit, add nodes to the cluster https://cloud.google.com/bigtable/quotas#storage-per-node https://cloud.google.com/bigtable/docs/monitoring-instance#disk E--> 100 ms is an order of magnitude lower latency than Google claimed (<10ms) https://cloud.google.com/bigtable/docs/performance#typical-workloads
  15. TVH_Data_Engineer 👍 2
    The question focus is on writing. BC is correct. when the writing pressure is above 100, it is time to increase. same logic with C
  16. haroldbenites 👍 2
    A, D is correct A: https://cloud.google.com/bigtable/docs/keyvis-metrics D: It makes sense unlike the others
  17. squishy_fishy 👍 2
    Answer is C, D. B is not correct, because B is Key Visualizer, it means the row key needs re-design again.
  18. John_Pongthorn 👍 2
    Well-designed row key : A B are not nessary Write : CD both are involved in the question the most.
  19. norwayping 👍 2
    c.d https://cloud.google.com/bigtable/docs/scaling
  20. sumanshu 👍 2
    B, C , D - all three looks okay to me
  21. KokkiKumar 👍 2
    I am Voting for CD
  22. musumusu 👍 2
    why not B ?
  23. carbonscave 👍 1
    C is correct: For a pipeline actively writing data to Bigtable, a sustained increase in write latency indicates that the cluster's compute resources are struggling to keep up with the ingestion rate. Because the prompt specifies that the row keys are already well-designed (which rules out localized hotspots as the root cause), this global latency degradation is a strong indicator that the cluster is undersized for the workload and requires more nodes. D is correct: In Bigtable, total storage capacity is directly based on the number of nodes in the cluster. Google explicitly recommends keeping storage utilization below 70% of the maximum capacity so that there is always sufficient room to add more data. If utilization approaches or crosses this threshold, you must scale up the cluster to increase the storage limit.
  24. hauhau 👍 1
    BC D:you can just add node, not cluster The percentage of the cluster's storage capacity that is being used. The capacity is based on the number of nodes in your cluster.(https://cloud.google.com/bigtable/docs/monitoring-instance) After you create a Cloud Bigtable instance, you can update any of the following settings without any downtime: (The number of nodes in each cluster) https://cloud.google.com/bigtable/docs/modifying-instance
  25. sergio6 👍 1
    Adding nodes to the cluster In Bigtable scales linearly the performances both read and write https://cloud.google.com/bigtable/docs/performance#typical-workloads

Reader comments & corrections

No reader corrections yet — spotted an error or have a note? Share it below.

Log in to comment, report an error, or add a note about this question.

How to establish consistent BigQuery spend?

S-grade

Triggering Airflow DAGs Reactively in Private Cloud Composer

S-grade

How to resolve Cloud Composer 2 worker pod evictions and task failures?

S-grade

Ensuring Consistency in BigQuery Streaming Analytics

S-grade

← Back to PDE Study Guide