Restoring Service Health After Canary Deployment Failure in Cloud Run
Short answer
B Configure traffic splitting to send 100% of the traffic to the previous revision to instantly revert the canary and restore service health.
Question
Key Takeaways
The question tests the distinction between incident mitigation (restoring service) and incident resolution (fixing the bug); the trap is choosing diagnostic tools over immediate rollback actions.
When a Cloud Run canary deployment causes an SLO breach, the fastest remediation is to revert traffic to the previous stable revision. Learn why immediate mitigation takes precedence over debugging in SRE practices.
Common mistake: Candidates often choose Option D (identify the cause using Cloud Trace) because they prioritize understanding the root cause. However, this delays restoration of service, violating the requirement to make the application healthy 'as quickly as possible.'
Expert Analysis
The core scenario involves a canary deployment on Cloud Run where an SLO violation has occurred. The primary goal is to restore application health as quickly as possible. In Site Reliability Engineering (SRE), the immediate response to a service degradation or SLO breach is often mitigation rather than root cause analysis. Mitigation aims to stop the bleeding and return the service to a known good state.
Option B, configuring traffic splitting to send 100% of traffic to the previous revision, is the fastest way to mitigate the issue. Since the previous revision was stable before the canary was introduced, reverting traffic flow instantly removes the potentially faulty new version from production without requiring any new builds or deployments. This aligns with the principle of minimizing mean time to recovery (MTTR).
Option A worsens the problem by fully committing to the failing version. Option C introduces unnecessary latency; creating a new revision requires building, testing, and deploying, which takes significantly longer than simply shifting traffic weights. Option D, using Cloud Trace for debugging, is a valid troubleshooting step but does not address the immediate need to restore service health. Debugging while the user experience is degraded violates the priority of incident response, which dictates stabilization before diagnosis.
Exam Strategy
In SRE-focused questions, always distinguish between 'mitigation' (stopping the impact) and 'resolution' (fixing the root cause). If the question asks for the 'fastest' way to restore health or stability, look for rollback or traffic reversal options first. Only select diagnostic steps if the question explicitly states the service is already stable or asks for root cause identification.
Frequently Asked Questions
Why not create a new revision (Option C) instead of reverting?
Is it ever okay to debug (Option D) during an active SLO breach?
Discussion 1 contributions
Compiled from public community threads and reader submissions, lightly edited for clarity. Handles and vote tallies are shown as recorded, and comments reflect the views of individual contributors.
From public community threads 1
-
BlackCoffee 👍 1 Answered By updating Cloud Run's traffic splitting configuration to route 100% of traffic back to the previous stable revision, you instantly remove the faulty canary version from the traffic pool.
Reader comments & corrections
No reader corrections yet — spotted an error or have a note? Share it below.
Related Analysis
Practice All PCDOE Questions
Access 75 questions with complete answers and detailed explanations.
View Full PCDOE Practice Test →