Why Performance Bugs Appear Only in Production

PCA By islever Editorial Last updated

Ensuring the reliability of solutions in production (e.g., chaos engineering, penetration testing, and load testing) Evaluating quality control measures Deployment and release management

Short answer

C Increase the load on your test and staging environments so performance bugs surface before production.

Question

Your solution is producing performance bugs in production that you did not see in staging and test environments. You want to adjust your test and deployment
procedures to avoid this problem in the future.
What should you do?
  1. Deploy fewer changes to production
  2. Deploy smaller changes to production
  3. Increase the load on your test and staging environments Correct Answer
  4. Deploy changes to a small subset of users before rolling out to production

Key Takeaways

The question tests whether your test environment realistically mirrors production load; the trap is picking canary rollout (D), which limits blast radius but may never generate the traffic that triggers the performance bug.

This PCA question tests how to fix test and deployment procedures when performance defects escape to production. It establishes that raising load in test and staging environments (option C) reproduces production conditions and catches performance bugs before release.

Common mistake: Option D (canary rollout to a small subset of users) is the most-chosen wrong answer: it only narrows the impact and still puts real users at risk, and a small user subset typically does not reach the load level that exposes performance bugs.

Expert Analysis

Why the Answer Is Correct

The defect is a coverage gap: staging never ran at production load, so load-dependent performance bugs stayed invisible. Option C directly closes that gap by increasing the load on test and staging environments, which is exactly the load testing the Well-Architected reliability pillar recommends before release. It is preventive rather than reactive, because failures surface in QA instead of after deployment. Only option C changes the environment where the stated problem actually occurred.

Why the Other Options Are Wrong

Option A merely slows the release cadence; deploying fewer changes does not make hidden performance bugs detectable and just delays their eventual discovery. Option B shrinks the change set, which helps root-cause analysis after a failure but still leaves test environments unable to reproduce production-scale load. Option D is a canary rollout that limits blast radius, yet it deliberately sends only partial traffic, so load-dependent performance defects can stay hidden until full rollout and real users still receive the buggy build. It also contradicts the goal of avoiding the problem, because it detects rather than prevents exposure.

Community Comment Notes

ghitesh argued C is "the only one talking about doing changes in the test environment," and Sreekey agreed that the issue concerns existing code performance, so "increase the load in test environment." VedaSW pushed back that C covers only testing, contending "the only option that cover test and deployment is D," while tartar simply asserted "D is ok." certificatores supported that pick by linking the canary test pattern page, but francescogugliottagm undercut it by noting that with only part of traffic on a canary, "we don't see performance bugs in the canary release." Urban_Life concluded flatly "The answer is D," showing the community is genuinely split.

Official Reference

These links open the official vendor documentation in a new tab.

Exam Strategy

When a question says a bug appeared only in production, first identify which environment lacked fidelity (load, data, topology) and choose the option that restores that fidelity. Reserve canary and staged rollout answers for questions about limiting blast radius of changes, not for uncovering load-dependent defects.

Frequently Asked Questions

Why isn't canary deployment (D) the right answer for performance bugs?

A canary sends only a small share of traffic, so load-dependent performance bugs may never appear until full rollout. It limits blast radius but does not reproduce production load during testing.

Does increasing staging load address deployment procedures too?

The question permits adjusting either test or deployment procedures; fixing unrealistic test load removes the root cause before any deployment occurs.

More PCA FAQ →

Discussion 211 contributions

Compiled from public community threads and reader submissions, lightly edited for clarity. Handles and vote tallies are shown as recorded, and comments reflect the views of individual contributors.

From public community threads 30 of 211

  1. ghitesh 👍 88
    Question Statement: You want to adjust your test and deployment procedures to avoid this problem in the future So based on this, I think the option "C" is correct, since it is the only one talking about doing changes in the test environment.
  2. Eroc 👍 39
    A wouldn't prevent the bugs, it would just avoid them. B would help with root-cause analysis because it'd be a smaller change to review. C would test the performance of the system at its peak processing rates, so this assumes the bugs in production only occur because of usage. D would allow you to test the new code against smaller user sets to see if it occurs then, and if it still does you know it is not because of more user responses. So it's a tossup between C and D, D would be the cheaper/quicker answer so I'd choose D first then C if it's because of usage.
  3. VedaSW 👍 20
    C. Increase the load on your test and staging environments. As you have pointed out in "Question Statement", I do not see C covering "deployment procedures". Test and Staging environment is more on testing, but not about deployment procedure to production. So, the only option that cover test and deployment is D. (Yes, kind of unacceptable to have the users to do "testing", but we make it "ok" by calling it "canary deployment")
  4. RegisFTM 👍 19
    "Your solution is producing performance bugs in production..." - I don't see how "D" would help to detect performance bugs. - "C" looks more adequate.
  5. tartar 👍 19
    D is ok.
Show 25 more comments
  1. Sreekey 👍 18
    The question is about the performance of the existing Code that they did not detect in Test environments . This is not about new API release . In order to test the performance they should increase the load in test environment and hence answer C.
  2. certificatores 👍 12
    if you want to pass the test, trust tartar's answers. to support his claim, check this https://cloud.google.com/solutions/application-deployment-and-testing-strategies#canary_test_pattern Canary test pattern In canary testing, you partially roll out a change and then evaluate its performance against a baseline deployment, as the following diagram shows.
  3. francescogugliottagm 👍 10
    With canary deployment we expose the new version to a small portion of users. With this approach maybe we don't see performance bugs in the canary release, since we don't have the 100% of traffic on the canary. But when we migrate the 100% of traffic to the new release (previous canary) we can see performance bugs.
  4. gautam23 👍 9
    U should copy and paste TarTar... what a lame logic. Noone is perfect. Please understand. C looks like a better option to me as performance is the issue and customer is unable to find the same in Dev and Test scenarios. So scaling up might help. U may disagree and thats fine
  5. cetanx 👍 8
    Canary testing sounds like a good approach but performance is the problem here so therefore redirecting 10% of users to new release may not help you to test the performance thoroughly. So I'd vote for "C"
  6. chiar 👍 7
    I think, it`s C, because you test the performance increasing the load, in D you could do too, but don't say anythink about increasing load to check the performance
  7. clouddude 👍 7
    I'll go with C as it's the only solution that involves modifying the test environment.
  8. nitinz 👍 7
    D, canary rollout
  9. MahAli 👍 6 Answered C
    Canary deployment is perfect to test new feature but to do stress testing, I do development for 25 years, when we want to resolve performance and scalability issues we do stress and load testing in pre prod environment, something you can't do by exposing the new feature to subset of users.
  10. aviratna 👍 5
    C: Is the correct answer as problem statement is performance issue in production. D: Is not the right answer as its talking about releasing the code to small batch of users which only delay the performance issue from happening but will not avoid it. Application needs to be tested with load similar to production to identify the issue early in test & staging environment before pushing it to production.
  11. RitwickKumar 👍 5
    @eroc Looks like you missed the point here it is about the "performance" bugs which are related to the load. Apart from Option C there is no other way to test out the load prior to production role out. I would chose Option "C": Increase the load on your test and staging environments
  12. mpguard 👍 5
    Performance bugs are not necessarily linked to high load; you can have a performance bug even with 1 user. So while C sounds reasonable, the right answer is D (canary deployment)
  13. addy007 👍 5
    It should be D, think Canary Analysis, for similar issues. https://whatis.techtarget.com/definition/canary-canary-testing
  14. trainor 👍 5
    So I think C is better.
  15. jrisl1991 👍 4 Answered D
    I'm going for D. According to ChatGPT: D. The best approach to avoid performance bugs in production that weren't detected in staging and test environments is to gradually roll out changes to a small subset of users before deploying them to production. This way, you can identify and address any issues that may arise in a controlled environment before affecting all users. Why not C: Increasing the load on your test and staging environments, as suggested in option C, can be a valuable strategy for detecting certain types of performance issues related to scalability and load handling. However, it may not address all types of performance bugs or issues that are specific to the production environment. A and B seem a bit obviously wrong. A is incorrect because it would be incompatible with CI/CD, and deploying fewer changes may still introduce bugs regardless of the low frequency. B would help root causing, but may also introduce new bugs that are exclusive to production.
  16. jaxclain 👍 4 Answered C
    If you watched the Design Material from the PCA course, you will remember one of the videos where they recommend to increase the load on your test/staging to instead of just having a small test to because the more you test, obviously you will have more chances to find bugs that will not be found it you do a small test so once you go live and the load is high, of course the bug will be there but you can prevent by giving more load against the app during test/staging so yes, Option C is correct.
  17. Rathish 👍 4
    You want to adjust your test and deployment procedures to avoid this problem in the future: Instead of deploying at one go, deploy to few users and see the response and monitor the system, then proceed with entire roll-out. Its one of the best practise to test it for smaller number of users in production. The question talks about test and deployment procedure, not the test & stag environment, any changes to these environment, does not guarantee of occurence of this bug.
  18. Davidik79 👍 4 Answered D
    The answer must be "D". I assume that any architect roughly knows the number of requests per minute of their applications, the peak time, etc... Load tests would be done accordingly. Therefore, if you did not notice this in the test/staging envs, it means that something in production effects the performance issues, and the Canary deployment approach helps you to observe the application in the prod env before to hit the wider audience
  19. trainor 👍 4
    Deploying changes to a small subset of users wouldn't provide enough load to highlight performance bugs. Also, we'd notice the bug once in production (even thoug for a small number of users), not satisfying the requirement "to avoid this problem in the future". We'd be faster to fix them with a rollback, though.
  20. PeppaPig 👍 4
    I go with D. As a developer I can tell that In real life some performance issue only happen in production no matter how much load testing you perform in test and staging.
  21. nezih 👍 4
    This is about performance issue and testing the app with small group won't reveal the performance bug. D is not correct, I'll go with C
  22. Gobblegobble 👍 4
    D could detect functional issues not performance bug. Since question is related to performance issue correct answer is C
  23. certificationjjmmm 👍 4
    it is a performance issue. So the answer is C
  24. MaxNRG 👍 3
    D or C (prefer D) D – deploy changes to a small subset of users before rolling out to production. D is most likely, assuming that staging/test env provides an equivalent load to production. If this is not a heavy load case, then that’s smth else, an unaccounted bottleneck in production. In this case D would work perfect. C would be right if the problem is caused by heavy production traffic. In this case, an increase in test load will solve the problem. But, since Q asks for changes in “test and deployment” procedures, then D will be most probably answer – canary releases.
  25. YAS007 👍 3
    from the question, the problem is "performance bugs in production that you did not see in staging and test environments", therefore, I guess that there is more load in prod than in test and staging. the other part of the question said : "You want to adjust your test and deployment procedures to avoid this problem in the future." so, the action has to be taken in test environment so : A : incorrect (action taken in prod B: same C: correct, will try to simulate prod load + action taken in test env D: incorrect, because, bugs are not due to users,

Reader comments & corrections

No reader corrections yet — spotted an error or have a note? Share it below.

Log in to comment, report an error, or add a note about this question.

Multi-Regional vs Regional Cloud Storage for Global Software Downloads

S-grade

Surviving a Zone Outage With SQL Server on GCP

S-grade

Fixing BigQuery connection errors from a Compute Engine VM

S-grade

Moving 75 TB into Cloud Storage with Google-recommended practices

S-grade

Practice All PCA Questions

Access 81 questions with complete answers and detailed explanations.

View Full PCA Practice Test →

← Back to PCA Study Guide