How to Monitor Demographic Fairness Across Two Bedrock Prompt Variants?
Implement evaluation systems for GenAI.
Implement monitoring systems for GenAI applications.
Implement responsible AI principles.
Answer
Correct answer: C — Use Amazon SageMaker Clarify to measure demographic bias for both Bedrock prompt variants, publish fairness metrics to CloudWatch, and alarm on >15% disparity.
A retail company has a generative AI (GenAI) product recommendation application that uses Amazon Bedrock. The application suggests products to customers based on browsing history and demographics. The company needs to implement fairness evaluation across multiple demographic groups to detect and measure bias in recommendations between two prompt approaches. The company wants to collect and monitor fairness metrics in real time. The company must receive an alert if the fairness metrics show a discrepancy of more than 15% between demographic groups. The company must receive weekly reports that compare the performance of the two prompt approaches. Which solution will meet these requirements with the LEAST custom development effort?
Community Insight
The question tests whether you map demographic fairness measurement to SageMaker Clarify instead of assuming Bedrock Guardrails content filters or offline model evaluation jobs provide group-bias metrics.
SageMaker Clarify is the AWS service purpose-built to measure bias across demographic groups, and publishing its fairness metrics to CloudWatch enables threshold alarms and weekly comparison reporting. Bedrock Guardrails and model evaluation jobs do not natively supply continuous demographic fairness metrics for the two prompt variants.
Learners often choose Bedrock Guardrails because InvocationsIntervened sounds like a fairness metric, but content filters intervene on unsafe content, not on disparate recommendation outcomes across demographic groups.
Community Discussion (25 comments)
blieskies
👍 1
Selected: C
I guessed B but see now why it's wrong. Guardrails can't score outputs on bias but this is what Sagemaker Clarify specifically does - therefore it's C
SSSam26
👍 1
Selected: D
Why D is Correct ? - Compare two prompt approaches : Bedrock Model Evaluation job natively supports comparing prompt variants - Fairness metrics across demographics : Model evaluation provides bias/fairness assessment built-in - Monitoring : Model invocation logging → CloudWatch - Weekly reports : Bedrock evaluation jobs can be scheduled; CloudWatch dashboards provide reporting https://docs.aws.amazon.com/bedrock/latest/userguide/evaluation.html
QuinnGT
👍 1
Selected: D
Without a doubt, D is the correct answer. It's the least amount of custom development and follows their best practice guides. Also, SageMaker Clarify was announced to be shut down 3 months ago, why would they include it in the exam as a recommended choice?
Sun01921
👍 1
Selected: A
C seems like a reasonable choice, however it is not an "least custom development" path
Chibuzo1
👍 1
Selected: D
D wins because it uses Bedrock's native evaluation jobs (prompt comparison), native invocation logging (real-time monitoring), and native CloudWatch alarms (threshold alerting) — all configured features, not built ones.
HarryNZ1982
👍 2
Selected: C
Okay gave my AWS Certified Generative AI Developer - Professional AIP-C01 and obtained the certification. Thank you for the assistance.
Gaparod
👍 1
Selected: C
D. Bedrock Model Evaluation Jobs Model evaluation jobs are useful for offline comparisons, but: They are not designed for continuous real-time monitoring InvocationsIntervened metrics are tied to Guardrails, not fairness They do not provide operational bias alerts by demographic group cleared my exam, and the updated questions on CertsTopic helped me feel more confident.
AWSCer1937
👍 1
Selected: C
D. Bedrock Model Evaluation Jobs Model evaluation jobs are useful for offline comparisons, but: They are not designed for continuous real-time monitoring InvocationsIntervened metrics are tied to Guardrails, not fairness They do not provide operational bias alerts by demographic group
teashoppe
👍 1
Selected: A
Clarify doesn't do real-time nor integrate with Bedrock. Hard to choose, since A is valid but is a do it yourself solution and they want the least customer development.
awstaro
👍 1
Selected: C
Has anyone taken this exam recently? How much of the material was covered?
AmyJoy
👍 1
Selected: C
Real-time monitoring is a hard requirement that Bedrock evaluation jobs don't fully address Production monitoring vs. evaluation jobs are different use cases The maturity and specialization of SageMaker Clarify for this exact scenario
jainparag1
👍 1
Selected: C
C is correct.
Thirukj
👍 3
Selected: C
Use Amazon SageMaker Clarify to evaluate fairness across demographic groups and publish metrics to CloudWatch for monitoring and alarms.
eesa
👍 1
Selected: C
Clarify puede generar análisis y reportes exportables, lo que encaja mejor con la comparación periódica entre dos enfoques de prompt. La parte de “latency” en la opción C no es estrictamente necesaria para fairness, pero sigue siendo la alternativa que más se alinea con el requerimiento general y con menos desarrollo a medida
hedykim
👍 1
Selected: C
Bedrock Guardrails is a tool for the 'safety' of content, not for measuring 'statistical fairness'.
xyztest
👍 1
Selected: D
Correct answer
eCosinus
👍 1
Selected: C
Clarify computes fairness metrics automatically Metrics can be published to CloudWatch CloudWatch alarms can trigger alerts when metrics exceed thresholds (like >15% discrepancy)
Piddi
👍 1
Selected: A
Option C sounds appealing because SageMaker Clarify is actually built for bias detection. However, combining Clarify bias metrics with Bedrock latency metrics in a "composite alarm" doesn't make logical sense — latency has nothing to do with fairness. Also, SageMaker Clarify is typically used for traditional ML model bias (pre/post-training), and integrating it with Bedrock GenAI outputs in real time would require significant custom work, which goes against the "least custom development effort" requirement.
Piddi
👍 1
Selected: A
Option D can be eliminated first. Bedrock model evaluation jobs are designed to compare models, not prompt variants for ongoing fairness monitoring. They're also batch jobs, not real-time monitoring solutions. The InvocationsIntervened metric is related to guardrails, not model evaluation jobs. Option B has issues too. Bedrock guardrails with content filters are designed to block harmful/toxic content, not to measure statistical fairness discrepancies across demographic groups. The InvocationsIntervened metric tells you how often a guardrail intervened, not whether recommendations are biased across demographics. This doesn't genuinely measure fairness in the way the question requires.
Piddi
👍 1
Selected: A
Option A is the most practical answer. Here's why: CloudWatch dashboards provide real-time monitoring EventBridge rules with Lambda functions can perform post-processing analysis on Bedrock responses as they happen, extracting fairness metrics per demographic group Custom CloudWatch metrics can capture the specific fairness discrepancies (the 15% threshold can be set as a CloudWatch alarm) Weekly reports can be generated from CloudWatch data While it does involve Lambda functions for post-processing, this is a well-established, relatively lightweight pattern using managed AWS services. The other options either misuse services (B, D) or propose illogical combinations (C).
Piddi
👍 2
Selected: B
Option B leverages native Amazon Bedrock capabilities end-to-end. Bedrock Prompt Management handles the two prompt variants, Bedrock Flows manages traffic allocation between them (enabling A/B comparison), and Bedrock Guardrails with content filters can be configured to monitor outputs for demographic fairness issues. CloudWatch alarms on the guardrail metrics (InvocationsIntervened with the GuardrailContentSource dimension) provide the real-time alerting when discrepancies exceed the threshold. Weekly reports can be generated from CloudWatch dashboards comparing the two variants. This approach uses managed services with minimal custom code.
apcertification
👍 1
Selected: D
D is the right answer
GiorgioGss
👍 8
Selected: C
A - Too Much Custom Development B - Guardrails Are Not Fairness Evaluation Tools D - Model Evaluation Jobs Are Not Continuous Monitoring https://docs.aws.amazon.com/sagemaker/latest/dg/clarify-measure-data-bias.html
nylon66
👍 1
Selected: D
D is collect.
Kennai
👍 1
Selected: A
no idea
Comments & Corrections
No comments yet — spotted an error or have a note? Share it below.
Expert Analysis
Why the Answer Is Correct
Amazon SageMaker Clarify is the AWS service purpose-built for measuring bias across demographic groups, so it directly satisfies the fairness-evaluation requirement rather than forcing custom scoring. Publishing Clarify's bias metrics into CloudWatch gives a managed metric store and alarm engine: a CloudWatch alarm can fire when the group disparity exceeds 15%, and dashboards can support weekly comparison of the two prompt variants. Option C still needs some plumbing from Bedrock outputs into Clarify, but that is far less custom development than building Lambda post-processing or inventing fairness metrics from Guardrails. The combination of Clarify for measurement and CloudWatch for alerting/reporting is the only option that names a true fairness metric source.Why the Other Options Are Wrong
Option A requires custom metrics plus EventBridge-triggered Lambda post-processing and custom fairness logic, which is exactly the custom development the question wants to minimize. Option B repurposes Bedrock Guardrails content filters and InvocationsIntervened, but those metrics report content-policy interventions, not demographic fairness or a 15% disparity between groups. Option D uses Bedrock model evaluation jobs for an offline prompt-variant comparison and then alarms on Guardrails InvocationsIntervened; model evaluation is not continuous monitoring, and InvocationsIntervened has no demographic-group fairness dimension. None of those alternatives can natively produce the required demographic fairness metric.Community Comment Notes
GiorgioGss captures the elimination logic by noting that A is "Too Much Custom Development" and that "Guardrails Are Not Fairness Evaluation Tools." blieskies makes the positive case for C, observing that "Guardrails can't score outputs on bias but this is what Sagemaker Clarify specifically does." Gaparod reinforces why D fails, saying model evaluation jobs "are not designed for continuous real-time monitoring" and that "InvocationsIntervened metrics are tied to Guardrails, not fairness." Some learners argued for D because Bedrock model evaluation natively compares prompt variants, but that only covers the offline comparison and not the continuous group-disparity alarm.Official Reference
Exam Strategy
When an option names a managed service that directly matches the required ML metric, such as SageMaker Clarify for bias/fairness, prefer it over custom Lambda analysis or repurposed guardrail metrics. Check whether each option can actually produce demographic-group fairness metrics; offline evaluation jobs and content filters cannot satisfy continuous group-disparity alerting.
Frequently Asked Questions
Why can't Bedrock Guardrails monitor demographic fairness?
Guardrails content filters and InvocationsIntervened measure policy interventions on unsafe content, not disparate recommendation outcomes across demographic groups; they cannot compute Clarify-style bias metrics.
Why is Bedrock Model Evaluation insufficient for real-time fairness alerts?
Model evaluation jobs provide offline comparisons of prompt variants; they do not natively emit continuous demographic-group fairness metrics or trigger >15% CloudWatch alarms.
Related Analysis
Practice All AIP-C01 Questions
Access 110 questions with complete answers and detailed explanations.
View Full AIP-C01 Practice Test →