Which Solution Detects Hallucinations in a Bedrock Clinical Knowledge Base Before Release?
A healthcare company is implementing a clinical knowledge base application on Amazon Bed-rock that provides medical information to doctors. During testing, a quality team discovers that the generative AI (GenAI) model occasionally fabricates medical treatment recommendations that do not originate from the approved clinical guidelines. The quality team needs to implement a solution to detect the hallucinations before releasing the application. The solution must analyze model responses against verified medical information. The solution must identify semantic inconsistencies when users ask similar questions in different ways. Which solution will meet these requirements?
Community Insight
The scenario tests pre-release hallucination evaluation — only a validated reference dataset with Bedrock evaluation and response-consistency diffing can spot fabricated advice when users paraphrase the same clinical question.
A Bedrock clinical knowledge base that fabricates treatment advice must be validated before release against a reference dataset of approved question-answer pairs. This page shows why output diffing with Amazon Bedrock Guardrails plus Amazon Bedrock evaluation (B) catches semantic inconsistencies that keyword and pattern rules cannot.
Many pick C because Guardrails sounds like the hallucination fix, but its custom rules match explicit word and topic patterns, not semantic fabrication, and SageMaker Feature Store holds ML feature vectors, not a repository of clinical guidelines.
Community Discussion (1 comments)
Comments & Corrections
No comments yet — spotted an error or have a note? Share it below.
Expert Analysis
Why the Answer Is Correct
Option B satisfies all three stated requirements. A reference dataset of validated question-answer pairs built from approved clinical guidelines supplies the verified medical information against which model responses must be analyzed. Output diffing with Amazon Bedrock Guardrails for response consistency analysis compares answers to the same clinical question phrased in different ways, which directly addresses the semantic-inconsistency requirement. Amazon Bedrock evaluation then systematically detects factual inaccuracies before release, making this a pre-release validation pipeline rather than runtime monitoring.Why the Other Options Are Wrong
Option A depends on predefined keywords and Lambda comparisons against a DynamoDB table, which only match explicit strings and cannot judge semantic consistency when a question is paraphrased; it is also a runtime monitoring design, not a pre-release evaluation. Option C misuses two services: Guardrails custom rules are denied-topic and word filters that flag explicit patterns, not fabricated medical facts, and SageMaker Feature Store holds ML feature vectors, not a repository of clinical guideline documents. Option D's automatic model evaluation applies built-in metrics to a custom prompt dataset rather than scoring responses against validated clinical answers, and CloudWatch anomaly alarms with SNS notifications amount to post-release monitoring instead of analysis against verified information.Community Comment Notes
TNT87's comment selects B, reasoning that you should "Create a reference dataset with validated question-answer pairs from clinical guidelines" and pair it with Guardrails output diffing and Bedrock evaluation — a mapping that matches the three requirements one-to-one. No commenter defends the keyword-based, pattern-rule, or anomaly-alarm alternatives, so the community consensus aligns with this verdict.Official Reference
Exam Strategy
For pre-release hallucination questions, choose the option that builds a validated reference dataset and runs Bedrock evaluation with response-consistency diffing. Keyword filters, pattern rules, and CloudWatch alarms all inspect only surface text and can never detect semantic drift across paraphrased questions.
Frequently Asked Questions
Why isn't Amazon Bedrock Guardrails alone enough to block hallucinated medical content?
Guardrails word, topic, and pattern rules match explicit strings, not meaning; catching fabricated treatments requires evaluation against validated clinical references.
Why is SageMaker Feature Store the wrong place for verified clinical guidelines?
Feature Store is built for storing and serving ML feature vectors, not for maintaining a document repository of clinical guidelines used to validate model responses.
Related Analysis
Practice All AIP-C01 Questions
Access 110 questions with complete answers and detailed explanations.
View Full AIP-C01 Practice Test →